2024-03-27T06:19:14.416 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-27T06:19:14.416 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597 --name rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi --verbose -- /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597/orig.config.yaml 2024-03-27T06:19:14.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-27T06:19:14.575 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597 branch: wip-rishabh-testing-20240326.131558 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} email: null first_in_suite: false job_id: '7625597' kernel: client: sha1: distro flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240326.131558 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync rbytes: false syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240326.131558 sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6931 sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 sleep_before_teardown: 0 subset: 80/512 suite: fs suite_branch: wip-rishabh-testing-20240326.131558 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAUILPhzLsGfSCSvYbGY9vGUexePBRSZJwLFthe6CMejLETGGgCz1FeRyQmS+eUtRATNGjWItvaos78nvbBfeI8= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCJKtpwkSTQt5KObIqKSUm8UIwSn4UK9QuJ/DHO/yD6NoGV/kO9sVJXsRq2Xu9GEIvhiiI17ggozwL+yNj/n6Gw= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFkD99kVyPMz+D2zMUtsLdpGdVmlYfNkRbMbJu9wR0CUtYM+/HiOBrOwqbL1S8DhKeS6uNtjm2o3rZwjoItXJHQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-27_05:27:11 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-27T06:19:14.575 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa; will attempt to use it 2024-03-27T06:19:14.576 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks 2024-03-27T06:19:14.576 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-27T06:19:14.578 INFO:teuthology.task.internal:Checking packages... 2024-03-27T06:19:14.621 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '155268c4e432a12433aa833f174f9fe3b1016ae0' 2024-03-27T06:19:14.621 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-27T06:19:14.621 INFO:teuthology.packaging:ref: None 2024-03-27T06:19:14.621 INFO:teuthology.packaging:tag: None 2024-03-27T06:19:14.621 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240326.131558 2024-03-27T06:19:14.621 INFO:teuthology.packaging:sha1: 155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:19:14.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240326.131558 2024-03-27T06:19:14.767 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2478.g155268c4 2024-03-27T06:19:14.768 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-27T06:19:14.774 INFO:teuthology.task.internal:no buildpackages task found 2024-03-27T06:19:14.775 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-27T06:19:14.781 INFO:teuthology.task.internal:Saving configuration 2024-03-27T06:19:14.802 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-27T06:19:14.807 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T06:19:14.833 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 06:14:04.520064', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAUILPhzLsGfSCSvYbGY9vGUexePBRSZJwLFthe6CMejLETGGgCz1FeRyQmS+eUtRATNGjWItvaos78nvbBfeI8='} 2024-03-27T06:19:14.856 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 06:14:04.518078', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCJKtpwkSTQt5KObIqKSUm8UIwSn4UK9QuJ/DHO/yD6NoGV/kO9sVJXsRq2Xu9GEIvhiiI17ggozwL+yNj/n6Gw='} 2024-03-27T06:19:14.878 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 06:14:04.521888', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFkD99kVyPMz+D2zMUtsLdpGdVmlYfNkRbMbJu9wR0CUtYM+/HiOBrOwqbL1S8DhKeS6uNtjm2o3rZwjoItXJHQ='} 2024-03-27T06:19:14.878 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-27T06:19:14.884 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-27T06:19:14.884 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-27T06:19:14.884 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-27T06:19:14.884 INFO:teuthology.run_tasks:Running task console_log... 2024-03-27T06:19:14.988 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7e9a0ad160>, signals=[15]) 2024-03-27T06:19:14.988 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-27T06:19:14.994 INFO:teuthology.task.internal:Opening connections... 2024-03-27T06:19:14.994 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-03-27T06:19:14.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T06:19:15.103 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2024-03-27T06:19:15.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T06:19:15.194 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-03-27T06:19:15.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T06:19:15.282 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-27T06:19:15.289 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-03-27T06:19:15.307 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-03-27T06:19:15.308 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-03-27T06:19:15.365 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-27T06:19:15.366 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-27T06:19:15.367 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-27T06:19:15.367 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-03-27T06:19:15.392 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2024-03-27T06:19:15.410 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2024-03-27T06:19:15.410 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2024-03-27T06:19:15.468 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="CentOS Stream" 2024-03-27T06:19:15.468 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="9" 2024-03-27T06:19:15.468 INFO:teuthology.orchestra.run.smithi156.stdout:ID="centos" 2024-03-27T06:19:15.468 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="rhel fedora" 2024-03-27T06:19:15.468 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="9" 2024-03-27T06:19:15.468 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el9" 2024-03-27T06:19:15.468 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-27T06:19:15.468 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2024-03-27T06:19:15.468 INFO:teuthology.orchestra.run.smithi156.stdout:LOGO="fedora-logo-icon" 2024-03-27T06:19:15.469 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-27T06:19:15.469 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://centos.org/" 2024-03-27T06:19:15.469 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-27T06:19:15.469 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-27T06:19:15.469 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-27T06:19:15.469 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2024-03-27T06:19:15.491 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-03-27T06:19:15.509 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-03-27T06:19:15.510 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-03-27T06:19:15.567 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2024-03-27T06:19:15.568 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2024-03-27T06:19:15.568 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2024-03-27T06:19:15.568 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2024-03-27T06:19:15.568 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2024-03-27T06:19:15.568 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2024-03-27T06:19:15.569 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-27T06:19:15.569 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-03-27T06:19:15.569 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2024-03-27T06:19:15.569 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-27T06:19:15.569 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2024-03-27T06:19:15.569 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-27T06:19:15.569 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-27T06:19:15.569 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-27T06:19:15.570 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-03-27T06:19:15.591 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-27T06:19:15.603 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-27T06:19:15.609 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-27T06:19:15.609 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-03-27T06:19:15.612 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2024-03-27T06:19:15.614 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-03-27T06:19:15.630 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-27T06:19:15.636 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-27T06:19:15.636 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-03-27T06:19:15.669 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2024-03-27T06:19:15.671 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-03-27T06:19:15.943 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-27T06:19:16.026 INFO:teuthology.run_tasks:Running task kernel... 2024-03-27T06:19:16.043 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-27T06:19:16.043 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-27T06:19:16.043 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-27T06:19:16.044 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-27T06:19:16.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:19:16.062 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-03-27T06:19:16.120 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-361.el9.x86_64 2024-03-27T06:19:16.120 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-361.el9.x86_64 2024-03-27T06:19:16.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-03-27T06:19:18.258 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-27T06:19:19.683 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - BaseOS 7.8 MB/s | 2.0 MB 00:00 2024-03-27T06:19:21.791 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-03-27T06:19:43.365 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 945 kB/s | 19 MB 00:20 2024-03-27T06:19:47.543 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-27T06:19:49.274 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-03-27T06:23:04.462 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 109 kB/s | 21 MB 03:15 2024-03-27T06:23:09.678 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-03-27T06:23:11.909 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-27T06:23:11.910 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-27T06:23:12.004 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-03-27T06:23:12.014 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-03-27T06:23:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-03-27T06:23:12.014 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-03-27T06:23:12.014 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-03-27T06:23:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-03-27T06:23:12.015 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-03-27T06:23:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-03-27T06:23:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-03-27T06:23:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-03-27T06:23:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:23:12.015 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-03-27T06:23:12.015 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-03-27T06:23:12.015 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-03-27T06:23:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:23:12.016 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 76 M 2024-03-27T06:23:12.016 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 123 M 2024-03-27T06:23:12.016 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-03-27T06:23:12.188 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 491 kB/s | 66 kB 00:00 2024-03-27T06:23:15.163 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-core-5.14.0-432.el9.x86_64.rpm 4.8 MB/s | 15 MB 00:03 2024-03-27T06:23:16.080 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-core-5.14.0-432.el9.x86_6 7.1 MB/s | 27 MB 00:03 2024-03-27T06:23:16.455 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 7.6 MB/s | 33 MB 00:04 2024-03-27T06:23:16.455 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-03-27T06:23:16.455 INFO:teuthology.orchestra.run.smithi016.stdout:Total 17 MB/s | 76 MB 00:04 2024-03-27T06:23:17.150 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-03-27T06:23:17.574 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-03-27T06:23:17.574 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-03-27T06:23:18.635 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-03-27T06:23:18.635 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-03-27T06:23:21.234 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-03-27T06:23:21.879 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-03-27T06:23:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T06:23:24.856 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T06:23:24.992 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T06:23:30.065 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T06:23:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-03-27T06:23:35.202 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T06:24:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T06:24:05.956 INFO:teuthology.orchestra.run.smithi016.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-27T06:24:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:24:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-03-27T06:24:06.385 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-03-27T06:24:06.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-03-27T06:24:06.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-27T06:24:06.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-27T06:24:06.804 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-27T06:24:06.804 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:24:06.805 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-03-27T06:24:06.805 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-432.el9.x86_64 2024-03-27T06:24:06.805 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-03-27T06:24:06.805 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-03-27T06:24:06.805 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-03-27T06:24:06.805 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:24:06.805 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-03-27T06:24:07.136 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-03-27T06:24:07.235 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-27T06:24:07.236 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-27T06:24:07.236 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-03-27T06:24:07.236 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-27T06:24:07.236 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-27T06:24:07.236 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-27T06:24:07.236 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-03-27T06:24:07.236 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-03-27T06:24:07.319 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-27T06:24:07.320 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-27T06:24:07.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-03-27T06:24:07.356 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-03-27T06:24:07.357 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-03-27T06:24:07.357 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-03-27T06:24:07.357 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 68.el9 2024-03-27T06:24:07.357 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-03-27T06:24:07.357 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-27T06:24:07.357 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-03-27T06:24:07.357 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8264254 2024-03-27T06:24:07.358 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-03-27T06:24:07.358 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-27T06:24:07.358 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-27T06:24:07.358 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-27T06:24:07.358 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-27T06:24:07.358 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-03-27T06:24:07.358 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-03-27T06:24:07.359 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-27T06:24:07.359 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-03-27T06:24:07.359 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-03-27T06:24:07.359 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:24:07.359 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-27T06:24:07.359 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-27T06:24:07.359 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-27T06:24:07.360 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-03-27T06:24:07.360 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:24:07.360 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-03-27T06:24:07.361 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-27T06:24:07.362 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-432.el9.x86_64 2024-03-27T06:24:07.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-27T06:24:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-03-27T06:24:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-27T06:24:09.469 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-03-27T06:24:09.471 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-03-27T06:24:09.526 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.6gWDadxJs9 2024-03-27T06:24:09.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6gWDadxJs9 2024-03-27T06:24:09.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.6gWDadxJs9 2024-03-27T06:24:09.747 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.6gWDadxJs9 is 6KB 2024-03-27T06:24:09.796 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.6gWDadxJs9 2024-03-27T06:24:09.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-03-27T06:24:09.879 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-27T06:24:09.879 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-27T06:24:09.879 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-27T06:24:09.879 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-03-27T06:24:09.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-03-27T06:24:10.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-03-27T06:24:40.077 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-27T06:24:40.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-03-27T06:24:40.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T06:25:40.081 DEBUG:teuthology.orchestra.remote:timed out 2024-03-27T06:25:49.083 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-03-27T06:25:49.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T06:25:49.484 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-03-27T06:25:50.049 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-03-27T06:25:50.049 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-432.el9.x86_64"... 2024-03-27T06:25:50.049 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-03-27T06:25:50.063 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-432.el9.x86_64 2024-03-27T06:25:50.063 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-03-27T06:25:50.063 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-27T06:25:50.064 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-27T06:25:51.064 DEBUG:teuthology.parallel:result is None 2024-03-27T06:25:51.065 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-27T06:25:51.072 INFO:teuthology.task.internal:Creating test directory... 2024-03-27T06:25:51.073 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-27T06:25:51.075 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-27T06:25:51.078 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-27T06:25:51.098 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-27T06:25:51.104 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-27T06:25:51.110 INFO:teuthology.task.internal:Creating archive directory... 2024-03-27T06:25:51.110 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-27T06:25:51.140 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-27T06:25:51.142 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-27T06:25:51.321 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-27T06:25:51.328 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-27T06:25:51.328 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-27T06:25:51.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:25:51.345 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-27T06:25:51.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:25:51.363 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-27T06:25:51.365 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-27T06:25:51.388 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-27T06:25:51.444 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T06:25:51.465 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T06:25:51.485 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T06:25:51.506 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T06:25:51.560 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T06:25:51.695 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-27T06:25:51.697 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-27T06:25:51.704 INFO:teuthology.task.internal:Configuring sudo... 2024-03-27T06:25:51.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-27T06:25:51.707 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-27T06:25:51.709 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-27T06:25:51.741 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-27T06:25:51.750 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-27T06:25:51.751 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-27T06:25:51.775 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-27T06:25:51.780 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-27T06:25:51.797 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T06:25:51.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T06:25:51.926 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T06:25:52.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T06:25:52.050 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:25:52.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-27T06:25:52.118 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T06:25:52.162 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T06:25:52.228 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T06:25:52.278 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T06:25:52.319 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-27T06:25:52.319 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-27T06:25:52.389 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T06:25:52.433 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-27T06:25:52.493 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T06:25:52.535 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-27T06:25:52.581 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-27T06:25:52.581 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-27T06:25:52.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-03-27T06:25:52.654 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2024-03-27T06:25:52.656 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-03-27T06:25:52.698 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T06:25:52.699 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T06:25:52.733 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T06:25:53.144 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-27T06:25:53.152 INFO:teuthology.task.internal:Starting timer... 2024-03-27T06:25:53.153 INFO:teuthology.run_tasks:Running task pcp... 2024-03-27T06:25:53.213 INFO:teuthology.run_tasks:Running task selinux... 2024-03-27T06:25:53.222 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-27T06:25:53.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-03-27T06:25:53.283 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-03-27T06:25:53.286 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2024-03-27T06:25:53.340 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2024-03-27T06:25:53.342 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2024-03-27T06:25:53.402 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2024-03-27T06:25:53.402 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-27T06:25:53.402 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-03-27T06:25:53.420 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-03-27T06:25:53.421 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2024-03-27T06:25:53.442 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2024-03-27T06:25:53.442 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2024-03-27T06:25:53.467 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2024-03-27T06:25:53.467 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2024-03-27T06:25:53.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T06:25:53.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:25:53.496 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T06:25:53.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:25:53.527 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T06:25:53.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:25:53.555 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-27T06:25:53.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-03-27T06:25:53.598 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2024-03-27T06:25:53.644 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2024-03-27T06:25:53.676 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-27T06:25:53.687 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-27T06:25:53.694 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-27T06:25:53.819 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-27T06:25:53.834 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-27T06:25:53.835 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi156.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-27T06:30:27.768 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2024-03-27T06:30:27.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-03-27T06:30:27.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T06:30:27.861 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-03-27T06:30:27.945 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-03-27T06:30:27.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-03-27T06:30:27.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T06:30:28.033 DEBUG:teuthology.orchestra.run.smithi156:> true 2024-03-27T06:30:28.101 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2024-03-27T06:30:28.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-27T06:30:28.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-27T06:30:28.231 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-03-27T06:30:28.298 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-27T06:30:28.299 INFO:teuthology.run_tasks:Running task clock... 2024-03-27T06:30:28.311 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-27T06:30:28.311 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-27T06:30:28.312 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T06:30:28.314 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-27T06:30:28.315 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T06:30:28.317 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-27T06:30:28.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T06:30:28.347 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-27T06:30:28.350 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-27T06:30:28.366 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-27T06:30:28.368 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-27T06:30:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-27T06:30:28.390 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-27T06:30:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-03-27T06:30:28.399 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2024-03-27T06:30:28.411 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-03-27T06:30:28.413 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2024-03-27T06:30:28.420 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2024-03-27T06:30:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-27T06:30:28.430 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-27T06:30:28.434 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2024-03-27T06:30:28.445 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-27T06:30:28.447 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-27T06:30:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-27T06:30:28.467 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-27T06:30:28.491 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-03-27T06:30:28.494 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T06:30:28.495 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-03-27T06:30:28.495 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.495 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.495 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.495 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.500 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-03-27T06:30:28.503 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T06:30:28.503 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-03-27T06:30:28.504 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.504 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.504 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.504 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.511 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-03-27T06:30:28.514 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T06:30:28.514 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-03-27T06:30:28.514 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.514 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.515 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.515 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:30:28.516 INFO:teuthology.run_tasks:Running task install... 2024-03-27T06:30:28.525 DEBUG:teuthology.task.install:project ceph 2024-03-27T06:30:28.525 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0'}} 2024-03-27T06:30:28.525 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0'} 2024-03-27T06:30:28.525 INFO:teuthology.task.install:Using flavor: default 2024-03-27T06:30:28.531 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-27T06:30:28.531 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-27T06:30:28.531 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0', 'tag': None, 'wait_for_package': False} 2024-03-27T06:30:28.532 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:30:28.534 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0', 'tag': None, 'wait_for_package': False} 2024-03-27T06:30:28.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:30:28.535 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0', 'tag': None, 'wait_for_package': False} 2024-03-27T06:30:28.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:30:28.684 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/ 2024-03-27T06:30:28.685 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2478.g155268c4 2024-03-27T06:30:28.688 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/ 2024-03-27T06:30:28.688 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2478.g155268c4 2024-03-27T06:30:28.745 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/ 2024-03-27T06:30:28.746 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2478.g155268c4 2024-03-27T06:30:28.794 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-27T06:30:28.794 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-27T06:30:28.794 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-27T06:30:28.822 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-27T06:30:28.823 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/155268c4e432a12433aa833f174f9fe3b1016ae0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-27T06:30:28.856 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-27T06:30:28.856 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:30:28.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-27T06:30:28.884 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-27T06:30:28.884 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/155268c4e432a12433aa833f174f9fe3b1016ae0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-27T06:30:28.891 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-27T06:30:28.912 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-27T06:30:28.912 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-27T06:30:28.912 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-27T06:30:28.941 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-27T06:30:28.942 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/155268c4e432a12433aa833f174f9fe3b1016ae0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-27T06:30:28.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-27T06:30:28.970 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-27T06:30:29.011 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-27T06:30:29.031 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-27T06:30:29.036 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2024-03-27T06:30:29.038 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2024-03-27T06:30:29.091 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-27T06:30:29.098 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-03-27T06:30:29.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-03-27T06:30:29.157 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2024-03-27T06:30:29.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2024-03-27T06:30:29.271 INFO:teuthology.orchestra.run.smithi170.stdout:74 files removed 2024-03-27T06:30:29.299 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-27T06:30:29.353 INFO:teuthology.orchestra.run.smithi016.stdout:74 files removed 2024-03-27T06:30:29.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-27T06:30:29.394 INFO:teuthology.orchestra.run.smithi156.stdout:74 files removed 2024-03-27T06:30:29.422 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-27T06:30:30.041 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 183 kB/s | 83 kB 00:00 2024-03-27T06:30:30.082 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 211 kB/s | 83 kB 00:00 2024-03-27T06:30:30.110 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 210 kB/s | 83 kB 00:00 2024-03-27T06:30:30.358 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-03-27T06:30:30.448 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-03-27T06:30:30.461 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-03-27T06:30:30.667 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 6.8 kB/s | 1.9 kB 00:00 2024-03-27T06:30:30.766 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 6.8 kB/s | 1.9 kB 00:00 2024-03-27T06:30:30.794 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-03-27T06:30:31.061 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-27T06:30:31.221 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-27T06:30:31.238 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-27T06:30:32.437 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2024-03-27T06:30:32.605 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2024-03-27T06:30:32.644 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.0 MB 00:00 2024-03-27T06:30:33.638 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 9.2 MB/s | 8.0 MB 00:00 2024-03-27T06:30:34.134 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.0 MB 00:01 2024-03-27T06:30:35.812 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-03-27T06:30:38.385 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 19 MB 00:03 2024-03-27T06:30:42.462 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - BaseOS 863 kB/s | 8.0 MB 00:09 2024-03-27T06:30:45.544 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.1 MB 00:03 2024-03-27T06:30:46.138 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 19 MB 00:02 2024-03-27T06:30:47.153 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-03-27T06:30:47.653 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 752 kB/s | 6.1 MB 00:08 2024-03-27T06:30:49.079 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-03-27T06:30:50.105 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-27T06:30:50.389 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-27T06:30:50.613 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 21 MB 00:03 2024-03-27T06:30:51.614 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-03-27T06:30:53.316 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-27T06:30:55.372 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-03-27T06:30:55.763 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-27T06:30:57.759 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T06:30:57.760 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T06:30:57.775 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-27T06:30:57.776 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-27T06:30:57.777 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-27T06:30:57.778 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-27T06:30:57.781 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-27T06:30:57.784 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-27T06:30:57.785 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-27T06:30:57.866 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-03-27T06:30:57.882 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================ 2024-03-27T06:30:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================ 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:19.0.0-2478.g155268c4.el9 ceph 6.6 k 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.4 M 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 890 k 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2478.g155268c4.el9 ceph 145 k 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:19.0.0-2478.g155268c4.el9 ceph 1.5 M 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 141 k 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 3.5 M 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 7.4 M 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 49 k 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 9.8 M 2024-03-27T06:30:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:19.0.0-2478.g155268c4.el9 ceph 48 M 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 264 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 762 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 221 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell x86_64 2:19.0.0-2478.g155268c4.el9 ceph 27 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 18 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 32 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 746 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 127 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-27T06:30:57.884 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-27T06:30:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-27T06:30:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-27T06:30:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-27T06:30:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-27T06:30:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:19.0.0-2478.g155268c4.el9 ceph 162 k 2024-03-27T06:30:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:19.0.0-2478.g155268c4.el9 ceph 320 k 2024-03-27T06:30:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 298 k 2024-03-27T06:30:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 99 k 2024-03-27T06:30:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 85 k 2024-03-27T06:30:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T06:30:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 170 k 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-27T06:30:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-27T06:30:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-27T06:30:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-27T06:30:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-27T06:30:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.4 M 2024-03-27T06:30:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T06:30:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-27T06:30:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-27T06:30:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-27T06:30:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-27T06:30:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 21 M 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 34 k 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:19.0.0-2478.g155268c4.el9 ceph 2.4 M 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 249 k 2024-03-27T06:30:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:19.0.0-2478.g155268c4.el9 ceph 4.7 M 2024-03-27T06:30:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 17 M 2024-03-27T06:30:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 16 k 2024-03-27T06:30:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:19.0.0-2478.g155268c4.el9 ceph 25 k 2024-03-27T06:30:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-27T06:30:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-27T06:30:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-27T06:30:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-27T06:30:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-27T06:30:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-27T06:30:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:19.0.0-2478.g155268c4.el9 ceph 163 k 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 501 k 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.1 M 2024-03-27T06:30:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-27T06:30:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 45 k 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 134 k 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-27T06:30:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-27T06:30:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-27T06:30:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-27T06:30:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-27T06:30:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-27T06:30:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-27T06:30:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-27T06:30:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-27T06:30:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-27T06:30:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-27T06:30:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-27T06:30:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-27T06:30:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-27T06:30:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout:Installing weak dependencies: 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:30:57.897 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-03-27T06:30:57.898 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================ 2024-03-27T06:30:57.898 INFO:teuthology.orchestra.run.smithi170.stdout:Install 150 Packages 2024-03-27T06:30:57.898 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 24 Packages 2024-03-27T06:30:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:30:57.898 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 227 M 2024-03-27T06:30:57.898 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-03-27T06:30:58.254 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T06:30:58.255 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T06:30:58.270 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-27T06:30:58.272 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-27T06:30:58.273 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-27T06:30:58.274 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-27T06:30:58.276 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-27T06:30:58.279 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-27T06:30:58.280 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-27T06:30:58.349 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-27T06:30:58.366 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-03-27T06:30:58.379 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2024-03-27T06:30:58.379 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-03-27T06:30:58.379 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2024-03-27T06:30:58.379 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-03-27T06:30:58.379 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-27T06:30:58.379 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.0.0-2478.g155268c4.el9 ceph 6.6 k 2024-03-27T06:30:58.379 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.4 M 2024-03-27T06:30:58.379 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 890 k 2024-03-27T06:30:58.379 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2478.g155268c4.el9 ceph 145 k 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.0.0-2478.g155268c4.el9 ceph 1.5 M 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 141 k 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 3.5 M 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 7.4 M 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 49 k 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 9.8 M 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.0.0-2478.g155268c4.el9 ceph 48 M 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 264 k 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 762 k 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 221 k 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.0.0-2478.g155268c4.el9 ceph 27 k 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 18 k 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-27T06:30:58.380 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 32 k 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 746 k 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 127 k 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.0.0-2478.g155268c4.el9 ceph 162 k 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.0.0-2478.g155268c4.el9 ceph 320 k 2024-03-27T06:30:58.381 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 298 k 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 99 k 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 85 k 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 170 k 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-27T06:30:58.382 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.4 M 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-27T06:30:58.383 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 21 M 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 34 k 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.0.0-2478.g155268c4.el9 ceph 2.4 M 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 249 k 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.0.0-2478.g155268c4.el9 ceph 4.7 M 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 17 M 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 16 k 2024-03-27T06:30:58.384 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.0.0-2478.g155268c4.el9 ceph 25 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.0.0-2478.g155268c4.el9 ceph 163 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-27T06:30:58.385 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 501 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.1 M 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-27T06:30:58.386 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 45 k 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 134 k 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-27T06:30:58.387 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-27T06:30:58.388 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-27T06:30:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-27T06:30:58.390 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-27T06:30:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout:Install 150 Packages 2024-03-27T06:30:58.392 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 24 Packages 2024-03-27T06:30:58.393 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:30:58.393 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 227 M 2024-03-27T06:30:58.393 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-03-27T06:30:59.109 INFO:teuthology.orchestra.run.smithi170.stdout:(1/174): ceph-19.0.0-2478.g155268c4.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-03-27T06:30:59.406 INFO:teuthology.orchestra.run.smithi016.stdout:(1/174): ceph-19.0.0-2478.g155268c4.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-03-27T06:31:00.048 INFO:teuthology.orchestra.run.smithi170.stdout:(2/174): ceph-fuse-19.0.0-2478.g155268c4.el9.x8 946 kB/s | 890 kB 00:00 2024-03-27T06:31:00.198 INFO:teuthology.orchestra.run.smithi016.stdout:(2/174): ceph-fuse-19.0.0-2478.g155268c4.el9.x8 1.1 MB/s | 890 kB 00:00 2024-03-27T06:31:00.265 INFO:teuthology.orchestra.run.smithi170.stdout:(3/174): ceph-immutable-object-cache-19.0.0-247 668 kB/s | 145 kB 00:00 2024-03-27T06:31:00.407 INFO:teuthology.orchestra.run.smithi016.stdout:(3/174): ceph-immutable-object-cache-19.0.0-247 695 kB/s | 145 kB 00:00 2024-03-27T06:31:00.731 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T06:31:00.732 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-27T06:31:00.747 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-27T06:31:00.748 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-27T06:31:00.749 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-27T06:31:00.750 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-27T06:31:00.752 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-27T06:31:00.755 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-27T06:31:00.756 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-27T06:31:00.837 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2024-03-27T06:31:00.849 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2024-03-27T06:31:00.850 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2024-03-27T06:31:00.850 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2024-03-27T06:31:00.850 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2024-03-27T06:31:00.850 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-27T06:31:00.850 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:19.0.0-2478.g155268c4.el9 ceph 6.6 k 2024-03-27T06:31:00.850 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.4 M 2024-03-27T06:31:00.850 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 890 k 2024-03-27T06:31:00.850 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2478.g155268c4.el9 ceph 145 k 2024-03-27T06:31:00.850 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:19.0.0-2478.g155268c4.el9 ceph 1.5 M 2024-03-27T06:31:00.851 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 141 k 2024-03-27T06:31:00.851 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 3.5 M 2024-03-27T06:31:00.851 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 7.4 M 2024-03-27T06:31:00.851 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 49 k 2024-03-27T06:31:00.851 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 9.8 M 2024-03-27T06:31:00.851 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:19.0.0-2478.g155268c4.el9 ceph 48 M 2024-03-27T06:31:00.851 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 264 k 2024-03-27T06:31:00.851 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 762 k 2024-03-27T06:31:00.851 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 221 k 2024-03-27T06:31:00.851 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-shell x86_64 2:19.0.0-2478.g155268c4.el9 ceph 27 k 2024-03-27T06:31:00.851 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 18 k 2024-03-27T06:31:00.852 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-27T06:31:00.852 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-27T06:31:00.852 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-27T06:31:00.852 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 32 k 2024-03-27T06:31:00.852 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 746 k 2024-03-27T06:31:00.852 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:19.0.0-2478.g155268c4.el9 ceph 127 k 2024-03-27T06:31:00.852 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-27T06:31:00.852 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-27T06:31:00.852 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-27T06:31:00.852 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-27T06:31:00.852 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-27T06:31:00.853 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-27T06:31:00.853 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:19.0.0-2478.g155268c4.el9 ceph 162 k 2024-03-27T06:31:00.853 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:19.0.0-2478.g155268c4.el9 ceph 320 k 2024-03-27T06:31:00.853 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 298 k 2024-03-27T06:31:00.853 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:19.0.0-2478.g155268c4.el9 ceph 99 k 2024-03-27T06:31:00.853 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 85 k 2024-03-27T06:31:00.853 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T06:31:00.853 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 170 k 2024-03-27T06:31:00.853 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-27T06:31:00.853 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-27T06:31:00.854 INFO:teuthology.orchestra.run.smithi156.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-27T06:31:00.855 INFO:teuthology.orchestra.run.smithi156.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-27T06:31:00.855 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.4 M 2024-03-27T06:31:00.855 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 3.1 M 2024-03-27T06:31:00.855 INFO:teuthology.orchestra.run.smithi156.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-27T06:31:00.855 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-27T06:31:00.855 INFO:teuthology.orchestra.run.smithi156.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-27T06:31:00.855 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-27T06:31:00.855 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-27T06:31:00.855 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-27T06:31:00.855 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-27T06:31:00.855 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 21 M 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 34 k 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:19.0.0-2478.g155268c4.el9 ceph 2.4 M 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 249 k 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:19.0.0-2478.g155268c4.el9 ceph 4.7 M 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:19.0.0-2478.g155268c4.el9 ceph 17 M 2024-03-27T06:31:00.856 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2478.g155268c4.el9 ceph-noarch 16 k 2024-03-27T06:31:00.857 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:19.0.0-2478.g155268c4.el9 ceph 25 k 2024-03-27T06:31:00.857 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-27T06:31:00.857 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-27T06:31:00.857 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-27T06:31:00.857 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-27T06:31:00.857 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-27T06:31:00.857 INFO:teuthology.orchestra.run.smithi156.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-27T06:31:00.857 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-27T06:31:00.858 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-27T06:31:00.858 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-27T06:31:00.858 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:19.0.0-2478.g155268c4.el9 ceph 163 k 2024-03-27T06:31:00.858 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-27T06:31:00.858 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-27T06:31:00.858 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-27T06:31:00.858 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-27T06:31:00.858 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-27T06:31:00.858 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 501 k 2024-03-27T06:31:00.858 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-27T06:31:00.858 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:19.0.0-2478.g155268c4.el9 ceph 5.1 M 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-27T06:31:00.859 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-27T06:31:00.860 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-27T06:31:00.860 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-27T06:31:00.860 INFO:teuthology.orchestra.run.smithi156.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-27T06:31:00.860 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-27T06:31:00.860 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-27T06:31:00.860 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-27T06:31:00.860 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:19.0.0-2478.g155268c4.el9 ceph 45 k 2024-03-27T06:31:00.860 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:19.0.0-2478.g155268c4.el9 ceph 134 k 2024-03-27T06:31:00.860 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-27T06:31:00.860 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-27T06:31:00.861 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-27T06:31:00.861 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-27T06:31:00.861 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-27T06:31:00.861 INFO:teuthology.orchestra.run.smithi156.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-27T06:31:00.861 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-27T06:31:00.861 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-27T06:31:00.861 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-27T06:31:00.861 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-27T06:31:00.861 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-27T06:31:00.861 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-27T06:31:00.861 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-27T06:31:00.862 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-27T06:31:00.862 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-27T06:31:00.862 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-27T06:31:00.862 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-27T06:31:00.862 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-27T06:31:00.862 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-27T06:31:00.862 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-27T06:31:00.862 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-27T06:31:00.862 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-27T06:31:00.862 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-27T06:31:00.862 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-27T06:31:00.863 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-27T06:31:00.864 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-27T06:31:00.865 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-27T06:31:00.865 INFO:teuthology.orchestra.run.smithi156.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-27T06:31:00.865 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-27T06:31:00.865 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-27T06:31:00.865 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-27T06:31:00.865 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-27T06:31:00.865 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-27T06:31:00.865 INFO:teuthology.orchestra.run.smithi156.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-27T06:31:00.865 INFO:teuthology.orchestra.run.smithi156.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-27T06:31:00.865 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-27T06:31:00.865 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-27T06:31:00.866 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-27T06:31:00.866 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-27T06:31:00.866 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-27T06:31:00.866 INFO:teuthology.orchestra.run.smithi156.stdout:Installing weak dependencies: 2024-03-27T06:31:00.866 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-27T06:31:00.866 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:31:00.866 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2024-03-27T06:31:00.866 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2024-03-27T06:31:00.866 INFO:teuthology.orchestra.run.smithi156.stdout:Install 150 Packages 2024-03-27T06:31:00.867 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 24 Packages 2024-03-27T06:31:00.867 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:31:00.867 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 227 M 2024-03-27T06:31:00.867 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2024-03-27T06:31:01.290 INFO:teuthology.orchestra.run.smithi016.stdout:(4/174): ceph-common-19.0.0-2478.g155268c4.el9. 9.0 MB/s | 21 MB 00:02 2024-03-27T06:31:01.524 INFO:teuthology.orchestra.run.smithi016.stdout:(5/174): ceph-mgr-19.0.0-2478.g155268c4.el9.x86 6.4 MB/s | 1.5 MB 00:00 2024-03-27T06:31:01.816 INFO:teuthology.orchestra.run.smithi016.stdout:(6/174): ceph-mds-19.0.0-2478.g155268c4.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-03-27T06:31:01.940 INFO:teuthology.orchestra.run.smithi170.stdout:(4/174): ceph-mds-19.0.0-2478.g155268c4.el9.x86 1.4 MB/s | 2.4 MB 00:01 2024-03-27T06:31:01.983 INFO:teuthology.orchestra.run.smithi016.stdout:(7/174): ceph-base-19.0.0-2478.g155268c4.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-03-27T06:31:02.008 INFO:teuthology.orchestra.run.smithi156.stdout:(1/174): ceph-19.0.0-2478.g155268c4.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-03-27T06:31:02.066 INFO:teuthology.orchestra.run.smithi016.stdout:(8/174): ceph-mon-19.0.0-2478.g155268c4.el9.x86 8.7 MB/s | 4.7 MB 00:00 2024-03-27T06:31:02.115 INFO:teuthology.orchestra.run.smithi170.stdout:(5/174): ceph-base-19.0.0-2478.g155268c4.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-03-27T06:31:02.141 INFO:teuthology.orchestra.run.smithi016.stdout:(9/174): ceph-selinux-19.0.0-2478.g155268c4.el9 335 kB/s | 25 kB 00:00 2024-03-27T06:31:02.550 INFO:teuthology.orchestra.run.smithi156.stdout:(2/174): ceph-fuse-19.0.0-2478.g155268c4.el9.x8 1.6 MB/s | 890 kB 00:00 2024-03-27T06:31:02.642 INFO:teuthology.orchestra.run.smithi156.stdout:(3/174): ceph-immutable-object-cache-19.0.0-247 1.5 MB/s | 145 kB 00:00 2024-03-27T06:31:03.049 INFO:teuthology.orchestra.run.smithi170.stdout:(6/174): ceph-mgr-19.0.0-2478.g155268c4.el9.x86 1.3 MB/s | 1.5 MB 00:01 2024-03-27T06:31:03.492 INFO:teuthology.orchestra.run.smithi156.stdout:(4/174): ceph-mds-19.0.0-2478.g155268c4.el9.x86 2.8 MB/s | 2.4 MB 00:00 2024-03-27T06:31:03.859 INFO:teuthology.orchestra.run.smithi156.stdout:(5/174): ceph-mgr-19.0.0-2478.g155268c4.el9.x86 4.1 MB/s | 1.5 MB 00:00 2024-03-27T06:31:04.226 INFO:teuthology.orchestra.run.smithi156.stdout:(6/174): ceph-base-19.0.0-2478.g155268c4.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-03-27T06:31:04.727 INFO:teuthology.orchestra.run.smithi156.stdout:(7/174): ceph-mon-19.0.0-2478.g155268c4.el9.x86 5.4 MB/s | 4.7 MB 00:00 2024-03-27T06:31:05.162 INFO:teuthology.orchestra.run.smithi170.stdout:(7/174): ceph-mon-19.0.0-2478.g155268c4.el9.x86 1.5 MB/s | 4.7 MB 00:03 2024-03-27T06:31:05.658 INFO:teuthology.orchestra.run.smithi016.stdout:(10/174): ceph-radosgw-19.0.0-2478.g155268c4.el 2.7 MB/s | 9.8 MB 00:03 2024-03-27T06:31:05.825 INFO:teuthology.orchestra.run.smithi016.stdout:(11/174): cephfs-mirror-19.0.0-2478.g155268c4.e 1.3 MB/s | 221 kB 00:00 2024-03-27T06:31:05.917 INFO:teuthology.orchestra.run.smithi016.stdout:(12/174): cephfs-shell-19.0.0-2478.g155268c4.el 295 kB/s | 27 kB 00:00 2024-03-27T06:31:06.009 INFO:teuthology.orchestra.run.smithi016.stdout:(13/174): libcephfs-devel-19.0.0-2478.g155268c4 347 kB/s | 32 kB 00:00 2024-03-27T06:31:06.359 INFO:teuthology.orchestra.run.smithi016.stdout:(14/174): libcephfs2-19.0.0-2478.g155268c4.el9. 2.1 MB/s | 746 kB 00:00 2024-03-27T06:31:06.435 INFO:teuthology.orchestra.run.smithi156.stdout:(8/174): ceph-radosgw-19.0.0-2478.g155268c4.el9 5.8 MB/s | 9.8 MB 00:01 2024-03-27T06:31:06.443 INFO:teuthology.orchestra.run.smithi016.stdout:(15/174): libcephsqlite-19.0.0-2478.g155268c4.e 1.9 MB/s | 163 kB 00:00 2024-03-27T06:31:06.502 INFO:teuthology.orchestra.run.smithi156.stdout:(9/174): ceph-selinux-19.0.0-2478.g155268c4.el9 378 kB/s | 25 kB 00:00 2024-03-27T06:31:06.518 INFO:teuthology.orchestra.run.smithi016.stdout:(16/174): librados-devel-19.0.0-2478.g155268c4. 1.7 MB/s | 127 kB 00:00 2024-03-27T06:31:06.777 INFO:teuthology.orchestra.run.smithi016.stdout:(17/174): libradosstriper1-19.0.0-2478.g155268c 1.9 MB/s | 501 kB 00:00 2024-03-27T06:31:10.060 INFO:teuthology.orchestra.run.smithi016.stdout:(18/174): librgw2-19.0.0-2478.g155268c4.el9.x86 1.6 MB/s | 5.1 MB 00:03 2024-03-27T06:31:10.185 INFO:teuthology.orchestra.run.smithi016.stdout:(19/174): python3-ceph-argparse-19.0.0-2478.g15 361 kB/s | 45 kB 00:00 2024-03-27T06:31:10.357 INFO:teuthology.orchestra.run.smithi170.stdout:(8/174): ceph-radosgw-19.0.0-2478.g155268c4.el9 1.9 MB/s | 9.8 MB 00:05 2024-03-27T06:31:10.360 INFO:teuthology.orchestra.run.smithi016.stdout:(20/174): python3-ceph-common-19.0.0-2478.g1552 766 kB/s | 134 kB 00:00 2024-03-27T06:31:10.449 INFO:teuthology.orchestra.run.smithi170.stdout:(9/174): ceph-selinux-19.0.0-2478.g155268c4.el9 275 kB/s | 25 kB 00:00 2024-03-27T06:31:10.894 INFO:teuthology.orchestra.run.smithi016.stdout:(21/174): ceph-test-19.0.0-2478.g155268c4.el9.x 5.4 MB/s | 48 MB 00:08 2024-03-27T06:31:10.919 INFO:teuthology.orchestra.run.smithi016.stdout:(22/174): python3-cephfs-19.0.0-2478.g155268c4. 290 kB/s | 162 kB 00:00 2024-03-27T06:31:11.103 INFO:teuthology.orchestra.run.smithi016.stdout:(23/174): ceph-osd-19.0.0-2478.g155268c4.el9.x8 1.8 MB/s | 17 MB 00:09 2024-03-27T06:31:11.109 INFO:teuthology.orchestra.run.smithi156.stdout:(10/174): ceph-common-19.0.0-2478.g155268c4.el9 2.2 MB/s | 21 MB 00:09 2024-03-27T06:31:11.128 INFO:teuthology.orchestra.run.smithi016.stdout:(24/174): python3-rados-19.0.0-2478.g155268c4.e 1.3 MB/s | 320 kB 00:00 2024-03-27T06:31:11.162 INFO:teuthology.orchestra.run.smithi016.stdout:(25/174): python3-rbd-19.0.0-2478.g155268c4.el9 1.2 MB/s | 298 kB 00:00 2024-03-27T06:31:11.204 INFO:teuthology.orchestra.run.smithi016.stdout:(26/174): rbd-fuse-19.0.0-2478.g155268c4.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-27T06:31:11.237 INFO:teuthology.orchestra.run.smithi016.stdout:(27/174): python3-rgw-19.0.0-2478.g155268c4.el9 743 kB/s | 99 kB 00:00 2024-03-27T06:31:11.277 INFO:teuthology.orchestra.run.smithi156.stdout:(11/174): ceph-osd-19.0.0-2478.g155268c4.el9.x8 2.4 MB/s | 17 MB 00:07 2024-03-27T06:31:11.287 INFO:teuthology.orchestra.run.smithi016.stdout:(28/174): rbd-nbd-19.0.0-2478.g155268c4.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-27T06:31:11.310 INFO:teuthology.orchestra.run.smithi156.stdout:(12/174): cephfs-mirror-19.0.0-2478.g155268c4.e 1.1 MB/s | 221 kB 00:00 2024-03-27T06:31:11.329 INFO:teuthology.orchestra.run.smithi016.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2478.g 369 kB/s | 34 kB 00:00 2024-03-27T06:31:11.344 INFO:teuthology.orchestra.run.smithi156.stdout:(13/174): cephfs-shell-19.0.0-2478.g155268c4.el 406 kB/s | 27 kB 00:00 2024-03-27T06:31:11.363 INFO:teuthology.orchestra.run.smithi016.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2478.g155268c 1.8 MB/s | 141 kB 00:00 2024-03-27T06:31:11.369 INFO:teuthology.orchestra.run.smithi156.stdout:(14/174): libcephfs-devel-19.0.0-2478.g155268c4 542 kB/s | 32 kB 00:00 2024-03-27T06:31:11.477 INFO:teuthology.orchestra.run.smithi156.stdout:(15/174): libcephsqlite-19.0.0-2478.g155268c4.e 1.5 MB/s | 163 kB 00:00 2024-03-27T06:31:11.578 INFO:teuthology.orchestra.run.smithi156.stdout:(16/174): librados-devel-19.0.0-2478.g155268c4. 1.2 MB/s | 127 kB 00:00 2024-03-27T06:31:11.645 INFO:teuthology.orchestra.run.smithi156.stdout:(17/174): libcephfs2-19.0.0-2478.g155268c4.el9. 2.4 MB/s | 746 kB 00:00 2024-03-27T06:31:11.795 INFO:teuthology.orchestra.run.smithi156.stdout:(18/174): libradosstriper1-19.0.0-2478.g155268c 2.3 MB/s | 501 kB 00:00 2024-03-27T06:31:11.887 INFO:teuthology.orchestra.run.smithi156.stdout:(19/174): python3-ceph-argparse-19.0.0-2478.g15 494 kB/s | 45 kB 00:00 2024-03-27T06:31:11.970 INFO:teuthology.orchestra.run.smithi156.stdout:(20/174): python3-ceph-common-19.0.0-2478.g1552 1.6 MB/s | 134 kB 00:00 2024-03-27T06:31:12.070 INFO:teuthology.orchestra.run.smithi156.stdout:(21/174): python3-cephfs-19.0.0-2478.g155268c4. 1.6 MB/s | 162 kB 00:00 2024-03-27T06:31:12.238 INFO:teuthology.orchestra.run.smithi156.stdout:(22/174): python3-rados-19.0.0-2478.g155268c4.e 1.9 MB/s | 320 kB 00:00 2024-03-27T06:31:12.396 INFO:teuthology.orchestra.run.smithi156.stdout:(23/174): python3-rbd-19.0.0-2478.g155268c4.el9 1.8 MB/s | 298 kB 00:00 2024-03-27T06:31:12.424 INFO:teuthology.orchestra.run.smithi170.stdout:(10/174): ceph-common-19.0.0-2478.g155268c4.el9 1.5 MB/s | 21 MB 00:13 2024-03-27T06:31:12.521 INFO:teuthology.orchestra.run.smithi156.stdout:(24/174): python3-rgw-19.0.0-2478.g155268c4.el9 797 kB/s | 99 kB 00:00 2024-03-27T06:31:12.605 INFO:teuthology.orchestra.run.smithi156.stdout:(25/174): rbd-fuse-19.0.0-2478.g155268c4.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-27T06:31:12.699 INFO:teuthology.orchestra.run.smithi170.stdout:(11/174): cephfs-mirror-19.0.0-2478.g155268c4.e 806 kB/s | 221 kB 00:00 2024-03-27T06:31:12.833 INFO:teuthology.orchestra.run.smithi170.stdout:(12/174): cephfs-shell-19.0.0-2478.g155268c4.el 203 kB/s | 27 kB 00:00 2024-03-27T06:31:12.958 INFO:teuthology.orchestra.run.smithi170.stdout:(13/174): libcephfs-devel-19.0.0-2478.g155268c4 254 kB/s | 32 kB 00:00 2024-03-27T06:31:12.972 INFO:teuthology.orchestra.run.smithi016.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 4.6 MB/s | 7.4 MB 00:01 2024-03-27T06:31:13.038 INFO:teuthology.orchestra.run.smithi016.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2478.g15 3.7 MB/s | 249 kB 00:00 2024-03-27T06:31:13.130 INFO:teuthology.orchestra.run.smithi016.stdout:(33/174): rbd-mirror-19.0.0-2478.g155268c4.el9. 1.6 MB/s | 3.1 MB 00:01 2024-03-27T06:31:13.147 INFO:teuthology.orchestra.run.smithi016.stdout:(34/174): ceph-mgr-rook-19.0.0-2478.g155268c4.e 455 kB/s | 49 kB 00:00 2024-03-27T06:31:13.172 INFO:teuthology.orchestra.run.smithi016.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2478.g1 394 kB/s | 16 kB 00:00 2024-03-27T06:31:13.214 INFO:teuthology.orchestra.run.smithi016.stdout:(36/174): ceph-volume-19.0.0-2478.g155268c4.el9 3.9 MB/s | 264 kB 00:00 2024-03-27T06:31:13.264 INFO:teuthology.orchestra.run.smithi156.stdout:(26/174): ceph-test-19.0.0-2478.g155268c4.el9.x 7.0 MB/s | 48 MB 00:06 2024-03-27T06:31:13.265 INFO:teuthology.orchestra.run.smithi016.stdout:(37/174): cephfs-top-19.0.0-2478.g155268c4.el9. 366 kB/s | 18 kB 00:00 2024-03-27T06:31:13.356 INFO:teuthology.orchestra.run.smithi156.stdout:(27/174): librgw2-19.0.0-2478.g155268c4.el9.x86 3.0 MB/s | 5.1 MB 00:01 2024-03-27T06:31:13.381 INFO:teuthology.orchestra.run.smithi156.stdout:(28/174): rbd-nbd-19.0.0-2478.g155268c4.el9.x86 1.4 MB/s | 170 kB 00:00 2024-03-27T06:31:13.414 INFO:teuthology.orchestra.run.smithi156.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2478.g 583 kB/s | 34 kB 00:00 2024-03-27T06:31:13.448 INFO:teuthology.orchestra.run.smithi156.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2478.g155268c 2.1 MB/s | 141 kB 00:00 2024-03-27T06:31:13.507 INFO:teuthology.orchestra.run.smithi016.stdout:(38/174): cephadm-19.0.0-2478.g155268c4.el9.noa 2.2 MB/s | 762 kB 00:00 2024-03-27T06:31:13.508 INFO:teuthology.orchestra.run.smithi170.stdout:(14/174): libcephfs2-19.0.0-2478.g155268c4.el9. 1.3 MB/s | 746 kB 00:00 2024-03-27T06:31:13.557 INFO:teuthology.orchestra.run.smithi156.stdout:(31/174): rbd-mirror-19.0.0-2478.g155268c4.el9. 3.2 MB/s | 3.1 MB 00:00 2024-03-27T06:31:13.574 INFO:teuthology.orchestra.run.smithi016.stdout:(39/174): ceph-mgr-dashboard-19.0.0-2478.g15526 1.6 MB/s | 3.5 MB 00:02 2024-03-27T06:31:13.615 INFO:teuthology.orchestra.run.smithi016.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 662 kB 00:00 2024-03-27T06:31:13.690 INFO:teuthology.orchestra.run.smithi156.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2478.g15 1.8 MB/s | 249 kB 00:00 2024-03-27T06:31:13.721 INFO:teuthology.orchestra.run.smithi170.stdout:(15/174): libcephsqlite-19.0.0-2478.g155268c4.e 768 kB/s | 163 kB 00:00 2024-03-27T06:31:13.749 INFO:teuthology.orchestra.run.smithi156.stdout:(33/174): ceph-mgr-rook-19.0.0-2478.g155268c4.e 849 kB/s | 49 kB 00:00 2024-03-27T06:31:13.757 INFO:teuthology.orchestra.run.smithi016.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 920 kB 00:00 2024-03-27T06:31:13.774 INFO:teuthology.orchestra.run.smithi016.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 529 kB/s | 106 kB 00:00 2024-03-27T06:31:13.800 INFO:teuthology.orchestra.run.smithi016.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 1.6 MB/s | 307 kB 00:00 2024-03-27T06:31:13.817 INFO:teuthology.orchestra.run.smithi170.stdout:(16/174): librados-devel-19.0.0-2478.g155268c4. 1.3 MB/s | 127 kB 00:00 2024-03-27T06:31:13.824 INFO:teuthology.orchestra.run.smithi016.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 477 kB/s | 32 kB 00:00 2024-03-27T06:31:13.832 INFO:teuthology.orchestra.run.smithi156.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2478.g1 198 kB/s | 16 kB 00:00 2024-03-27T06:31:13.966 INFO:teuthology.orchestra.run.smithi156.stdout:(35/174): ceph-volume-19.0.0-2478.g155268c4.el9 1.9 MB/s | 264 kB 00:00 2024-03-27T06:31:14.059 INFO:teuthology.orchestra.run.smithi170.stdout:(17/174): ceph-osd-19.0.0-2478.g155268c4.el9.x8 1.5 MB/s | 17 MB 00:11 2024-03-27T06:31:14.092 INFO:teuthology.orchestra.run.smithi016.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-03-27T06:31:14.117 INFO:teuthology.orchestra.run.smithi016.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 59 kB/s | 17 kB 00:00 2024-03-27T06:31:14.151 INFO:teuthology.orchestra.run.smithi016.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 31 kB/s | 10 kB 00:00 2024-03-27T06:31:14.176 INFO:teuthology.orchestra.run.smithi016.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2024-03-27T06:31:14.184 INFO:teuthology.orchestra.run.smithi170.stdout:(18/174): libradosstriper1-19.0.0-2478.g155268c 1.3 MB/s | 501 kB 00:00 2024-03-27T06:31:14.201 INFO:teuthology.orchestra.run.smithi016.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 891 kB/s | 44 kB 00:00 2024-03-27T06:31:14.208 INFO:teuthology.orchestra.run.smithi156.stdout:(36/174): ceph-mgr-dashboard-19.0.0-2478.g15526 4.5 MB/s | 3.5 MB 00:00 2024-03-27T06:31:14.241 INFO:teuthology.orchestra.run.smithi156.stdout:(37/174): cephadm-19.0.0-2478.g155268c4.el9.noa 2.7 MB/s | 762 kB 00:00 2024-03-27T06:31:14.243 INFO:teuthology.orchestra.run.smithi016.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2024-03-27T06:31:14.251 INFO:teuthology.orchestra.run.smithi170.stdout:(19/174): python3-ceph-argparse-19.0.0-2478.g15 677 kB/s | 45 kB 00:00 2024-03-27T06:31:14.267 INFO:teuthology.orchestra.run.smithi156.stdout:(38/174): cephfs-top-19.0.0-2478.g155268c4.el9. 312 kB/s | 18 kB 00:00 2024-03-27T06:31:14.285 INFO:teuthology.orchestra.run.smithi016.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 578 kB 00:00 2024-03-27T06:31:14.310 INFO:teuthology.orchestra.run.smithi016.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 428 kB/s | 46 kB 00:00 2024-03-27T06:31:14.335 INFO:teuthology.orchestra.run.smithi016.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-03-27T06:31:14.350 INFO:teuthology.orchestra.run.smithi156.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 8.2 MB/s | 7.4 MB 00:00 2024-03-27T06:31:14.360 INFO:teuthology.orchestra.run.smithi170.stdout:(20/174): python3-ceph-common-19.0.0-2478.g1552 1.2 MB/s | 134 kB 00:00 2024-03-27T06:31:14.361 INFO:teuthology.orchestra.run.smithi016.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-27T06:31:14.386 INFO:teuthology.orchestra.run.smithi016.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-27T06:31:14.411 INFO:teuthology.orchestra.run.smithi016.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 456 kB/s | 34 kB 00:00 2024-03-27T06:31:14.477 INFO:teuthology.orchestra.run.smithi170.stdout:(21/174): python3-cephfs-19.0.0-2478.g155268c4. 1.4 MB/s | 162 kB 00:00 2024-03-27T06:31:14.526 INFO:teuthology.orchestra.run.smithi156.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 856 kB/s | 106 kB 00:00 2024-03-27T06:31:14.559 INFO:teuthology.orchestra.run.smithi156.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 662 kB 00:00 2024-03-27T06:31:14.593 INFO:teuthology.orchestra.run.smithi156.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 920 kB 00:00 2024-03-27T06:31:14.618 INFO:teuthology.orchestra.run.smithi156.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-27T06:31:14.627 INFO:teuthology.orchestra.run.smithi170.stdout:(22/174): python3-rados-19.0.0-2478.g155268c4.e 2.1 MB/s | 320 kB 00:00 2024-03-27T06:31:14.643 INFO:teuthology.orchestra.run.smithi156.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 382 kB/s | 32 kB 00:00 2024-03-27T06:31:14.670 INFO:teuthology.orchestra.run.smithi016.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-03-27T06:31:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 275 kB/s | 78 kB 00:00 2024-03-27T06:31:14.718 INFO:teuthology.orchestra.run.smithi156.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 171 kB/s | 17 kB 00:00 2024-03-27T06:31:14.729 INFO:teuthology.orchestra.run.smithi016.stdout:(59/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.2 MB/s | 1.0 MB 00:00 2024-03-27T06:31:14.735 INFO:teuthology.orchestra.run.smithi156.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 109 kB/s | 10 kB 00:00 2024-03-27T06:31:14.794 INFO:teuthology.orchestra.run.smithi170.stdout:(23/174): python3-rbd-19.0.0-2478.g155268c4.el9 1.7 MB/s | 298 kB 00:00 2024-03-27T06:31:14.810 INFO:teuthology.orchestra.run.smithi156.stdout:(47/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-03-27T06:31:14.836 INFO:teuthology.orchestra.run.smithi156.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 159 kB 00:00 2024-03-27T06:31:14.861 INFO:teuthology.orchestra.run.smithi170.stdout:(24/174): python3-rgw-19.0.0-2478.g155268c4.el9 1.5 MB/s | 99 kB 00:00 2024-03-27T06:31:14.861 INFO:teuthology.orchestra.run.smithi156.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 356 kB/s | 44 kB 00:00 2024-03-27T06:31:14.879 INFO:teuthology.orchestra.run.smithi016.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 134 kB/s | 24 kB 00:00 2024-03-27T06:31:14.886 INFO:teuthology.orchestra.run.smithi156.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-27T06:31:14.911 INFO:teuthology.orchestra.run.smithi156.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-03-27T06:31:14.928 INFO:teuthology.orchestra.run.smithi156.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 707 kB/s | 46 kB 00:00 2024-03-27T06:31:14.936 INFO:teuthology.orchestra.run.smithi170.stdout:(25/174): rbd-fuse-19.0.0-2478.g155268c4.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-27T06:31:14.953 INFO:teuthology.orchestra.run.smithi156.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-03-27T06:31:14.962 INFO:teuthology.orchestra.run.smithi016.stdout:(61/174): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-03-27T06:31:14.979 INFO:teuthology.orchestra.run.smithi156.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-27T06:31:14.979 INFO:teuthology.orchestra.run.smithi016.stdout:(62/174): python3-jsonpointer-2.0-4.el9.noarch. 73 kB/s | 18 kB 00:00 2024-03-27T06:31:15.004 INFO:teuthology.orchestra.run.smithi156.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-27T06:31:15.004 INFO:teuthology.orchestra.run.smithi016.stdout:(63/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2024-03-27T06:31:15.021 INFO:teuthology.orchestra.run.smithi016.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 763 kB/s | 32 kB 00:00 2024-03-27T06:31:15.029 INFO:teuthology.orchestra.run.smithi156.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 457 kB/s | 34 kB 00:00 2024-03-27T06:31:15.146 INFO:teuthology.orchestra.run.smithi156.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-03-27T06:31:15.214 INFO:teuthology.orchestra.run.smithi016.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-03-27T06:31:15.267 INFO:teuthology.orchestra.run.smithi156.stdout:(58/174): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-03-27T06:31:15.288 INFO:teuthology.orchestra.run.smithi156.stdout:(59/174): python3-attrs-20.3.0-7.el9.noarch.rpm 302 kB/s | 78 kB 00:00 2024-03-27T06:31:15.347 INFO:teuthology.orchestra.run.smithi016.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 637 kB/s | 217 kB 00:00 2024-03-27T06:31:15.372 INFO:teuthology.orchestra.run.smithi016.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 500 kB/s | 174 kB 00:00 2024-03-27T06:31:15.397 INFO:teuthology.orchestra.run.smithi156.stdout:(60/174): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-03-27T06:31:15.397 INFO:teuthology.orchestra.run.smithi016.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 376 kB/s | 69 kB 00:00 2024-03-27T06:31:15.414 INFO:teuthology.orchestra.run.smithi016.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 993 kB/s | 41 kB 00:00 2024-03-27T06:31:15.422 INFO:teuthology.orchestra.run.smithi156.stdout:(61/174): python3-jsonpatch-1.21-16.el9.noarch. 186 kB/s | 24 kB 00:00 2024-03-27T06:31:15.440 INFO:teuthology.orchestra.run.smithi016.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-27T06:31:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:(62/174): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-03-27T06:31:15.465 INFO:teuthology.orchestra.run.smithi016.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-27T06:31:15.481 INFO:teuthology.orchestra.run.smithi156.stdout:(63/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-27T06:31:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-27T06:31:15.507 INFO:teuthology.orchestra.run.smithi016.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 712 kB/s | 47 kB 00:00 2024-03-27T06:31:15.515 INFO:teuthology.orchestra.run.smithi156.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 473 kB/s | 32 kB 00:00 2024-03-27T06:31:15.524 INFO:teuthology.orchestra.run.smithi016.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 737 kB/s | 43 kB 00:00 2024-03-27T06:31:15.632 INFO:teuthology.orchestra.run.smithi156.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-03-27T06:31:15.641 INFO:teuthology.orchestra.run.smithi016.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 290 kB/s | 39 kB 00:00 2024-03-27T06:31:15.657 INFO:teuthology.orchestra.run.smithi156.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-03-27T06:31:15.682 INFO:teuthology.orchestra.run.smithi156.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-27T06:31:15.699 INFO:teuthology.orchestra.run.smithi156.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-27T06:31:15.728 INFO:teuthology.orchestra.run.smithi170.stdout:(26/174): librgw2-19.0.0-2478.g155268c4.el9.x86 3.1 MB/s | 5.1 MB 00:01 2024-03-27T06:31:15.735 INFO:teuthology.orchestra.run.smithi156.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 788 kB/s | 41 kB 00:00 2024-03-27T06:31:15.761 INFO:teuthology.orchestra.run.smithi156.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 3.0 MB/s | 238 kB 00:00 2024-03-27T06:31:15.786 INFO:teuthology.orchestra.run.smithi156.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-03-27T06:31:15.791 INFO:teuthology.orchestra.run.smithi016.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 153 kB/s | 41 kB 00:00 2024-03-27T06:31:15.811 INFO:teuthology.orchestra.run.smithi156.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-03-27T06:31:15.845 INFO:teuthology.orchestra.run.smithi156.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 815 kB/s | 47 kB 00:00 2024-03-27T06:31:15.870 INFO:teuthology.orchestra.run.smithi170.stdout:(27/174): rbd-nbd-19.0.0-2478.g155268c4.el9.x86 1.2 MB/s | 170 kB 00:00 2024-03-27T06:31:15.870 INFO:teuthology.orchestra.run.smithi156.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 513 kB/s | 43 kB 00:00 2024-03-27T06:31:15.900 INFO:teuthology.orchestra.run.smithi016.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2024-03-27T06:31:15.978 INFO:teuthology.orchestra.run.smithi170.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2478.g 313 kB/s | 34 kB 00:00 2024-03-27T06:31:16.070 INFO:teuthology.orchestra.run.smithi170.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2478.g155268c 1.5 MB/s | 141 kB 00:00 2024-03-27T06:31:16.154 INFO:teuthology.orchestra.run.smithi156.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 125 kB/s | 39 kB 00:00 2024-03-27T06:31:16.187 INFO:teuthology.orchestra.run.smithi170.stdout:(30/174): rbd-mirror-19.0.0-2478.g155268c4.el9. 2.5 MB/s | 3.1 MB 00:01 2024-03-27T06:31:16.225 INFO:teuthology.orchestra.run.smithi016.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-03-27T06:31:16.242 INFO:teuthology.orchestra.run.smithi016.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 125 kB/s | 56 kB 00:00 2024-03-27T06:31:16.259 INFO:teuthology.orchestra.run.smithi016.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 175 kB/s | 63 kB 00:00 2024-03-27T06:31:16.262 INFO:teuthology.orchestra.run.smithi156.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 105 kB/s | 41 kB 00:00 2024-03-27T06:31:16.284 INFO:teuthology.orchestra.run.smithi016.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 755 kB/s | 44 kB 00:00 2024-03-27T06:31:16.309 INFO:teuthology.orchestra.run.smithi016.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-27T06:31:16.326 INFO:teuthology.orchestra.run.smithi016.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-27T06:31:16.360 INFO:teuthology.orchestra.run.smithi016.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-27T06:31:16.377 INFO:teuthology.orchestra.run.smithi016.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-27T06:31:16.402 INFO:teuthology.orchestra.run.smithi016.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 124 kB 00:00 2024-03-27T06:31:16.419 INFO:teuthology.orchestra.run.smithi016.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-27T06:31:16.421 INFO:teuthology.orchestra.run.smithi156.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-03-27T06:31:16.446 INFO:teuthology.orchestra.run.smithi156.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2024-03-27T06:31:16.471 INFO:teuthology.orchestra.run.smithi156.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 274 kB/s | 56 kB 00:00 2024-03-27T06:31:16.486 INFO:teuthology.orchestra.run.smithi016.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-03-27T06:31:16.496 INFO:teuthology.orchestra.run.smithi156.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 840 kB/s | 63 kB 00:00 2024-03-27T06:31:16.511 INFO:teuthology.orchestra.run.smithi016.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-03-27T06:31:16.522 INFO:teuthology.orchestra.run.smithi156.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-03-27T06:31:16.545 INFO:teuthology.orchestra.run.smithi016.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 5.5 MB/s | 803 kB 00:00 2024-03-27T06:31:16.547 INFO:teuthology.orchestra.run.smithi156.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-27T06:31:16.572 INFO:teuthology.orchestra.run.smithi156.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-27T06:31:16.586 INFO:teuthology.orchestra.run.smithi016.stdout:(91/174): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-03-27T06:31:16.612 INFO:teuthology.orchestra.run.smithi016.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-03-27T06:31:16.614 INFO:teuthology.orchestra.run.smithi156.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-27T06:31:16.639 INFO:teuthology.orchestra.run.smithi156.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-27T06:31:16.646 INFO:teuthology.orchestra.run.smithi016.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 253 kB/s | 25 kB 00:00 2024-03-27T06:31:16.664 INFO:teuthology.orchestra.run.smithi156.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-27T06:31:16.690 INFO:teuthology.orchestra.run.smithi156.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-27T06:31:16.815 INFO:teuthology.orchestra.run.smithi156.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-27T06:31:16.840 INFO:teuthology.orchestra.run.smithi156.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-27T06:31:16.865 INFO:teuthology.orchestra.run.smithi156.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-27T06:31:16.899 INFO:teuthology.orchestra.run.smithi156.stdout:(91/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-03-27T06:31:16.912 INFO:teuthology.orchestra.run.smithi016.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 3.3 MB/s | 911 kB 00:00 2024-03-27T06:31:16.958 INFO:teuthology.orchestra.run.smithi156.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 277 kB/s | 25 kB 00:00 2024-03-27T06:31:16.971 INFO:teuthology.orchestra.run.smithi016.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-03-27T06:31:16.983 INFO:teuthology.orchestra.run.smithi156.stdout:(93/174): elfutils-libelf-devel-0.190-2.el9.x86 151 kB/s | 21 kB 00:00 2024-03-27T06:31:17.013 INFO:teuthology.orchestra.run.smithi016.stdout:(96/174): postgresql-13.14-1.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-03-27T06:31:17.113 INFO:teuthology.orchestra.run.smithi016.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-03-27T06:31:17.213 INFO:teuthology.orchestra.run.smithi016.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-03-27T06:31:17.308 INFO:teuthology.orchestra.run.smithi156.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 2.7 MB/s | 911 kB 00:00 2024-03-27T06:31:17.314 INFO:teuthology.orchestra.run.smithi016.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-03-27T06:31:17.350 INFO:teuthology.orchestra.run.smithi156.stdout:(95/174): postgresql-13.14-1.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-03-27T06:31:17.375 INFO:teuthology.orchestra.run.smithi156.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-03-27T06:31:17.389 INFO:teuthology.orchestra.run.smithi016.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 480 kB/s | 36 kB 00:00 2024-03-27T06:31:17.456 INFO:teuthology.orchestra.run.smithi016.stdout:(101/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 713 kB/s | 47 kB 00:00 2024-03-27T06:31:17.467 INFO:teuthology.orchestra.run.smithi156.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-03-27T06:31:17.514 INFO:teuthology.orchestra.run.smithi016.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-27T06:31:17.567 INFO:teuthology.orchestra.run.smithi156.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-03-27T06:31:17.579 INFO:teuthology.orchestra.run.smithi170.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2478.g15526 2.3 MB/s | 3.5 MB 00:01 2024-03-27T06:31:17.615 INFO:teuthology.orchestra.run.smithi016.stdout:(103/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.0 MB/s | 4.1 MB 00:01 2024-03-27T06:31:17.659 INFO:teuthology.orchestra.run.smithi156.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-03-27T06:31:17.704 INFO:teuthology.orchestra.run.smithi170.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2478.g15 1.9 MB/s | 249 kB 00:00 2024-03-27T06:31:17.718 INFO:teuthology.orchestra.run.smithi156.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 615 kB/s | 36 kB 00:00 2024-03-27T06:31:17.771 INFO:teuthology.orchestra.run.smithi170.stdout:(33/174): ceph-mgr-rook-19.0.0-2478.g155268c4.e 741 kB/s | 49 kB 00:00 2024-03-27T06:31:17.777 INFO:teuthology.orchestra.run.smithi156.stdout:(101/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-03-27T06:31:17.823 INFO:teuthology.orchestra.run.smithi016.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-03-27T06:31:17.835 INFO:teuthology.orchestra.run.smithi156.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-27T06:31:17.838 INFO:teuthology.orchestra.run.smithi170.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2478.g1 247 kB/s | 16 kB 00:00 2024-03-27T06:31:17.894 INFO:teuthology.orchestra.run.smithi156.stdout:(103/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.2 MB/s | 4.1 MB 00:00 2024-03-27T06:31:17.938 INFO:teuthology.orchestra.run.smithi170.stdout:(35/174): ceph-volume-19.0.0-2478.g155268c4.el9 2.6 MB/s | 264 kB 00:00 2024-03-27T06:31:17.944 INFO:teuthology.orchestra.run.smithi156.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-03-27T06:31:18.016 INFO:teuthology.orchestra.run.smithi016.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.8 MB/s | 548 kB 00:00 2024-03-27T06:31:18.019 INFO:teuthology.orchestra.run.smithi156.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.8 MB/s | 863 kB 00:00 2024-03-27T06:31:18.066 INFO:teuthology.orchestra.run.smithi016.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 6.1 MB/s | 309 kB 00:00 2024-03-27T06:31:18.099 INFO:teuthology.orchestra.run.smithi016.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 587 kB/s | 19 kB 00:00 2024-03-27T06:31:18.174 INFO:teuthology.orchestra.run.smithi016.stdout:(108/174): postgresql-server-13.14-1.el9.x86_64 4.8 MB/s | 5.8 MB 00:01 2024-03-27T06:31:18.181 INFO:teuthology.orchestra.run.smithi170.stdout:(36/174): cephadm-19.0.0-2478.g155268c4.el9.noa 3.1 MB/s | 762 kB 00:00 2024-03-27T06:31:18.225 INFO:teuthology.orchestra.run.smithi016.stdout:(109/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-03-27T06:31:18.239 INFO:teuthology.orchestra.run.smithi170.stdout:(37/174): cephfs-top-19.0.0-2478.g155268c4.el9. 314 kB/s | 18 kB 00:00 2024-03-27T06:31:18.250 INFO:teuthology.orchestra.run.smithi016.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-03-27T06:31:18.384 INFO:teuthology.orchestra.run.smithi016.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-03-27T06:31:18.389 INFO:teuthology.orchestra.run.smithi170.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 662 kB 00:00 2024-03-27T06:31:18.426 INFO:teuthology.orchestra.run.smithi016.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 242 kB/s | 49 kB 00:00 2024-03-27T06:31:18.451 INFO:teuthology.orchestra.run.smithi016.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 335 kB/s | 67 kB 00:00 2024-03-27T06:31:18.456 INFO:teuthology.orchestra.run.smithi170.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 13 MB/s | 920 kB 00:00 2024-03-27T06:31:18.461 INFO:teuthology.orchestra.run.smithi156.stdout:(106/174): postgresql-server-13.14-1.el9.x86_64 5.2 MB/s | 5.8 MB 00:01 2024-03-27T06:31:18.476 INFO:teuthology.orchestra.run.smithi016.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-03-27T06:31:18.486 INFO:teuthology.orchestra.run.smithi170.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.5 MB/s | 106 kB 00:00 2024-03-27T06:31:18.487 INFO:teuthology.orchestra.run.smithi156.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 700 kB/s | 309 kB 00:00 2024-03-27T06:31:18.518 INFO:teuthology.orchestra.run.smithi016.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-03-27T06:31:18.520 INFO:teuthology.orchestra.run.smithi156.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 952 kB/s | 548 kB 00:00 2024-03-27T06:31:18.528 INFO:teuthology.orchestra.run.smithi170.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-27T06:31:18.560 INFO:teuthology.orchestra.run.smithi016.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 776 kB/s | 32 kB 00:00 2024-03-27T06:31:18.578 INFO:teuthology.orchestra.run.smithi170.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 633 kB/s | 32 kB 00:00 2024-03-27T06:31:18.585 INFO:teuthology.orchestra.run.smithi016.stdout:(117/174): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-03-27T06:31:18.611 INFO:teuthology.orchestra.run.smithi016.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-03-27T06:31:18.636 INFO:teuthology.orchestra.run.smithi016.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-03-27T06:31:18.637 INFO:teuthology.orchestra.run.smithi156.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 286 kB/s | 26 kB 00:00 2024-03-27T06:31:18.670 INFO:teuthology.orchestra.run.smithi016.stdout:(120/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-03-27T06:31:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-27T06:31:18.720 INFO:teuthology.orchestra.run.smithi016.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 703 kB/s | 35 kB 00:00 2024-03-27T06:31:18.738 INFO:teuthology.orchestra.run.smithi156.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-03-27T06:31:18.745 INFO:teuthology.orchestra.run.smithi170.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-03-27T06:31:18.746 INFO:teuthology.orchestra.run.smithi016.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-03-27T06:31:18.762 INFO:teuthology.orchestra.run.smithi156.stdout:(111/174): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-03-27T06:31:18.771 INFO:teuthology.orchestra.run.smithi016.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-27T06:31:18.796 INFO:teuthology.orchestra.run.smithi016.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-03-27T06:31:18.796 INFO:teuthology.orchestra.run.smithi170.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 340 kB/s | 17 kB 00:00 2024-03-27T06:31:18.821 INFO:teuthology.orchestra.run.smithi016.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-03-27T06:31:18.829 INFO:teuthology.orchestra.run.smithi156.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 735 kB/s | 67 kB 00:00 2024-03-27T06:31:18.846 INFO:teuthology.orchestra.run.smithi016.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-03-27T06:31:18.863 INFO:teuthology.orchestra.run.smithi170.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 151 kB/s | 10 kB 00:00 2024-03-27T06:31:18.872 INFO:teuthology.orchestra.run.smithi016.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-03-27T06:31:18.897 INFO:teuthology.orchestra.run.smithi016.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-03-27T06:31:18.922 INFO:teuthology.orchestra.run.smithi016.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-03-27T06:31:18.947 INFO:teuthology.orchestra.run.smithi170.stdout:(46/174): ceph-mgr-diskprediction-local-19.0.0- 2.7 MB/s | 7.4 MB 00:02 2024-03-27T06:31:18.963 INFO:teuthology.orchestra.run.smithi156.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-03-27T06:31:18.964 INFO:teuthology.orchestra.run.smithi016.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-03-27T06:31:18.972 INFO:teuthology.orchestra.run.smithi170.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 159 kB 00:00 2024-03-27T06:31:18.989 INFO:teuthology.orchestra.run.smithi016.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-03-27T06:31:18.997 INFO:teuthology.orchestra.run.smithi156.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 646 kB/s | 151 kB 00:00 2024-03-27T06:31:18.997 INFO:teuthology.orchestra.run.smithi170.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-03-27T06:31:19.015 INFO:teuthology.orchestra.run.smithi016.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-03-27T06:31:19.030 INFO:teuthology.orchestra.run.smithi156.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-03-27T06:31:19.031 INFO:teuthology.orchestra.run.smithi170.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-27T06:31:19.040 INFO:teuthology.orchestra.run.smithi016.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-27T06:31:19.056 INFO:teuthology.orchestra.run.smithi156.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-03-27T06:31:19.056 INFO:teuthology.orchestra.run.smithi170.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-03-27T06:31:19.065 INFO:teuthology.orchestra.run.smithi016.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-27T06:31:19.081 INFO:teuthology.orchestra.run.smithi170.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 46 kB 00:00 2024-03-27T06:31:19.089 INFO:teuthology.orchestra.run.smithi156.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-03-27T06:31:19.090 INFO:teuthology.orchestra.run.smithi016.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-03-27T06:31:19.115 INFO:teuthology.orchestra.run.smithi156.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-03-27T06:31:19.115 INFO:teuthology.orchestra.run.smithi170.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-03-27T06:31:19.116 INFO:teuthology.orchestra.run.smithi016.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-03-27T06:31:19.140 INFO:teuthology.orchestra.run.smithi156.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-03-27T06:31:19.141 INFO:teuthology.orchestra.run.smithi170.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-03-27T06:31:19.141 INFO:teuthology.orchestra.run.smithi016.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-03-27T06:31:19.166 INFO:teuthology.orchestra.run.smithi170.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-27T06:31:19.166 INFO:teuthology.orchestra.run.smithi016.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-03-27T06:31:19.191 INFO:teuthology.orchestra.run.smithi170.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-03-27T06:31:19.192 INFO:teuthology.orchestra.run.smithi016.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-03-27T06:31:19.198 INFO:teuthology.orchestra.run.smithi156.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-27T06:31:19.223 INFO:teuthology.orchestra.run.smithi156.stdout:(121/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-03-27T06:31:19.242 INFO:teuthology.orchestra.run.smithi016.stdout:(141/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.9 MB/s | 2.0 MB 00:00 2024-03-27T06:31:19.249 INFO:teuthology.orchestra.run.smithi156.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-03-27T06:31:19.268 INFO:teuthology.orchestra.run.smithi016.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 586 kB/s | 59 kB 00:00 2024-03-27T06:31:19.284 INFO:teuthology.orchestra.run.smithi016.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-03-27T06:31:19.291 INFO:teuthology.orchestra.run.smithi156.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-03-27T06:31:19.301 INFO:teuthology.orchestra.run.smithi016.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 866 kB/s | 51 kB 00:00 2024-03-27T06:31:19.327 INFO:teuthology.orchestra.run.smithi016.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-03-27T06:31:19.333 INFO:teuthology.orchestra.run.smithi156.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-03-27T06:31:19.333 INFO:teuthology.orchestra.run.smithi170.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-03-27T06:31:19.352 INFO:teuthology.orchestra.run.smithi016.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-27T06:31:19.366 INFO:teuthology.orchestra.run.smithi170.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-03-27T06:31:19.369 INFO:teuthology.orchestra.run.smithi016.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-03-27T06:31:19.375 INFO:teuthology.orchestra.run.smithi156.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-03-27T06:31:19.392 INFO:teuthology.orchestra.run.smithi170.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-27T06:31:19.392 INFO:teuthology.orchestra.run.smithi156.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-03-27T06:31:19.402 INFO:teuthology.orchestra.run.smithi016.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-03-27T06:31:19.428 INFO:teuthology.orchestra.run.smithi016.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-27T06:31:19.442 INFO:teuthology.orchestra.run.smithi156.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-03-27T06:31:19.467 INFO:teuthology.orchestra.run.smithi156.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-03-27T06:31:19.469 INFO:teuthology.orchestra.run.smithi016.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-03-27T06:31:19.492 INFO:teuthology.orchestra.run.smithi156.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-03-27T06:31:19.518 INFO:teuthology.orchestra.run.smithi156.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-03-27T06:31:19.549 INFO:teuthology.orchestra.run.smithi170.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-03-27T06:31:19.550 INFO:teuthology.orchestra.run.smithi016.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-27T06:31:19.550 INFO:teuthology.orchestra.run.smithi156.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-03-27T06:31:19.561 INFO:teuthology.orchestra.run.smithi170.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 148 kB/s | 24 kB 00:00 2024-03-27T06:31:19.588 INFO:teuthology.orchestra.run.smithi170.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 543 kB/s | 18 kB 00:00 2024-03-27T06:31:19.594 INFO:teuthology.orchestra.run.smithi156.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-03-27T06:31:19.610 INFO:teuthology.orchestra.run.smithi170.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-27T06:31:19.627 INFO:teuthology.orchestra.run.smithi170.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-03-27T06:31:19.627 INFO:teuthology.orchestra.run.smithi156.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 943 kB/s | 79 kB 00:00 2024-03-27T06:31:19.672 INFO:teuthology.orchestra.run.smithi016.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 15 MB/s | 2.3 MB 00:00 2024-03-27T06:31:19.681 INFO:teuthology.orchestra.run.smithi156.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-03-27T06:31:19.712 INFO:teuthology.orchestra.run.smithi156.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-27T06:31:19.745 INFO:teuthology.orchestra.run.smithi156.stdout:(136/174): python3-kubernetes-26.1.0-2.el9.noar 4.1 MB/s | 1.0 MB 00:00 2024-03-27T06:31:19.770 INFO:teuthology.orchestra.run.smithi156.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-03-27T06:31:19.787 INFO:teuthology.orchestra.run.smithi016.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 16 MB/s | 1.8 MB 00:00 2024-03-27T06:31:19.796 INFO:teuthology.orchestra.run.smithi156.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-03-27T06:31:19.823 INFO:teuthology.orchestra.run.smithi156.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-03-27T06:31:19.846 INFO:teuthology.orchestra.run.smithi016.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 725 kB/s | 42 kB 00:00 2024-03-27T06:31:19.846 INFO:teuthology.orchestra.run.smithi156.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-03-27T06:31:19.871 INFO:teuthology.orchestra.run.smithi156.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-27T06:31:19.896 INFO:teuthology.orchestra.run.smithi156.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-03-27T06:31:19.921 INFO:teuthology.orchestra.run.smithi016.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 3.2 MB/s | 248 kB 00:00 2024-03-27T06:31:19.921 INFO:teuthology.orchestra.run.smithi156.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-03-27T06:31:19.947 INFO:teuthology.orchestra.run.smithi156.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-03-27T06:31:19.972 INFO:teuthology.orchestra.run.smithi156.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-27T06:31:19.980 INFO:teuthology.orchestra.run.smithi016.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-27T06:31:19.997 INFO:teuthology.orchestra.run.smithi156.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-27T06:31:20.039 INFO:teuthology.orchestra.run.smithi156.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-03-27T06:31:20.052 INFO:teuthology.orchestra.run.smithi170.stdout:(64/174): ceph-test-19.0.0-2478.g155268c4.el9.x 5.0 MB/s | 48 MB 00:09 2024-03-27T06:31:20.063 INFO:teuthology.orchestra.run.smithi016.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 438 kB/s | 36 kB 00:00 2024-03-27T06:31:20.064 INFO:teuthology.orchestra.run.smithi156.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-03-27T06:31:20.090 INFO:teuthology.orchestra.run.smithi156.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-03-27T06:31:20.130 INFO:teuthology.orchestra.run.smithi016.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-03-27T06:31:20.152 INFO:teuthology.orchestra.run.smithi170.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.8 MB/s | 4.8 MB 00:00 2024-03-27T06:31:20.178 INFO:teuthology.orchestra.run.smithi170.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 395 kB/s | 217 kB 00:00 2024-03-27T06:31:20.203 INFO:teuthology.orchestra.run.smithi170.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-27T06:31:20.205 INFO:teuthology.orchestra.run.smithi016.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 256 kB 00:00 2024-03-27T06:31:20.223 INFO:teuthology.orchestra.run.smithi156.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-03-27T06:31:20.228 INFO:teuthology.orchestra.run.smithi170.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 920 kB/s | 69 kB 00:00 2024-03-27T06:31:20.253 INFO:teuthology.orchestra.run.smithi170.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 825 kB/s | 41 kB 00:00 2024-03-27T06:31:20.256 INFO:teuthology.orchestra.run.smithi016.stdout:(160/174): librbd1-19.0.0-2478.g155268c4.el9.x8 3.8 MB/s | 3.1 MB 00:00 2024-03-27T06:31:20.278 INFO:teuthology.orchestra.run.smithi170.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-27T06:31:20.285 INFO:teuthology.orchestra.run.smithi156.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.1 MB/s | 321 kB 00:00 2024-03-27T06:31:20.298 INFO:teuthology.orchestra.run.smithi016.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 251 kB/s | 23 kB 00:00 2024-03-27T06:31:20.304 INFO:teuthology.orchestra.run.smithi170.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-27T06:31:20.329 INFO:teuthology.orchestra.run.smithi170.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-27T06:31:20.354 INFO:teuthology.orchestra.run.smithi170.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 631 kB/s | 47 kB 00:00 2024-03-27T06:31:20.380 INFO:teuthology.orchestra.run.smithi170.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 571 kB/s | 43 kB 00:00 2024-03-27T06:31:20.406 INFO:teuthology.orchestra.run.smithi016.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 723 kB/s | 108 kB 00:00 2024-03-27T06:31:20.446 INFO:teuthology.orchestra.run.smithi170.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 422 kB/s | 39 kB 00:00 2024-03-27T06:31:20.448 INFO:teuthology.orchestra.run.smithi016.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 155 kB 00:00 2024-03-27T06:31:20.499 INFO:teuthology.orchestra.run.smithi016.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 136 kB 00:00 2024-03-27T06:31:20.516 INFO:teuthology.orchestra.run.smithi156.stdout:(152/174): librados2-19.0.0-2478.g155268c4.el9. 7.5 MB/s | 3.4 MB 00:00 2024-03-27T06:31:20.524 INFO:teuthology.orchestra.run.smithi016.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 849 kB/s | 64 kB 00:00 2024-03-27T06:31:20.566 INFO:teuthology.orchestra.run.smithi016.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 429 kB/s | 29 kB 00:00 2024-03-27T06:31:20.574 INFO:teuthology.orchestra.run.smithi156.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 8.0 MB/s | 2.3 MB 00:00 2024-03-27T06:31:20.625 INFO:teuthology.orchestra.run.smithi156.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 841 kB/s | 42 kB 00:00 2024-03-27T06:31:20.638 INFO:teuthology.orchestra.run.smithi170.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 159 kB/s | 41 kB 00:00 2024-03-27T06:31:20.663 INFO:teuthology.orchestra.run.smithi170.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-03-27T06:31:20.684 INFO:teuthology.orchestra.run.smithi016.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 7.6 MB/s | 1.2 MB 00:00 2024-03-27T06:31:20.716 INFO:teuthology.orchestra.run.smithi156.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.7 MB/s | 248 kB 00:00 2024-03-27T06:31:20.758 INFO:teuthology.orchestra.run.smithi016.stdout:(168/174): librados2-19.0.0-2478.g155268c4.el9. 2.5 MB/s | 3.4 MB 00:01 2024-03-27T06:31:20.800 INFO:teuthology.orchestra.run.smithi016.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 9.2 MB/s | 2.2 MB 00:00 2024-03-27T06:31:20.825 INFO:teuthology.orchestra.run.smithi156.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 658 kB/s | 71 kB 00:00 2024-03-27T06:31:20.850 INFO:teuthology.orchestra.run.smithi016.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 624 kB/s | 31 kB 00:00 2024-03-27T06:31:20.867 INFO:teuthology.orchestra.run.smithi156.stdout:(157/174): NetworkManager-libnm-1.47.2-1.el9.x8 5.2 MB/s | 1.8 MB 00:00 2024-03-27T06:31:20.889 INFO:teuthology.orchestra.run.smithi170.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-03-27T06:31:20.892 INFO:teuthology.orchestra.run.smithi156.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 545 kB/s | 36 kB 00:00 2024-03-27T06:31:20.893 INFO:teuthology.orchestra.run.smithi016.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 3.4 MB/s | 465 kB 00:00 2024-03-27T06:31:20.914 INFO:teuthology.orchestra.run.smithi170.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 206 kB/s | 56 kB 00:00 2024-03-27T06:31:20.917 INFO:teuthology.orchestra.run.smithi156.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 192 kB 00:00 2024-03-27T06:31:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 229 kB/s | 63 kB 00:00 2024-03-27T06:31:20.943 INFO:teuthology.orchestra.run.smithi016.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.8 MB/s | 2.3 MB 00:00 2024-03-27T06:31:20.943 INFO:teuthology.orchestra.run.smithi156.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 256 kB 00:00 2024-03-27T06:31:20.964 INFO:teuthology.orchestra.run.smithi170.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-27T06:31:20.990 INFO:teuthology.orchestra.run.smithi170.stdout:(82/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-03-27T06:31:21.001 INFO:teuthology.orchestra.run.smithi016.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 686 kB/s | 74 kB 00:00 2024-03-27T06:31:21.009 INFO:teuthology.orchestra.run.smithi156.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 251 kB/s | 23 kB 00:00 2024-03-27T06:31:21.015 INFO:teuthology.orchestra.run.smithi170.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-27T06:31:21.035 INFO:teuthology.orchestra.run.smithi156.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 108 kB 00:00 2024-03-27T06:31:21.043 INFO:teuthology.orchestra.run.smithi016.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.6 MB/s | 1.1 MB 00:00 2024-03-27T06:31:21.050 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-03-27T06:31:21.050 INFO:teuthology.orchestra.run.smithi016.stdout:Total 10 MB/s | 227 MB 00:22 2024-03-27T06:31:21.050 INFO:teuthology.orchestra.run.smithi170.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-27T06:31:21.060 INFO:teuthology.orchestra.run.smithi156.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-27T06:31:21.074 INFO:teuthology.orchestra.run.smithi170.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-27T06:31:21.085 INFO:teuthology.orchestra.run.smithi156.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-27T06:31:21.099 INFO:teuthology.orchestra.run.smithi170.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-27T06:31:21.111 INFO:teuthology.orchestra.run.smithi156.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-27T06:31:21.124 INFO:teuthology.orchestra.run.smithi170.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-27T06:31:21.128 INFO:teuthology.orchestra.run.smithi156.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 681 kB/s | 29 kB 00:00 2024-03-27T06:31:21.275 INFO:teuthology.orchestra.run.smithi170.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-27T06:31:21.295 INFO:teuthology.orchestra.run.smithi156.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 6.6 MB/s | 1.2 MB 00:00 2024-03-27T06:31:21.300 INFO:teuthology.orchestra.run.smithi170.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-03-27T06:31:21.334 INFO:teuthology.orchestra.run.smithi170.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-27T06:31:21.387 INFO:teuthology.orchestra.run.smithi156.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 8.3 MB/s | 2.2 MB 00:00 2024-03-27T06:31:21.417 INFO:teuthology.orchestra.run.smithi170.stdout:(91/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-03-27T06:31:21.454 INFO:teuthology.orchestra.run.smithi156.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.8 MB/s | 465 kB 00:00 2024-03-27T06:31:21.513 INFO:teuthology.orchestra.run.smithi156.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-03-27T06:31:21.529 INFO:teuthology.orchestra.run.smithi156.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 414 kB/s | 31 kB 00:00 2024-03-27T06:31:21.576 INFO:teuthology.orchestra.run.smithi170.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 105 kB/s | 25 kB 00:00 2024-03-27T06:31:21.579 INFO:teuthology.orchestra.run.smithi156.stdout:(172/174): librbd1-19.0.0-2478.g155268c4.el9.x8 2.1 MB/s | 3.1 MB 00:01 2024-03-27T06:31:21.601 INFO:teuthology.orchestra.run.smithi170.stdout:(93/174): elfutils-libelf-devel-0.190-2.el9.x86 71 kB/s | 21 kB 00:00 2024-03-27T06:31:21.630 INFO:teuthology.orchestra.run.smithi156.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 742 kB/s | 74 kB 00:00 2024-03-27T06:31:21.633 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-03-27T06:31:21.655 INFO:teuthology.orchestra.run.smithi156.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-03-27T06:31:21.662 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2024-03-27T06:31:21.662 INFO:teuthology.orchestra.run.smithi156.stdout:Total 11 MB/s | 227 MB 00:20 2024-03-27T06:31:21.777 INFO:teuthology.orchestra.run.smithi170.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-03-27T06:31:21.869 INFO:teuthology.orchestra.run.smithi170.stdout:(95/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 4.1 MB 00:00 2024-03-27T06:31:21.886 INFO:teuthology.orchestra.run.smithi170.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-03-27T06:31:21.933 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-03-27T06:31:21.934 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-03-27T06:31:21.936 INFO:teuthology.orchestra.run.smithi170.stdout:(97/174): postgresql-13.14-1.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-03-27T06:31:22.003 INFO:teuthology.orchestra.run.smithi170.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-03-27T06:31:22.037 INFO:teuthology.orchestra.run.smithi170.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-03-27T06:31:22.088 INFO:teuthology.orchestra.run.smithi170.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-03-27T06:31:22.113 INFO:teuthology.orchestra.run.smithi170.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 476 kB/s | 36 kB 00:00 2024-03-27T06:31:22.171 INFO:teuthology.orchestra.run.smithi170.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-03-27T06:31:22.188 INFO:teuthology.orchestra.run.smithi170.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-03-27T06:31:22.223 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2024-03-27T06:31:22.238 INFO:teuthology.orchestra.run.smithi170.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 333 kB/s | 22 kB 00:00 2024-03-27T06:31:22.322 INFO:teuthology.orchestra.run.smithi170.stdout:(105/174): postgresql-server-13.14-1.el9.x86_64 13 MB/s | 5.8 MB 00:00 2024-03-27T06:31:22.414 INFO:teuthology.orchestra.run.smithi170.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-03-27T06:31:22.496 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2024-03-27T06:31:22.496 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2024-03-27T06:31:22.664 INFO:teuthology.orchestra.run.smithi170.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 86 kB/s | 19 kB 00:00 2024-03-27T06:31:22.715 INFO:teuthology.orchestra.run.smithi170.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-03-27T06:31:22.765 INFO:teuthology.orchestra.run.smithi170.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 740 kB/s | 309 kB 00:00 2024-03-27T06:31:22.790 INFO:teuthology.orchestra.run.smithi170.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-03-27T06:31:22.849 INFO:teuthology.orchestra.run.smithi170.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-03-27T06:31:22.866 INFO:teuthology.orchestra.run.smithi170.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 894 kB/s | 67 kB 00:00 2024-03-27T06:31:22.949 INFO:teuthology.orchestra.run.smithi170.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-03-27T06:31:23.016 INFO:teuthology.orchestra.run.smithi170.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.5 MB/s | 838 kB 00:00 2024-03-27T06:31:23.091 INFO:teuthology.orchestra.run.smithi170.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 580 kB/s | 43 kB 00:00 2024-03-27T06:31:23.125 INFO:teuthology.orchestra.run.smithi170.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-03-27T06:31:23.175 INFO:teuthology.orchestra.run.smithi170.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-03-27T06:31:23.209 INFO:teuthology.orchestra.run.smithi170.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-03-27T06:31:23.276 INFO:teuthology.orchestra.run.smithi170.stdout:(119/174): libarrow-9.0.0-10.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-03-27T06:31:23.301 INFO:teuthology.orchestra.run.smithi170.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-03-27T06:31:23.326 INFO:teuthology.orchestra.run.smithi170.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-03-27T06:31:23.360 INFO:teuthology.orchestra.run.smithi170.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-03-27T06:31:23.402 INFO:teuthology.orchestra.run.smithi170.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-03-27T06:31:23.418 INFO:teuthology.orchestra.run.smithi170.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-27T06:31:23.511 INFO:teuthology.orchestra.run.smithi170.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-03-27T06:31:23.536 INFO:teuthology.orchestra.run.smithi170.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-03-27T06:31:23.561 INFO:teuthology.orchestra.run.smithi170.stdout:(127/174): python3-grpcio-tools-1.46.7-10.el9.x 908 kB/s | 144 kB 00:00 2024-03-27T06:31:23.595 INFO:teuthology.orchestra.run.smithi170.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-03-27T06:31:23.620 INFO:teuthology.orchestra.run.smithi170.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-03-27T06:31:23.645 INFO:teuthology.orchestra.run.smithi170.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 231 kB/s | 19 kB 00:00 2024-03-27T06:31:23.671 INFO:teuthology.orchestra.run.smithi170.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 259 kB/s | 20 kB 00:00 2024-03-27T06:31:23.729 INFO:teuthology.orchestra.run.smithi170.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-03-27T06:31:23.746 INFO:teuthology.orchestra.run.smithi170.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 460 kB/s | 46 kB 00:00 2024-03-27T06:31:23.772 INFO:teuthology.orchestra.run.smithi170.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 784 kB/s | 79 kB 00:00 2024-03-27T06:31:23.797 INFO:teuthology.orchestra.run.smithi170.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 860 kB/s | 58 kB 00:00 2024-03-27T06:31:23.830 INFO:teuthology.orchestra.run.smithi170.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-27T06:31:23.847 INFO:teuthology.orchestra.run.smithi170.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-03-27T06:31:23.881 INFO:teuthology.orchestra.run.smithi170.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-03-27T06:31:23.906 INFO:teuthology.orchestra.run.smithi170.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-03-27T06:31:23.932 INFO:teuthology.orchestra.run.smithi170.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-03-27T06:31:23.965 INFO:teuthology.orchestra.run.smithi170.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-03-27T06:31:23.990 INFO:teuthology.orchestra.run.smithi170.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 705 kB/s | 59 kB 00:00 2024-03-27T06:31:24.016 INFO:teuthology.orchestra.run.smithi170.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-03-27T06:31:24.058 INFO:teuthology.orchestra.run.smithi170.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-03-27T06:31:24.083 INFO:teuthology.orchestra.run.smithi170.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-03-27T06:31:24.085 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-03-27T06:31:24.085 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-03-27T06:31:24.110 INFO:teuthology.orchestra.run.smithi170.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 974 kB/s | 90 kB 00:00 2024-03-27T06:31:24.158 INFO:teuthology.orchestra.run.smithi170.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-27T06:31:24.184 INFO:teuthology.orchestra.run.smithi170.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-03-27T06:31:24.209 INFO:teuthology.orchestra.run.smithi170.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-27T06:31:24.232 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2024-03-27T06:31:24.232 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2024-03-27T06:31:24.292 INFO:teuthology.orchestra.run.smithi170.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-03-27T06:31:24.576 INFO:teuthology.orchestra.run.smithi170.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.1 MB/s | 321 kB 00:00 2024-03-27T06:31:25.151 INFO:teuthology.orchestra.run.smithi170.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 4.0 MB/s | 2.3 MB 00:00 2024-03-27T06:31:25.493 INFO:teuthology.orchestra.run.smithi170.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 5.4 MB/s | 1.8 MB 00:00 2024-03-27T06:31:25.552 INFO:teuthology.orchestra.run.smithi170.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 722 kB/s | 42 kB 00:00 2024-03-27T06:31:25.619 INFO:teuthology.orchestra.run.smithi170.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 3.6 MB/s | 248 kB 00:00 2024-03-27T06:31:25.694 INFO:teuthology.orchestra.run.smithi170.stdout:(156/174): librbd1-19.0.0-2478.g155268c4.el9.x8 2.1 MB/s | 3.1 MB 00:01 2024-03-27T06:31:25.719 INFO:teuthology.orchestra.run.smithi170.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 713 kB/s | 71 kB 00:00 2024-03-27T06:31:25.761 INFO:teuthology.orchestra.run.smithi170.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 545 kB/s | 36 kB 00:00 2024-03-27T06:31:25.795 INFO:teuthology.orchestra.run.smithi170.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-27T06:31:25.862 INFO:teuthology.orchestra.run.smithi170.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 346 kB/s | 23 kB 00:00 2024-03-27T06:31:25.887 INFO:teuthology.orchestra.run.smithi170.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-03-27T06:31:26.171 INFO:teuthology.orchestra.run.smithi170.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 351 kB/s | 108 kB 00:00 2024-03-27T06:31:26.196 INFO:teuthology.orchestra.run.smithi170.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 502 kB/s | 155 kB 00:00 2024-03-27T06:31:26.255 INFO:teuthology.orchestra.run.smithi170.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-27T06:31:26.280 INFO:teuthology.orchestra.run.smithi170.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-27T06:31:26.330 INFO:teuthology.orchestra.run.smithi170.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 381 kB/s | 29 kB 00:00 2024-03-27T06:31:26.539 INFO:teuthology.orchestra.run.smithi170.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 4.7 MB/s | 1.2 MB 00:00 2024-03-27T06:31:26.797 INFO:teuthology.orchestra.run.smithi170.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.6 MB/s | 2.2 MB 00:00 2024-03-27T06:31:26.856 INFO:teuthology.orchestra.run.smithi170.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.2 MB/s | 2.3 MB 00:00 2024-03-27T06:31:26.915 INFO:teuthology.orchestra.run.smithi170.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.9 MB/s | 465 kB 00:00 2024-03-27T06:31:26.940 INFO:teuthology.orchestra.run.smithi170.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 373 kB/s | 31 kB 00:00 2024-03-27T06:31:27.040 INFO:teuthology.orchestra.run.smithi170.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 743 kB/s | 74 kB 00:00 2024-03-27T06:31:27.074 INFO:teuthology.orchestra.run.smithi170.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2024-03-27T06:31:27.591 INFO:teuthology.orchestra.run.smithi170.stdout:(174/174): librados2-19.0.0-2478.g155268c4.el9. 1.0 MB/s | 3.4 MB 00:03 2024-03-27T06:31:27.597 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-03-27T06:31:27.597 INFO:teuthology.orchestra.run.smithi170.stdout:Total 7.6 MB/s | 227 MB 00:29 2024-03-27T06:31:28.191 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-03-27T06:31:28.470 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-03-27T06:31:28.471 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-03-27T06:31:30.242 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-03-27T06:31:30.242 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-03-27T06:31:31.392 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2024-03-27T06:31:31.500 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-27T06:31:31.525 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-03-27T06:31:31.534 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-27T06:31:31.565 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T06:31:31.574 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T06:31:31.623 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-27T06:31:31.623 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-27T06:31:31.657 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-27T06:31:31.677 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-27T06:31:31.695 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T06:31:31.706 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T06:31:31.710 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-27T06:31:31.746 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-27T06:31:31.746 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-27T06:31:31.788 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-27T06:31:31.800 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-27T06:31:31.841 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-27T06:31:31.874 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-27T06:31:31.925 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-27T06:31:32.109 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-27T06:31:32.173 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T06:31:32.269 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-27T06:31:32.294 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T06:31:32.322 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T06:31:32.324 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T06:31:32.398 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T06:31:32.428 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T06:31:32.446 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T06:31:32.473 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-27T06:31:32.473 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T06:31:32.565 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T06:31:32.594 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T06:31:32.634 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-27T06:31:32.635 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-27T06:31:32.684 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-27T06:31:32.704 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T06:31:32.808 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T06:31:32.822 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T06:31:32.825 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-27T06:31:32.886 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T06:31:32.893 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-27T06:31:32.920 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T06:31:32.926 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-27T06:31:32.940 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T06:31:32.956 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T06:31:33.026 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T06:31:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T06:31:33.115 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T06:31:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T06:31:33.138 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T06:31:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-27T06:31:33.170 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-27T06:31:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T06:31:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T06:31:33.275 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-27T06:31:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T06:31:33.389 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T06:31:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T06:31:33.449 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T06:31:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-27T06:31:33.509 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T06:31:33.584 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T06:31:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-27T06:31:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T06:31:33.797 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T06:31:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T06:31:33.896 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-27T06:31:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T06:31:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T06:31:33.990 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-27T06:31:34.022 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-27T06:31:34.048 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-27T06:31:34.084 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-27T06:31:34.157 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-27T06:31:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T06:31:34.203 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-27T06:31:34.242 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-27T06:31:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-27T06:31:34.306 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-27T06:31:34.341 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-27T06:31:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-27T06:31:34.385 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-27T06:31:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-27T06:31:34.418 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-27T06:31:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-27T06:31:34.446 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-27T06:31:34.468 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-27T06:31:34.515 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-27T06:31:34.544 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-27T06:31:34.550 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 39/198 2024-03-27T06:31:34.594 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 40/198 2024-03-27T06:31:34.596 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-27T06:31:34.623 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 41/198 2024-03-27T06:31:34.634 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-27T06:31:34.690 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-27T06:31:34.700 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-27T06:31:34.745 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-27T06:31:34.746 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-27T06:31:34.802 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-27T06:31:34.819 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-27T06:31:34.835 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-27T06:31:34.862 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-27T06:31:34.934 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-27T06:31:34.950 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-27T06:31:34.975 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 39/198 2024-03-27T06:31:35.006 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-27T06:31:35.019 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 40/198 2024-03-27T06:31:35.047 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 41/198 2024-03-27T06:31:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-27T06:31:35.170 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-27T06:31:35.246 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-27T06:31:35.383 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-27T06:31:35.441 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-27T06:31:35.519 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-27T06:31:35.601 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-27T06:31:35.706 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-27T06:31:35.756 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T06:31:35.765 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T06:31:35.795 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-27T06:31:35.845 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-27T06:31:35.882 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 53/198 2024-03-27T06:31:35.912 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-27T06:31:35.989 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-27T06:31:36.070 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-27T06:31:36.140 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-27T06:31:36.143 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-27T06:31:36.188 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T06:31:36.197 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T06:31:36.229 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-27T06:31:36.278 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-27T06:31:36.307 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-27T06:31:36.315 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 53/198 2024-03-27T06:31:36.337 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-27T06:31:36.365 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-27T06:31:36.400 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-27T06:31:36.546 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-27T06:31:36.587 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-27T06:31:36.603 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-27T06:31:36.642 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-27T06:31:36.706 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-27T06:31:36.738 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-27T06:31:36.774 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-27T06:31:36.778 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-27T06:31:36.821 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-27T06:31:36.858 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-27T06:31:36.866 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-27T06:31:36.900 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-27T06:31:36.910 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-27T06:31:36.942 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-27T06:31:37.061 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-27T06:31:37.075 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-27T06:31:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-27T06:31:37.123 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-27T06:31:37.154 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-27T06:31:37.161 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-27T06:31:37.192 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-27T06:31:37.205 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-03-27T06:31:37.234 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-27T06:31:37.263 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-27T06:31:37.277 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-27T06:31:37.300 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-27T06:31:37.311 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-27T06:31:37.345 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-27T06:31:37.348 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-27T06:31:37.350 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-27T06:31:37.383 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-27T06:31:37.384 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T06:31:37.390 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-27T06:31:37.394 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-27T06:31:37.416 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-27T06:31:37.426 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-27T06:31:37.434 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-27T06:31:37.441 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-27T06:31:37.468 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-27T06:31:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-27T06:31:37.513 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-27T06:31:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-27T06:31:37.547 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-27T06:31:37.589 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-27T06:31:37.607 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-27T06:31:37.632 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-27T06:31:37.657 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-27T06:31:37.687 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-27T06:31:37.729 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-27T06:31:37.730 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-27T06:31:37.774 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-27T06:31:37.822 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-27T06:31:37.826 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-27T06:31:37.871 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-27T06:31:37.878 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-27T06:31:37.911 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-27T06:31:37.911 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-27T06:31:37.937 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-27T06:31:37.961 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-27T06:31:37.964 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T06:31:38.039 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-27T06:31:38.058 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-27T06:31:38.085 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T06:31:38.104 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-27T06:31:38.107 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T06:31:38.175 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-27T06:31:38.198 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T06:31:38.228 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T06:31:38.248 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-27T06:31:38.267 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-27T06:31:38.292 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-27T06:31:38.293 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-27T06:31:38.326 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-27T06:31:38.337 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-27T06:31:38.357 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-27T06:31:38.385 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-27T06:31:38.386 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-27T06:31:38.419 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-27T06:31:38.442 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-27T06:31:38.447 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-27T06:31:38.458 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-27T06:31:38.490 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-27T06:31:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-27T06:31:38.521 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-27T06:31:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T06:31:38.589 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-27T06:31:38.609 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T06:31:38.632 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T06:31:38.632 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-27T06:31:38.649 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-27T06:31:38.664 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-27T06:31:38.691 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-27T06:31:38.695 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 17/198 2024-03-27T06:31:38.706 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-27T06:31:38.735 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-27T06:31:38.749 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-27T06:31:38.758 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T06:31:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-27T06:31:38.833 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-27T06:31:38.846 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-27T06:31:38.881 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-27T06:31:38.913 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-27T06:31:38.924 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T06:31:38.927 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-27T06:31:38.944 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-27T06:31:38.956 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-27T06:31:38.971 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-27T06:31:38.972 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-27T06:31:38.987 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-27T06:31:39.003 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-27T06:31:39.006 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-27T06:31:39.041 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-27T06:31:39.054 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-27T06:31:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-27T06:31:39.094 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-27T06:31:39.124 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-27T06:31:39.138 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-27T06:31:39.187 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-27T06:31:39.190 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T06:31:39.203 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-27T06:31:39.234 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-27T06:31:39.250 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-27T06:31:39.267 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-27T06:31:39.310 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-27T06:31:39.312 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T06:31:39.347 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-27T06:31:39.385 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T06:31:39.435 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-27T06:31:39.526 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-27T06:31:39.575 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-27T06:31:39.579 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-27T06:31:39.606 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-27T06:31:39.654 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-27T06:31:39.690 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-27T06:31:39.763 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-27T06:31:39.803 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-27T06:31:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-27T06:31:39.831 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-27T06:31:39.858 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-27T06:31:39.894 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-27T06:31:39.909 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-27T06:31:39.952 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-27T06:31:39.967 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-27T06:31:39.985 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-27T06:31:40.022 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-27T06:31:40.033 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-27T06:31:40.061 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-27T06:31:40.112 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-27T06:31:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-27T06:31:40.153 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-27T06:31:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-27T06:31:40.203 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-27T06:31:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-27T06:31:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-27T06:31:40.330 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-27T06:31:40.333 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-27T06:31:40.360 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-27T06:31:40.368 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 39/198 2024-03-27T06:31:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 40/198 2024-03-27T06:31:40.437 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-27T06:31:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 41/198 2024-03-27T06:31:40.509 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-27T06:31:40.554 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-27T06:31:40.576 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-27T06:31:40.622 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-27T06:31:40.628 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-27T06:31:40.655 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-27T06:31:40.704 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-27T06:31:40.780 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-27T06:31:40.793 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-27T06:31:40.841 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-27T06:31:40.853 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-27T06:31:41.068 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-27T06:31:41.128 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-27T06:31:41.173 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-27T06:31:41.208 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-27T06:31:41.296 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-27T06:31:41.366 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-27T06:31:41.453 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-27T06:31:41.527 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-27T06:31:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T06:31:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-27T06:31:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-27T06:31:41.662 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-27T06:31:41.699 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 53/198 2024-03-27T06:31:41.728 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-27T06:31:41.953 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-27T06:31:41.991 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-27T06:31:42.092 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-27T06:31:42.107 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-27T06:31:42.173 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-27T06:31:42.218 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-27T06:31:42.346 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-27T06:31:42.385 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-27T06:31:42.395 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-27T06:31:42.450 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-27T06:31:42.461 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-27T06:31:42.492 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-27T06:31:42.522 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-27T06:31:42.523 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-27T06:31:42.551 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-27T06:31:42.557 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-27T06:31:42.593 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-27T06:31:42.640 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-27T06:31:42.676 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-27T06:31:42.719 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-27T06:31:42.761 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-27T06:31:42.861 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-27T06:31:42.889 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T06:31:42.898 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-27T06:31:42.942 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-27T06:31:42.963 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T06:31:42.972 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-27T06:31:42.994 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 125/198 2024-03-27T06:31:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-27T06:31:43.030 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-27T06:31:43.109 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-27T06:31:43.147 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-27T06:31:43.187 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-27T06:31:43.213 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-27T06:31:43.237 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-27T06:31:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-27T06:31:43.380 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-27T06:31:43.439 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-27T06:31:43.460 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-27T06:31:43.490 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-27T06:31:43.490 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-27T06:31:43.518 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T06:31:43.530 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-27T06:31:43.538 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-27T06:31:43.558 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-27T06:31:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-27T06:31:43.588 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T06:31:43.611 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 129/198 2024-03-27T06:31:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-27T06:31:43.691 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-27T06:31:43.878 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-27T06:31:43.887 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-27T06:31:43.913 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T06:31:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-27T06:31:44.004 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T06:31:44.034 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 125/198 2024-03-27T06:31:44.054 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-27T06:31:44.063 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-27T06:31:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-27T06:31:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-27T06:31:44.152 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-27T06:31:44.189 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-27T06:31:44.223 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-27T06:31:44.263 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-27T06:31:44.294 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-27T06:31:44.324 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-27T06:31:44.409 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-27T06:31:44.443 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-27T06:31:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-27T06:31:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-27T06:31:44.548 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-27T06:31:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-27T06:31:44.574 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T06:31:44.637 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T06:31:44.650 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-27T06:31:44.667 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 129/198 2024-03-27T06:31:44.739 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-27T06:31:44.792 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-27T06:31:44.823 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-27T06:31:44.870 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-27T06:31:44.978 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-27T06:31:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-27T06:31:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-27T06:31:45.805 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-27T06:31:45.811 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T06:31:45.838 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-27T06:31:45.849 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T06:31:45.887 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-27T06:31:45.975 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-27T06:31:46.006 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-27T06:31:46.186 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-27T06:31:46.212 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-27T06:31:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-27T06:31:46.383 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T06:31:46.420 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T06:31:46.501 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T06:31:46.501 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-27T06:31:46.501 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-27T06:31:46.501 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:31:46.529 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T06:31:46.919 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T06:31:46.936 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-27T06:31:46.956 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T06:31:47.044 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-27T06:31:47.524 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T06:31:47.559 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T06:31:47.636 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T06:31:47.636 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-27T06:31:47.636 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-27T06:31:47.636 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:31:47.678 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T06:31:48.284 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-27T06:31:48.388 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-27T06:31:48.419 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-27T06:31:48.450 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-27T06:31:48.485 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-27T06:31:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-27T06:31:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T06:31:48.898 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 124/198 2024-03-27T06:31:48.929 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 125/198 2024-03-27T06:31:48.965 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-27T06:31:49.426 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-27T06:31:49.469 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T06:31:49.531 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 128/198 2024-03-27T06:31:49.554 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 129/198 2024-03-27T06:31:51.810 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T06:31:51.842 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T06:31:52.378 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 130/198 2024-03-27T06:31:52.420 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T06:31:52.501 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 131/198 2024-03-27T06:31:52.502 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-27T06:31:52.502 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-27T06:31:52.502 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:31:52.530 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T06:31:58.538 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T06:31:58.539 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2024-03-27T06:31:58.539 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2024-03-27T06:31:58.539 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2024-03-27T06:31:58.539 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2024-03-27T06:31:58.539 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2024-03-27T06:31:58.539 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /root 2024-03-27T06:31:58.539 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2024-03-27T06:31:58.539 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:31:58.703 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T06:31:58.719 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T06:31:58.719 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:31:58.719 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-27T06:31:58.719 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:32:00.207 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T06:32:00.223 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T06:32:00.224 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-03-27T06:32:00.224 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-03-27T06:32:00.224 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-03-27T06:32:00.224 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-03-27T06:32:00.224 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-03-27T06:32:00.224 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-03-27T06:32:00.224 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-03-27T06:32:00.224 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:00.232 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T06:32:00.232 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:00.232 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-27T06:32:00.232 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T06:32:00.232 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T06:32:00.232 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:32:00.393 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T06:32:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T06:32:00.408 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:00.408 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-27T06:32:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:00.457 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T06:32:00.479 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T06:32:00.480 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:00.480 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-27T06:32:00.480 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T06:32:00.480 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T06:32:00.480 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:32:00.941 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T06:32:00.964 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T06:32:00.964 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:00.964 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-27T06:32:00.964 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T06:32:00.964 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T06:32:00.964 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:32:01.060 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-27T06:32:01.479 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T06:32:01.508 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T06:32:01.532 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 139/198 2024-03-27T06:32:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 140/198 2024-03-27T06:32:01.615 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T06:32:01.938 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T06:32:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T06:32:01.964 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:01.964 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-27T06:32:01.964 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T06:32:01.964 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T06:32:01.964 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T06:32:02.227 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T06:32:02.228 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:02.228 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-27T06:32:02.228 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T06:32:02.228 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T06:32:02.228 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:02.653 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T06:32:02.683 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T06:32:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T06:32:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T06:32:02.712 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:02.712 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-27T06:32:02.712 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T06:32:02.712 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T06:32:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:02.808 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-27T06:32:03.269 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T06:32:03.299 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T06:32:03.340 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 139/198 2024-03-27T06:32:03.402 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 140/198 2024-03-27T06:32:03.430 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T06:32:03.644 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T06:32:03.675 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T06:32:03.777 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T06:32:03.939 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 144/198 2024-03-27T06:32:03.969 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T06:32:03.991 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T06:32:03.991 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:03.992 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-27T06:32:03.992 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T06:32:03.992 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T06:32:03.992 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:32:04.037 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T06:32:04.048 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T06:32:04.093 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 147/198 2024-03-27T06:32:04.493 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 132/198 2024-03-27T06:32:04.493 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2024-03-27T06:32:04.493 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2024-03-27T06:32:04.493 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2024-03-27T06:32:04.493 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2024-03-27T06:32:04.493 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2024-03-27T06:32:04.493 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /root 2024-03-27T06:32:04.493 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2024-03-27T06:32:04.494 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:04.519 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T06:32:04.557 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T06:32:04.660 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T06:32:04.675 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 133/198 2024-03-27T06:32:04.675 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:04.675 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-27T06:32:04.675 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:04.963 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-27T06:32:04.989 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T06:32:05.011 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T06:32:05.011 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:05.011 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-27T06:32:05.011 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T06:32:05.011 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T06:32:05.011 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:32:05.058 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T06:32:05.079 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T06:32:05.079 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:05.080 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-27T06:32:05.080 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:32:05.126 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T06:32:05.147 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T06:32:05.148 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:05.148 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-27T06:32:05.148 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T06:32:05.148 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T06:32:05.148 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:32:05.451 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T06:32:05.473 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T06:32:05.473 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:05.473 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-27T06:32:05.474 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T06:32:05.474 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T06:32:05.474 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:32:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T06:32:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T06:32:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T06:32:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 144/198 2024-03-27T06:32:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T06:32:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T06:32:05.924 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:05.924 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-27T06:32:05.924 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T06:32:05.924 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T06:32:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T06:32:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T06:32:06.031 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 147/198 2024-03-27T06:32:06.102 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T06:32:06.127 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 134/198 2024-03-27T06:32:06.127 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:06.127 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-27T06:32:06.127 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T06:32:06.127 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-27T06:32:06.127 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:06.344 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T06:32:06.366 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 135/198 2024-03-27T06:32:06.367 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:06.367 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-27T06:32:06.367 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T06:32:06.367 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-27T06:32:06.367 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:06.812 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T06:32:06.834 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 136/198 2024-03-27T06:32:06.834 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:06.834 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-27T06:32:06.834 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T06:32:06.834 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-27T06:32:06.835 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-27T06:32:06.956 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-27T06:32:06.987 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T06:32:07.009 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T06:32:07.010 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:07.010 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-27T06:32:07.010 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T06:32:07.010 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T06:32:07.010 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:07.055 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T06:32:07.076 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T06:32:07.077 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:07.077 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-27T06:32:07.077 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:07.122 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T06:32:07.145 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T06:32:07.145 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:07.145 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-27T06:32:07.145 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T06:32:07.145 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T06:32:07.145 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:07.289 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T06:32:07.312 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 138/198 2024-03-27T06:32:07.344 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 139/198 2024-03-27T06:32:07.405 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 140/198 2024-03-27T06:32:07.435 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T06:32:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T06:32:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T06:32:07.487 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:07.487 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-27T06:32:07.487 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T06:32:07.487 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T06:32:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:08.480 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 141/198 2024-03-27T06:32:08.512 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T06:32:09.477 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 142/198 2024-03-27T06:32:09.504 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T06:32:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 143/198 2024-03-27T06:32:09.766 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 144/198 2024-03-27T06:32:09.789 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T06:32:09.811 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 145/198 2024-03-27T06:32:09.811 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:09.811 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-27T06:32:09.811 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T06:32:09.811 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-27T06:32:09.811 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:09.848 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T06:32:09.859 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 146/198 2024-03-27T06:32:09.904 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 147/198 2024-03-27T06:32:09.914 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T06:32:09.955 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 154/198 2024-03-27T06:32:09.986 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 155/198 2024-03-27T06:32:10.013 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-27T06:32:10.194 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 157/198 2024-03-27T06:32:10.313 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-27T06:32:10.361 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 159/198 2024-03-27T06:32:10.400 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-27T06:32:10.462 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-27T06:32:10.505 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-27T06:32:10.538 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 163/198 2024-03-27T06:32:10.578 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-27T06:32:10.608 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-27T06:32:10.654 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-27T06:32:10.707 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-27T06:32:10.732 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-27T06:32:10.775 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-27T06:32:10.809 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T06:32:10.830 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 149/198 2024-03-27T06:32:10.830 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:10.830 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-27T06:32:10.831 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T06:32:10.831 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-27T06:32:10.831 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:10.867 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T06:32:10.890 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2478.g15526 150/198 2024-03-27T06:32:10.890 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:10.890 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-27T06:32:10.890 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:10.935 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T06:32:10.957 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T06:32:10.957 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:10.957 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-27T06:32:10.957 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T06:32:10.958 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-27T06:32:10.958 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:11.261 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T06:32:11.283 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 152/198 2024-03-27T06:32:11.283 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-27T06:32:11.283 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-27T06:32:11.283 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T06:32:11.283 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-27T06:32:11.283 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:11.310 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T06:32:11.347 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T06:32:11.366 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T06:32:12.176 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 170/198 2024-03-27T06:32:12.195 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T06:32:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 154/198 2024-03-27T06:32:12.259 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-27T06:32:12.267 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 155/198 2024-03-27T06:32:12.300 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-27T06:32:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-27T06:32:12.381 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-27T06:32:12.421 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-27T06:32:12.446 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-27T06:32:12.472 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-27T06:32:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 157/198 2024-03-27T06:32:12.498 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-27T06:32:12.498 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T06:32:12.578 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T06:32:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-27T06:32:12.639 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-27T06:32:12.639 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T06:32:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 159/198 2024-03-27T06:32:12.684 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T06:32:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-27T06:32:12.710 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-27T06:32:12.744 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-27T06:32:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-27T06:32:12.769 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-27T06:32:12.794 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-27T06:32:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-27T06:32:12.827 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-27T06:32:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 163/198 2024-03-27T06:32:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-27T06:32:12.866 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-27T06:32:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-27T06:32:12.898 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-27T06:32:12.924 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-27T06:32:12.949 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-27T06:32:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-27T06:32:12.974 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-27T06:32:13.000 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-27T06:32:13.006 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T06:32:13.006 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T06:32:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-27T06:32:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-27T06:32:13.186 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T06:32:13.186 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T06:32:13.278 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T06:32:13.302 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-27T06:32:13.335 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-27T06:32:13.362 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-27T06:32:13.396 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-27T06:32:13.459 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T06:32:13.657 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T06:32:13.694 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T06:32:13.712 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T06:32:14.581 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 170/198 2024-03-27T06:32:14.664 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-27T06:32:14.706 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-27T06:32:14.798 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-27T06:32:14.834 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-27T06:32:14.859 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-27T06:32:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-27T06:32:14.911 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-27T06:32:14.911 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T06:32:15.000 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T06:32:15.029 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-27T06:32:15.029 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T06:32:15.122 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T06:32:15.149 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-27T06:32:15.182 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-27T06:32:15.207 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-27T06:32:15.232 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-27T06:32:15.269 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-27T06:32:15.321 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-27T06:32:15.356 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-27T06:32:15.381 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-27T06:32:15.406 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-27T06:32:15.432 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-27T06:32:15.457 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-27T06:32:15.464 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T06:32:15.464 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T06:32:15.643 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T06:32:15.643 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T06:32:15.709 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T06:32:15.726 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-27T06:32:15.749 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T06:32:15.758 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-27T06:32:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-27T06:32:15.789 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 154/198 2024-03-27T06:32:15.810 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-27T06:32:15.820 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 155/198 2024-03-27T06:32:15.847 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-27T06:32:15.879 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T06:32:16.032 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 157/198 2024-03-27T06:32:16.147 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-27T06:32:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 159/198 2024-03-27T06:32:16.227 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-27T06:32:16.279 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-27T06:32:16.314 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-27T06:32:16.338 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 163/198 2024-03-27T06:32:16.371 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-27T06:32:16.400 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-27T06:32:16.446 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-27T06:32:16.508 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-27T06:32:16.540 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-27T06:32:17.132 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T06:32:17.174 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T06:32:17.193 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-27T06:32:18.021 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 170/198 2024-03-27T06:32:18.111 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-27T06:32:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-27T06:32:18.234 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-27T06:32:18.265 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-27T06:32:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-27T06:32:18.315 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-27T06:32:18.342 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-27T06:32:18.342 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T06:32:18.421 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-27T06:32:18.451 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-27T06:32:18.451 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T06:32:18.536 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-27T06:32:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-27T06:32:18.604 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-27T06:32:18.629 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-27T06:32:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-27T06:32:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-27T06:32:18.726 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-27T06:32:18.759 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-27T06:32:18.784 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-27T06:32:18.809 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-27T06:32:18.834 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-27T06:32:18.868 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-27T06:32:18.875 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T06:32:18.875 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T06:32:18.897 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T06:32:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 1/198 2024-03-27T06:32:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2/198 2024-03-27T06:32:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 3/198 2024-03-27T06:32:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 4/198 2024-03-27T06:32:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2478.g15526 5/198 2024-03-27T06:32:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 6/198 2024-03-27T06:32:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 7/198 2024-03-27T06:32:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 8/198 2024-03-27T06:32:18.898 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 9/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 13/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 14/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 15/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 17/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 18/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 19/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 20/198 2024-03-27T06:32:18.899 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 21/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 22/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 23/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 24/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 25/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 26/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 27/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 28/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 29/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 30/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 31/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 32/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 33/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 34/198 2024-03-27T06:32:18.900 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 35/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 36/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 37/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 38/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 39/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-27T06:32:18.901 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-27T06:32:18.902 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-27T06:32:18.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-27T06:32:18.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-27T06:32:18.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-27T06:32:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-27T06:32:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-27T06:32:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-27T06:32:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-27T06:32:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-27T06:32:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-27T06:32:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-27T06:32:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-27T06:32:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-27T06:32:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-27T06:32:18.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-27T06:32:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-27T06:32:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-27T06:32:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-27T06:32:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-27T06:32:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-27T06:32:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-27T06:32:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-27T06:32:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-27T06:32:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-27T06:32:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-27T06:32:18.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-27T06:32:18.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-27T06:32:18.909 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-27T06:32:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-27T06:32:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-27T06:32:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-27T06:32:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T06:32:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-27T06:32:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T06:32:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-27T06:32:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-27T06:32:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-27T06:32:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-27T06:32:18.910 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-27T06:32:18.911 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-27T06:32:18.912 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-27T06:32:18.913 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-27T06:32:18.914 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-27T06:32:19.055 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-27T06:32:19.055 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T06:32:19.138 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-27T06:32:19.162 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-27T06:32:19.203 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-27T06:32:19.230 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-27T06:32:19.272 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-27T06:32:19.337 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T06:32:21.327 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T06:32:21.327 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 1/198 2024-03-27T06:32:21.327 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2/198 2024-03-27T06:32:21.327 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 3/198 2024-03-27T06:32:21.327 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 4/198 2024-03-27T06:32:21.327 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2478.g15526 5/198 2024-03-27T06:32:21.327 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 6/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 7/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 8/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 9/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 13/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 14/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 15/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 17/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 18/198 2024-03-27T06:32:21.328 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 19/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 20/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 21/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 22/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 23/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 24/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 25/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 26/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 27/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 28/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 29/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 30/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 31/198 2024-03-27T06:32:21.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 32/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 33/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 34/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 35/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 36/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 37/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 38/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 39/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-27T06:32:21.330 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-27T06:32:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-27T06:32:21.332 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-27T06:32:21.333 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-27T06:32:21.334 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-27T06:32:21.335 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-27T06:32:21.336 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-27T06:32:21.337 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-27T06:32:21.338 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-27T06:32:21.339 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-27T06:32:21.340 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-27T06:32:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-27T06:32:21.342 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-27T06:32:21.342 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-27T06:32:21.342 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-27T06:32:21.342 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-27T06:32:21.342 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-27T06:32:21.342 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-27T06:32:21.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-27T06:32:21.619 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:32:21.619 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2024-03-27T06:32:21.619 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:21.619 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: acl-2.3.1-4.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-27T06:32:21.620 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2024-03-27T06:32:21.621 INFO:teuthology.orchestra.run.smithi156.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-27T06:32:21.622 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.2-8.el9.noarch 2024-03-27T06:32:21.622 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.7.4-5.el9.x86_64 2024-03-27T06:32:21.622 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-27T06:32:21.622 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.622 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.622 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.622 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.622 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:21.622 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.623 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.4-9.el9.x86_64 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-27T06:32:21.624 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-16.el9.x86_64 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-27T06:32:21.625 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-27T06:32:21.626 INFO:teuthology.orchestra.run.smithi156.stdout: lua-5.4.4-4.el9.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-13.14-1.el9.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-27T06:32:21.627 INFO:teuthology.orchestra.run.smithi156.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-27T06:32:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-27T06:32:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-27T06:32:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-27T06:32:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-27T06:32:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-27T06:32:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-27T06:32:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-27T06:32:21.628 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-27T06:32:21.629 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-27T06:32:21.630 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-27T06:32:21.630 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-27T06:32:21.630 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-27T06:32:21.630 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-27T06:32:21.630 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-27T06:32:21.630 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-27T06:32:21.630 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-27T06:32:21.630 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-27T06:32:21.630 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-27T06:32:21.630 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-27T06:32:21.630 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply-3.11-14.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-27T06:32:21.631 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-27T06:32:21.632 INFO:teuthology.orchestra.run.smithi156.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: re2-1:20211101-3.el9.x86_64 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-27T06:32:21.633 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-27T06:32:21.634 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-27T06:32:21.634 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-27T06:32:21.634 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-27T06:32:21.634 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-27T06:32:21.634 INFO:teuthology.orchestra.run.smithi156.stdout: 2024-03-27T06:32:21.634 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2024-03-27T06:32:22.071 DEBUG:teuthology.parallel:result is None 2024-03-27T06:32:24.356 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout: acl-2.3.1-4.el9.x86_64 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-27T06:32:24.357 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-27T06:32:24.358 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.360 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-16.el9.x86_64 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-27T06:32:24.361 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-27T06:32:24.362 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.14-1.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-27T06:32:24.363 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-27T06:32:24.364 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-27T06:32:24.365 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-27T06:32:24.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-27T06:32:24.367 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-27T06:32:24.368 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-27T06:32:24.369 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-27T06:32:24.369 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-27T06:32:24.369 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.369 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.369 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:24.369 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-3.el9.x86_64 2024-03-27T06:32:24.369 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-27T06:32:24.369 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-27T06:32:24.369 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-27T06:32:24.369 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-27T06:32:24.369 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-27T06:32:24.370 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-27T06:32:24.370 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:32:24.370 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-03-27T06:32:24.822 DEBUG:teuthology.parallel:result is None 2024-03-27T06:32:24.833 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-27T06:32:24.833 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:19.0.0-2478.g155268c4.el9.x86_64 1/198 2024-03-27T06:32:24.833 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 3/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 4/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2478.g15526 5/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 6/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 7/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 8/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 9/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 10/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 11/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 12/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 13/198 2024-03-27T06:32:24.834 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 14/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_ 15/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 16/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 17/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:19.0.0-2478.g155268c4.el9.x86_6 18/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86 19/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 20/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2478.g155268c4.el 21/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:19.0.0-2478.g155268c4.el9. 22/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_6 23/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 24/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 25/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 26/198 2024-03-27T06:32:24.835 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 27/198 2024-03-27T06:32:24.851 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 28/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 29/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2478.g155268c4. 30/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noa 31/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.n 32/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2478.g155 33/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el 34/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 35/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.e 36/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 37/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:19.0.0-2478.g155268c4.el9.noarch 38/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 39/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-27T06:32:24.852 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-27T06:32:24.853 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-27T06:32:24.854 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-27T06:32:24.855 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-27T06:32:24.856 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-27T06:32:24.857 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-27T06:32:24.858 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-27T06:32:24.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:19.0.0-2478.g155268c4.el9.x86_64 151/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 153/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-27T06:32:24.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-27T06:32:24.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-27T06:32:24.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-27T06:32:24.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-27T06:32:27.560 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-27T06:32:27.560 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: acl-2.3.1-4.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-27T06:32:27.561 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-27T06:32:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.2-8.el9.noarch 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.7.4-5.el9.x86_64 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:27.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.564 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:19.0.0-2478.g155268c4.el9.noarch 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.4-9.el9.x86_64 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: jq-1.6-16.el9.x86_64 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.565 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-27T06:32:27.566 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: lua-5.4.4-4.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-13.14-1.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-27T06:32:27.567 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-27T06:32:27.568 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-27T06:32:27.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply-3.11-14.el9.noarch 2024-03-27T06:32:27.570 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-27T06:32:27.571 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-27T06:32:27.572 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-27T06:32:27.573 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-27T06:32:27.573 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.573 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.573 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:19.0.0-2478.g155268c4.el9.x86_64 2024-03-27T06:32:27.573 INFO:teuthology.orchestra.run.smithi170.stdout: re2-1:20211101-3.el9.x86_64 2024-03-27T06:32:27.573 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-27T06:32:27.573 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-27T06:32:27.573 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-27T06:32:27.573 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-27T06:32:27.573 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-27T06:32:27.574 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-27T06:32:27.574 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:32:27.574 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-03-27T06:32:28.039 DEBUG:teuthology.parallel:result is None 2024-03-27T06:32:28.040 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:32:28.254 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-27T06:32:28.279 INFO:teuthology.orchestra.run.smithi016.stdout:19.0.0-2478.g155268c4.el9 2024-03-27T06:32:28.280 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2478.g155268c4.el9 2024-03-27T06:32:28.280 INFO:teuthology.task.install:The correct ceph version 19.0.0-2478.g155268c4 is installed. 2024-03-27T06:32:28.282 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:32:28.421 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-27T06:32:28.447 INFO:teuthology.orchestra.run.smithi156.stdout:19.0.0-2478.g155268c4.el9 2024-03-27T06:32:28.447 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2478.g155268c4.el9 2024-03-27T06:32:28.447 INFO:teuthology.task.install:The correct ceph version 19.0.0-2478.g155268c4 is installed. 2024-03-27T06:32:28.449 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:32:28.586 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-27T06:32:28.609 INFO:teuthology.orchestra.run.smithi170.stdout:19.0.0-2478.g155268c4.el9 2024-03-27T06:32:28.609 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2478.g155268c4.el9 2024-03-27T06:32:28.609 INFO:teuthology.task.install:The correct ceph version 19.0.0-2478.g155268c4 is installed. 2024-03-27T06:32:28.611 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-27T06:32:28.611 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:32:28.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-27T06:32:28.645 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-27T06:32:28.645 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-27T06:32:28.679 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-27T06:32:28.679 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-27T06:32:28.711 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-27T06:32:28.711 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:32:28.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-03-27T06:32:28.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-27T06:32:28.811 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-27T06:32:28.811 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2024-03-27T06:32:28.841 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-27T06:32:28.909 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-27T06:32:28.909 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-03-27T06:32:28.938 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-27T06:32:29.007 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-27T06:32:29.007 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:32:29.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-27T06:32:29.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-27T06:32:29.105 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-27T06:32:29.105 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-27T06:32:29.133 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-27T06:32:29.201 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-27T06:32:29.201 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-27T06:32:29.229 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-27T06:32:29.297 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-27T06:32:29.298 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:32:29.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-03-27T06:32:29.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-27T06:32:29.400 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-27T06:32:29.400 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2024-03-27T06:32:29.427 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-27T06:32:29.494 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-27T06:32:29.494 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-03-27T06:32:29.523 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-27T06:32:29.592 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-27T06:32:29.694 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-27T06:32:29.695 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:32:29.695 INFO:tasks.cephadm:Cluster fsid is c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:32:29.695 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-27T06:32:29.695 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.16', 'mon.b': '172.21.15.156', 'mon.c': '172.21.15.170'} 2024-03-27T06:32:29.695 INFO:tasks.cephadm:First mon is mon.a on smithi016 2024-03-27T06:32:29.696 INFO:tasks.cephadm:First mgr is x 2024-03-27T06:32:29.696 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-27T06:32:29.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-03-27T06:32:29.727 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname $(hostname -s) 2024-03-27T06:32:29.755 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2024-03-27T06:32:29.783 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-27T06:32:29.784 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:32:30.000 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '155268c4e432a12433aa833f174f9fe3b1016ae0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77988/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2478-g155268c4', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2478.g155268c4'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-26 19:55:12.154042', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240326.131558', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-27T06:32:30.140 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-rishabh-testing-20240326.131558, sha1 155268c4e432a12433aa833f174f9fe3b1016ae0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:32:30.142 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/x86_64/flavors/default/cephadm 2024-03-27T06:32:30.142 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-27T06:32:30.756 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 781299 Mar 27 06:32 /home/ubuntu/cephtest/cephadm 2024-03-27T06:32:30.756 DEBUG:teuthology.orchestra.run.smithi156:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-27T06:32:31.184 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-r--r--. 1 ubuntu ubuntu 781299 Mar 27 06:32 /home/ubuntu/cephtest/cephadm 2024-03-27T06:32:31.184 DEBUG:teuthology.orchestra.run.smithi170:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240326.131558/155268c4e432a12433aa833f174f9fe3b1016ae0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-27T06:32:31.759 INFO:teuthology.orchestra.run.smithi170.stdout:-rw-r--r--. 1 ubuntu ubuntu 781299 Mar 27 06:32 /home/ubuntu/cephtest/cephadm 2024-03-27T06:32:31.759 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-27T06:32:31.776 DEBUG:teuthology.orchestra.run.smithi156:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-27T06:32:31.794 DEBUG:teuthology.orchestra.run.smithi170:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-27T06:32:31.829 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 on all hosts... 2024-03-27T06:32:31.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 pull 2024-03-27T06:32:31.832 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 pull 2024-03-27T06:32:31.837 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 pull 2024-03-27T06:32:32.651 INFO:teuthology.orchestra.run.smithi156.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0... 2024-03-27T06:32:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0... 2024-03-27T06:32:32.854 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0... 2024-03-27T06:33:07.505 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-03-27T06:33:07.506 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev)", 2024-03-27T06:33:07.506 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801", 2024-03-27T06:33:07.506 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2024-03-27T06:33:07.506 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772" 2024-03-27T06:33:07.506 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-03-27T06:33:07.506 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-03-27T06:33:10.349 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-27T06:33:10.350 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev)", 2024-03-27T06:33:10.350 INFO:teuthology.orchestra.run.smithi156.stdout: "image_id": "b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801", 2024-03-27T06:33:10.350 INFO:teuthology.orchestra.run.smithi156.stdout: "repo_digests": [ 2024-03-27T06:33:10.350 INFO:teuthology.orchestra.run.smithi156.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772" 2024-03-27T06:33:10.350 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2024-03-27T06:33:10.350 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-27T06:33:16.711 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:33:16.711 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev)", 2024-03-27T06:33:16.711 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801", 2024-03-27T06:33:16.711 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-03-27T06:33:16.711 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772" 2024-03-27T06:33:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:33:16.712 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:33:16.735 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-03-27T06:33:16.766 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph 2024-03-27T06:33:16.801 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2024-03-27T06:33:16.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-03-27T06:33:16.863 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /etc/ceph 2024-03-27T06:33:16.895 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2024-03-27T06:33:16.926 INFO:tasks.cephadm:Writing seed config... 2024-03-27T06:33:16.927 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-03-27T06:33:16.927 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-27T06:33:16.927 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-27T06:33:16.928 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-27T06:33:16.929 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-27T06:33:16.929 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-27T06:33:16.929 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-27T06:33:16.929 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-27T06:33:16.929 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-27T06:33:16.929 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-27T06:33:16.929 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-27T06:33:16.929 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-03-27T06:33:16.929 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-27T06:33:16.929 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-27T06:33:16.930 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-27T06:33:16.931 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-27T06:33:16.931 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-27T06:33:16.931 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-27T06:33:16.931 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-27T06:33:16.931 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-27T06:33:16.931 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-27T06:33:16.931 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-27T06:33:16.932 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:33:16.932 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-27T06:33:16.946 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c9095bc4-ec03-11ee-95d0-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-27T06:33:16.947 DEBUG:teuthology.orchestra.run.smithi016:mon.a> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.a.service 2024-03-27T06:33:16.989 DEBUG:teuthology.orchestra.run.smithi016:mgr.x> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.x.service 2024-03-27T06:33:17.032 INFO:tasks.cephadm:Bootstrapping... 2024-03-27T06:33:17.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 -v bootstrap --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-27T06:33:17.228 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-03-27T06:33:17.228 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0', '-v', 'bootstrap', '--fsid', 'c9095bc4-ec03-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-03-27T06:33:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-27T06:33:17.229 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-03-27T06:33:17.254 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-27T06:33:17.254 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-03-27T06:33:17.254 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-03-27T06:33:17.261 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-27T06:33:17.261 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-27T06:33:17.267 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-27T06:33:17.267 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-03-27T06:33:17.273 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-03-27T06:33:17.279 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-03-27T06:33:17.280 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-03-27T06:33:17.280 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-03-27T06:33:17.302 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-27T06:33:17.302 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-27T06:33:17.302 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-03-27T06:33:17.303 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-03-27T06:33:17.309 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-27T06:33:17.309 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-27T06:33:17.314 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-27T06:33:17.314 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-03-27T06:33:17.320 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-03-27T06:33:17.327 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-03-27T06:33:17.327 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-03-27T06:33:17.327 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-03-27T06:33:17.327 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:33:17.327 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 140430415688368 on /run/cephadm/c9095bc4-ec03-11ee-95d0-87774f69a715.lock 2024-03-27T06:33:17.327 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 140430415688368 acquired on /run/cephadm/c9095bc4-ec03-11ee-95d0-87774f69a715.lock 2024-03-27T06:33:17.327 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-03-27T06:33:17.328 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-03-27T06:33:17.328 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-03-27T06:33:17.331 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.16 metric 100 2024-03-27T06:33:17.331 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-03-27T06:33:17.334 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-27T06:33:17.334 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-27T06:33:17.334 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-27T06:33:17.337 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-27T06:33:17.337 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-27T06:33:17.337 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-27T06:33:17.338 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-27T06:33:17.338 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-03-27T06:33:17.338 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-27T06:33:17.338 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-03-27T06:33:17.338 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-03-27T06:33:17.339 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-27T06:33:17.339 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-27T06:33:17.339 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0... 2024-03-27T06:33:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 2024-03-27T06:33:17.755 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0... 2024-03-27T06:33:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-03-27T06:33:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:e605917547793151c989dc0ed66d07767a9fed5a2365fd05914b705e7a7b6e21 2024-03-27T06:33:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:11da15500eca1f826c328eb3265caa30b0a3730510e2a32023084d6fe9cee20c 2024-03-27T06:33:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 2024-03-27T06:33:17.756 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-27T06:33:18.694 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev) 2024-03-27T06:33:18.694 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev) 2024-03-27T06:33:18.694 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-03-27T06:33:19.558 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-03-27T06:33:19.558 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-03-27T06:33:20.454 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQCvvQNmVcCjNhAAPeR4bIIsFpmVoU+0VWUpJg== 2024-03-27T06:33:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQCwvQNmjD4XMhAAsXDQExbmL3DI+cJlZV0w8Q== 2024-03-27T06:33:22.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQCxvQNmqk/gLRAA3YDHsRplfuNehGluqaFkyw== 2024-03-27T06:33:22.285 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-03-27T06:33:23.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-27T06:33:23.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-27T06:33:23.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:33:23.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-27T06:33:23.165 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for a [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-27T06:33:23.165 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-03-27T06:33:23.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:33:23.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-27T06:33:23.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:33:23.165 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-03-27T06:33:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.a on 2024-03-27T06:33:24.289 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-27T06:33:24.425 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-27T06:33:24.594 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c9095bc4-ec03-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-c9095bc4-ec03-11ee-95d0-87774f69a715.target. 2024-03-27T06:33:24.594 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c9095bc4-ec03-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-c9095bc4-ec03-11ee-95d0-87774f69a715.target. 2024-03-27T06:33:24.830 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.a 2024-03-27T06:33:24.830 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-27T06:33:24.989 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c9095bc4-ec03-11ee-95d0-87774f69a715.target.wants/ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@.service. 2024-03-27T06:33:25.549 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-27T06:33:25.549 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-03-27T06:33:25.557 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-27T06:33:25.557 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-03-27T06:33:25.557 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-03-27T06:33:25.557 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-03-27T06:33:25.558 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-03-27T06:33:25.558 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-03-27T06:33:26.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-03-27T06:33:26.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:33:26.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-27T06:33:26.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-03-27T06:33:26.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-03-27T06:33:26.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.544742s) 2024-03-27T06:33:26.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-27T06:33:26.656 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.995+0000 7fdf4f9d8700 1 Processor -- start 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.995+0000 7fdf4f9d8700 1 -- start start 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.995+0000 7fdf4f9d8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4810b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.995+0000 7fdf4f9d8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4807be70 con 0x7fdf48108da0 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.996+0000 7fdf4d774700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4810b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.996+0000 7fdf4d774700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4810b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44422/0 (socket says 172.21.15.16:44422) 2024-03-27T06:33:26.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.996+0000 7fdf4d774700 1 -- 172.21.15.16:0/201010683 learned_addr learned my addr 172.21.15.16:0/201010683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.996+0000 7fdf4d774700 1 -- 172.21.15.16:0/201010683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4810b6d0 con 0x7fdf48108da0 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.996+0000 7fdf4d774700 1 --2- 172.21.15.16:0/201010683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4810b190 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf440098a0 tx=0x7fdf44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc95b3a9be390dd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.996+0000 7fdf3ffff700 1 -- 172.21.15.16:0/201010683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf44004160 con 0x7fdf48108da0 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.996+0000 7fdf3ffff700 1 -- 172.21.15.16:0/201010683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf44004300 con 0x7fdf48108da0 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.997+0000 7fdf3ffff700 1 -- 172.21.15.16:0/201010683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf440036a0 con 0x7fdf48108da0 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.997+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/201010683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 msgr2=0x7fdf4810b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.997+0000 7fdf4f9d8700 1 --2- 172.21.15.16:0/201010683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4810b190 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf440098a0 tx=0x7fdf44009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.997+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/201010683 shutdown_connections 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.997+0000 7fdf4f9d8700 1 --2- 172.21.15.16:0/201010683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4810b190 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.997+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/201010683 >> 172.21.15.16:0/201010683 conn(0x7fdf481027b0 msgr2=0x7fdf48104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.998+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/201010683 shutdown_connections 2024-03-27T06:33:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.998+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/201010683 wait complete. 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.998+0000 7fdf4f9d8700 1 Processor -- start 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.998+0000 7fdf4f9d8700 1 -- start start 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.999+0000 7fdf4f9d8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.999+0000 7fdf4f9d8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4810bb40 con 0x7fdf48108da0 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.999+0000 7fdf4d774700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.999+0000 7fdf4d774700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4819ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44430/0 (socket says 172.21.15.16:44430) 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.999+0000 7fdf4d774700 1 -- 172.21.15.16:0/2297784960 learned_addr learned my addr 172.21.15.16:0/2297784960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.999+0000 7fdf4d774700 1 -- 172.21.15.16:0/2297784960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4819bf40 con 0x7fdf48108da0 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.999+0000 7fdf4d774700 1 --2- 172.21.15.16:0/2297784960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4819ba00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009fb0 tx=0x7fdf44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.999+0000 7fdf3e7fc700 1 -- 172.21.15.16:0/2297784960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf44003de0 con 0x7fdf48108da0 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.999+0000 7fdf3e7fc700 1 -- 172.21.15.16:0/2297784960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf44011630 con 0x7fdf48108da0 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:25.999+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/2297784960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4819c1d0 con 0x7fdf48108da0 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.000+0000 7fdf3e7fc700 1 -- 172.21.15.16:0/2297784960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf44011930 con 0x7fdf48108da0 2024-03-27T06:33:26.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.000+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/2297784960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf4819eec0 con 0x7fdf48108da0 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.000+0000 7fdf3e7fc700 1 -- 172.21.15.16:0/2297784960 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf44024480 con 0x7fdf48108da0 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.000+0000 7fdf3e7fc700 1 -- 172.21.15.16:0/2297784960 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdf4401ad70 con 0x7fdf48108da0 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.001+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/2297784960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf48063c70 con 0x7fdf48108da0 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.003+0000 7fdf3e7fc700 1 -- 172.21.15.16:0/2297784960 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdf4401a460 con 0x7fdf48108da0 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.076+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/2297784960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fdf4802dfc0 con 0x7fdf48108da0 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.077+0000 7fdf3e7fc700 1 -- 172.21.15.16:0/2297784960 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fdf44032150 con 0x7fdf48108da0 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.078+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/2297784960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 msgr2=0x7fdf4819ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.078+0000 7fdf4f9d8700 1 --2- 172.21.15.16:0/2297784960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4819ba00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009fb0 tx=0x7fdf44004750 comp rx=0 tx=0).stop 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.078+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/2297784960 shutdown_connections 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.078+0000 7fdf4f9d8700 1 --2- 172.21.15.16:0/2297784960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf48108da0 0x7fdf4819ba00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.078+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/2297784960 >> 172.21.15.16:0/2297784960 conn(0x7fdf481027b0 msgr2=0x7fdf48104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.078+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/2297784960 shutdown_connections 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:26.078+0000 7fdf4f9d8700 1 -- 172.21.15.16:0/2297784960 wait complete. 2024-03-27T06:33:26.660 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-03-27T06:33:26.661 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-03-27T06:33:27.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:27 smithi016 ceph-mon[31524]: from='client.? 172.21.15.16:0/3682811350' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-27T06:33:27.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-03-27T06:33:27.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-03-27T06:33:27.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:33:27.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-03-27T06:33:27.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-27T06:33:27.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-27T06:33:27.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.091+0000 7f38328d2700 1 Processor -- start 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.091+0000 7f38328d2700 1 -- start start 2024-03-27T06:33:27.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.091+0000 7f38328d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c107620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.091+0000 7f38328d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f382c107bf0 con 0x7f382c107200 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.091+0000 7f382bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c107620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.091+0000 7f382bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c107620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44442/0 (socket says 172.21.15.16:44442) 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.091+0000 7f382bfff700 1 -- 172.21.15.16:0/3611424693 learned_addr learned my addr 172.21.15.16:0/3611424693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.091+0000 7f382bfff700 1 -- 172.21.15.16:0/3611424693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c108400 con 0x7f382c107200 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.091+0000 7f382bfff700 1 --2- 172.21.15.16:0/3611424693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c107620 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f381c0098a0 tx=0x7f381c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6fd37c4f8e4693 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.092+0000 7f382affd700 1 -- 172.21.15.16:0/3611424693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f381c004160 con 0x7f382c107200 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.092+0000 7f382affd700 1 -- 172.21.15.16:0/3611424693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381c004300 con 0x7f382c107200 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.092+0000 7f382affd700 1 -- 172.21.15.16:0/3611424693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f381c0036a0 con 0x7f382c107200 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.092+0000 7f38328d2700 1 -- 172.21.15.16:0/3611424693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 msgr2=0x7f382c107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.092+0000 7f38328d2700 1 --2- 172.21.15.16:0/3611424693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c107620 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f381c0098a0 tx=0x7f381c009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.092+0000 7f38328d2700 1 -- 172.21.15.16:0/3611424693 shutdown_connections 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.092+0000 7f38328d2700 1 --2- 172.21.15.16:0/3611424693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c107620 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:27.673 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.092+0000 7f38328d2700 1 -- 172.21.15.16:0/3611424693 >> 172.21.15.16:0/3611424693 conn(0x7f382c102780 msgr2=0x7f382c104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.092+0000 7f38328d2700 1 -- 172.21.15.16:0/3611424693 shutdown_connections 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.092+0000 7f38328d2700 1 -- 172.21.15.16:0/3611424693 wait complete. 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.093+0000 7f38328d2700 1 Processor -- start 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.093+0000 7f38328d2700 1 -- start start 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.093+0000 7f38328d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.093+0000 7f38328d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f382c108930 con 0x7f382c107200 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.093+0000 7f382bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.093+0000 7f382bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44444/0 (socket says 172.21.15.16:44444) 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.093+0000 7f382bfff700 1 -- 172.21.15.16:0/3682811350 learned_addr learned my addr 172.21.15.16:0/3682811350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.094+0000 7f382bfff700 1 -- 172.21.15.16:0/3682811350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c19bf90 con 0x7f382c107200 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.094+0000 7f382bfff700 1 --2- 172.21.15.16:0/3682811350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c19ba50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f381c000c00 tx=0x7f381c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.094+0000 7f38297fa700 1 -- 172.21.15.16:0/3682811350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f381c003db0 con 0x7f382c107200 2024-03-27T06:33:27.674 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.094+0000 7f38297fa700 1 -- 172.21.15.16:0/3682811350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381c011630 con 0x7f382c107200 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.094+0000 7f38328d2700 1 -- 172.21.15.16:0/3682811350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382c19c220 con 0x7f382c107200 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.094+0000 7f38297fa700 1 -- 172.21.15.16:0/3682811350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f381c011950 con 0x7f382c107200 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.094+0000 7f38328d2700 1 -- 172.21.15.16:0/3682811350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f382c19ef10 con 0x7f382c107200 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.094+0000 7f38297fa700 1 -- 172.21.15.16:0/3682811350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f381c01a460 con 0x7f382c107200 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.094+0000 7f38297fa700 1 -- 172.21.15.16:0/3682811350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f381c01b780 con 0x7f382c107200 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.095+0000 7f38328d2700 1 -- 172.21.15.16:0/3682811350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f382c063c70 con 0x7f382c107200 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.097+0000 7f38297fa700 1 -- 172.21.15.16:0/3682811350 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f381c031020 con 0x7f382c107200 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.168+0000 7f38328d2700 1 -- 172.21.15.16:0/3682811350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f382c19c700 con 0x7f382c107200 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.171+0000 7f38297fa700 1 -- 172.21.15.16:0/3682811350 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f382c19c700 con 0x7f382c107200 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.172+0000 7f38297fa700 1 -- 172.21.15.16:0/3682811350 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f381c01a890 con 0x7f382c107200 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.173+0000 7f38328d2700 1 -- 172.21.15.16:0/3682811350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 msgr2=0x7f382c19ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.173+0000 7f38328d2700 1 --2- 172.21.15.16:0/3682811350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c19ba50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f381c000c00 tx=0x7f381c004750 comp rx=0 tx=0).stop 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.173+0000 7f38328d2700 1 -- 172.21.15.16:0/3682811350 shutdown_connections 2024-03-27T06:33:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.173+0000 7f38328d2700 1 --2- 172.21.15.16:0/3682811350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f382c107200 0x7f382c19ba50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:27.676 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.173+0000 7f38328d2700 1 -- 172.21.15.16:0/3682811350 >> 172.21.15.16:0/3682811350 conn(0x7f382c102780 msgr2=0x7f382c192650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:27.676 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.173+0000 7f38328d2700 1 -- 172.21.15.16:0/3682811350 shutdown_connections 2024-03-27T06:33:27.676 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:27.173+0000 7f38328d2700 1 -- 172.21.15.16:0/3682811350 wait complete. 2024-03-27T06:33:27.676 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-03-27T06:33:28.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.127+0000 7efece33f700 1 Processor -- start 2024-03-27T06:33:28.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.127+0000 7efece33f700 1 -- start start 2024-03-27T06:33:28.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.128+0000 7efece33f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec810b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:28.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.128+0000 7efece33f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efec807be70 con 0x7efec8108d60 2024-03-27T06:33:28.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.128+0000 7efec7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec810b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.128+0000 7efec7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec810b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44454/0 (socket says 172.21.15.16:44454) 2024-03-27T06:33:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.128+0000 7efec7fff700 1 -- 172.21.15.16:0/2330537896 learned_addr learned my addr 172.21.15.16:0/2330537896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.128+0000 7efec7fff700 1 -- 172.21.15.16:0/2330537896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec810b690 con 0x7efec8108d60 2024-03-27T06:33:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.128+0000 7efec7fff700 1 --2- 172.21.15.16:0/2330537896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec810b150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efeb80098a0 tx=0x7efeb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953d5fff0bdfa156 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.128+0000 7efec6ffd700 1 -- 172.21.15.16:0/2330537896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb8004160 con 0x7efec8108d60 2024-03-27T06:33:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.128+0000 7efec6ffd700 1 -- 172.21.15.16:0/2330537896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7efeb8004300 con 0x7efec8108d60 2024-03-27T06:33:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.128+0000 7efec6ffd700 1 -- 172.21.15.16:0/2330537896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb8011c00 con 0x7efec8108d60 2024-03-27T06:33:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.129+0000 7efece33f700 1 -- 172.21.15.16:0/2330537896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 msgr2=0x7efec810b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.129+0000 7efece33f700 1 --2- 172.21.15.16:0/2330537896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec810b150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efeb80098a0 tx=0x7efeb8009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.129+0000 7efece33f700 1 -- 172.21.15.16:0/2330537896 shutdown_connections 2024-03-27T06:33:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.129+0000 7efece33f700 1 --2- 172.21.15.16:0/2330537896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec810b150 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.129+0000 7efece33f700 1 -- 172.21.15.16:0/2330537896 >> 172.21.15.16:0/2330537896 conn(0x7efec8102790 msgr2=0x7efec8104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.129+0000 7efece33f700 1 -- 172.21.15.16:0/2330537896 shutdown_connections 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.129+0000 7efece33f700 1 -- 172.21.15.16:0/2330537896 wait complete. 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.130+0000 7efece33f700 1 Processor -- start 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.130+0000 7efece33f700 1 -- start start 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.130+0000 7efece33f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.130+0000 7efece33f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efec810c710 con 0x7efec8108d60 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.130+0000 7efec7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.130+0000 7efec7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec819c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44460/0 (socket says 172.21.15.16:44460) 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.130+0000 7efec7fff700 1 -- 172.21.15.16:0/609035380 learned_addr learned my addr 172.21.15.16:0/609035380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.130+0000 7efec7fff700 1 -- 172.21.15.16:0/609035380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec819c800 con 0x7efec8108d60 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.131+0000 7efec7fff700 1 --2- 172.21.15.16:0/609035380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec819c2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efeb8003fb0 tx=0x7efeb8011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.131+0000 7efec57fa700 1 -- 172.21.15.16:0/609035380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb80039c0 con 0x7efec8108d60 2024-03-27T06:33:28.711 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.131+0000 7efec57fa700 1 -- 172.21.15.16:0/609035380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7efeb8003b60 con 0x7efec8108d60 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.131+0000 7efece33f700 1 -- 172.21.15.16:0/609035380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec819ca90 con 0x7efec8108d60 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.131+0000 7efec57fa700 1 -- 172.21.15.16:0/609035380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeb801a6b0 con 0x7efec8108d60 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.131+0000 7efece33f700 1 -- 172.21.15.16:0/609035380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec819f780 con 0x7efec8108d60 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.131+0000 7efec57fa700 1 -- 172.21.15.16:0/609035380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efeb801ad10 con 0x7efec8108d60 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.131+0000 7efec57fa700 1 -- 172.21.15.16:0/609035380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efeb8019700 con 0x7efec8108d60 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.132+0000 7efece33f700 1 -- 172.21.15.16:0/609035380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec8063c70 con 0x7efec8108d60 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.134+0000 7efec57fa700 1 -- 172.21.15.16:0/609035380 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efeb8003d10 con 0x7efec8108d60 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.204+0000 7efece33f700 1 -- 172.21.15.16:0/609035380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efec819f950 con 0x7efec8108d60 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.205+0000 7efec57fa700 1 -- 172.21.15.16:0/609035380 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7efeb80249a0 con 0x7efec8108d60 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.206+0000 7efece33f700 1 -- 172.21.15.16:0/609035380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 msgr2=0x7efec819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.206+0000 7efece33f700 1 --2- 172.21.15.16:0/609035380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec819c2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efeb8003fb0 tx=0x7efeb8011bd0 comp rx=0 tx=0).stop 2024-03-27T06:33:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.206+0000 7efece33f700 1 -- 172.21.15.16:0/609035380 shutdown_connections 2024-03-27T06:33:28.713 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.206+0000 7efece33f700 1 --2- 172.21.15.16:0/609035380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec8108d60 0x7efec819c2c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:28.713 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.206+0000 7efece33f700 1 -- 172.21.15.16:0/609035380 >> 172.21.15.16:0/609035380 conn(0x7efec8102790 msgr2=0x7efec81032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:28.713 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.206+0000 7efece33f700 1 -- 172.21.15.16:0/609035380 shutdown_connections 2024-03-27T06:33:28.713 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:28.206+0000 7efece33f700 1 -- 172.21.15.16:0/609035380 wait complete. 2024-03-27T06:33:28.713 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-03-27T06:33:28.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:28 smithi016 ceph-mon[31524]: from='client.? 172.21.15.16:0/609035380' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:33:28.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:28 smithi016 systemd[1]: Stopping Ceph mon.a for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:33:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31519]: 2024-03-27T06:33:28.962+0000 7fd3d3856700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:33:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31519]: 2024-03-27T06:33:28.962+0000 7fd3d3856700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-27T06:33:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:29 smithi016 podman[31779]: 2024-03-27 06:33:29.178160841 +0000 UTC m=+0.302075521 container died cf1675ae7d488fe3842ec26a851b75d524c35dd8c581ec0de8204b9f2c232dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-27T06:33:29.697 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:29 smithi016 podman[31779]: 2024-03-27 06:33:29.520840895 +0000 UTC m=+0.644755573 container cleanup cf1675ae7d488fe3842ec26a851b75d524c35dd8c581ec0de8204b9f2c232dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:33:29.698 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:29 smithi016 bash[31779]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a 2024-03-27T06:33:30.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:29 smithi016 podman[31793]: 2024-03-27 06:33:29.704926118 +0000 UTC m=+0.522594350 container remove cf1675ae7d488fe3842ec26a851b75d524c35dd8c581ec0de8204b9f2c232dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:33:30.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:29 smithi016 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-27T06:33:30.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:29 smithi016 systemd[1]: Stopped Ceph mon.a for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:33:30.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:29 smithi016 systemd[1]: Starting Ceph mon.a for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:33:30.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 podman[31889]: 2024-03-27 06:33:30.056564406 +0000 UTC m=+0.017514031 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:33:30.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 podman[31889]: 2024-03-27 06:33:30.155817421 +0000 UTC m=+0.116767024 container create 06aedba2800884d1ea2407fd87a86eadf06a7409b67b48ac2dffa27527891498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:33:30.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 podman[31889]: 2024-03-27 06:33:30.341440716 +0000 UTC m=+0.302390323 container init 06aedba2800884d1ea2407fd87a86eadf06a7409b67b48ac2dffa27527891498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-03-27T06:33:30.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 podman[31889]: 2024-03-27 06:33:30.344158717 +0000 UTC m=+0.305108320 container start 06aedba2800884d1ea2407fd87a86eadf06a7409b67b48ac2dffa27527891498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-27T06:33:30.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: set uid:gid to 167:167 (ceph:ceph) 2024-03-27T06:33:30.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev), process ceph-mon, pid 2 2024-03-27T06:33:30.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: pidfile_write: ignore empty --pid-file 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: load: jerasure load: lrc 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: RocksDB version: 7.9.2 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Git sha 0 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Compile date 2024-03-26 18:55:08 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: DB SUMMARY 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: DB Session ID: XUGTGDT6OEXPUL4POZO6 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: CURRENT file: CURRENT 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: IDENTITY file: IDENTITY 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106740 ; 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.error_if_exists: 0 2024-03-27T06:33:30.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.create_if_missing: 0 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.paranoid_checks: 1 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.env: 0x564bf447bc60 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.fs: PosixFileSystem 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.info_log: 0x564bf6ad1ae0 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_file_opening_threads: 16 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.statistics: (nil) 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.use_fsync: 0 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_log_file_size: 0 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-27T06:33:30.414 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.keep_log_file_num: 1000 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.recycle_log_file_num: 0 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.allow_fallocate: 1 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.allow_mmap_reads: 0 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.allow_mmap_writes: 0 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.use_direct_reads: 0 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.create_missing_column_families: 0 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.db_log_dir: 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.wal_dir: 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-27T06:33:30.415 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-27T06:33:30.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-27T06:33:30.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.advise_random_on_open: 1 2024-03-27T06:33:30.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.db_write_buffer_size: 0 2024-03-27T06:33:30.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.write_buffer_manager: 0x564bf6aa21e0 2024-03-27T06:33:30.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-27T06:33:30.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-27T06:33:30.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-27T06:33:30.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.rate_limiter: (nil) 2024-03-27T06:33:30.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-27T06:33:30.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.wal_recovery_mode: 2 2024-03-27T06:33:30.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.enable_thread_tracking: 0 2024-03-27T06:33:30.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.enable_pipelined_write: 0 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.unordered_write: 0 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.row_cache: None 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.wal_filter: None 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.allow_ingest_behind: 0 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.two_write_queues: 0 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.manual_wal_flush: 0 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.wal_compression: 0 2024-03-27T06:33:30.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.atomic_flush: 0 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.log_readahead_size: 0 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.best_efforts_recovery: 0 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.allow_data_in_errors: 0 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.db_host_id: __hostname__ 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_background_jobs: 2 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_background_compactions: -1 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_subcompactions: 1 2024-03-27T06:33:30.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_total_wal_size: 0 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_open_files: -1 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bytes_per_sync: 0 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_readahead_size: 0 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_background_flushes: -1 2024-03-27T06:33:30.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Compression algorithms supported: 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: kZSTD supported: 0 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: kXpressCompression supported: 0 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: kLZ4HCCompression supported: 1 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: kZlibCompression supported: 1 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: kSnappyCompression supported: 1 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: kLZ4Compression supported: 1 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: kBZip2Compression supported: 0 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-27T06:33:30.421 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.merge_operator: 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_filter: None 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_filter_factory: None 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.sst_partitioner_factory: None 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564bf6ad1cc0) 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_type: 0 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_index_type: 0 2024-03-27T06:33:30.422 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_shortening: 1 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: checksum: 4 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: no_block_cache: 0 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache: 0x564bf6ad5090 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_options: 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: capacity : 536870912 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_shard_bits : 4 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: strict_capacity_limit : 0 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_compressed: (nil) 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: persistent_cache: (nil) 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size: 4096 2024-03-27T06:33:30.423 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size_deviation: 10 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_restart_interval: 16 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_block_restart_interval: 1 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: metadata_block_size: 4096 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: partition_filters: 0 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: use_delta_encoding: 1 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: filter_policy: bloomfilter 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: whole_key_filtering: 1 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: verify_compression: 0 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: format_version: 5 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_index_compression: 1 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_align: 0 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: max_auto_readahead_size: 262144 2024-03-27T06:33:30.424 INFO:journalctl@ceph.mon.a.smithi016.stdout: prepopulate_block_cache: 0 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout: initial_auto_readahead_size: 8192 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.write_buffer_size: 33554432 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_write_buffer_number: 2 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compression: NoCompression 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bottommost_compression: Disabled 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.prefix_extractor: nullptr 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.num_levels: 7 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-27T06:33:30.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compression_opts.level: 32767 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compression_opts.strategy: 0 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-27T06:33:30.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compression_opts.enabled: false 2024-03-27T06:33:30.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-27T06:33:30.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-27T06:33:30.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-27T06:33:30.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-27T06:33:30.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.target_file_size_base: 67108864 2024-03-27T06:33:30.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-27T06:33:30.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-27T06:33:30.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-27T06:33:30.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-27T06:33:30.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-27T06:33:30.429 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-27T06:33:30.429 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-27T06:33:30.429 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.arena_block_size: 1048576 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.disable_auto_compactions: 0 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-27T06:33:30.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.inplace_update_support: 0 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.bloom_locality: 0 2024-03-27T06:33:30.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.max_successive_merges: 0 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.paranoid_file_checks: 0 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.force_consistency_checks: 1 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.report_bg_io_stats: 0 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.ttl: 2592000 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.enable_blob_files: false 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.min_blob_size: 0 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.blob_file_size: 268435456 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-27T06:33:30.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.blob_file_starting_level: 0 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cf16d17e-90f5-4c67-b563-5cd003ed49e1 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711521210372813, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711521210373833, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101694, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99581, "index_size": 305, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14566, "raw_average_key_size": 50, "raw_value_size": 91709, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711521210, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cf16d17e-90f5-4c67-b563-5cd003ed49e1", "db_session_id": "XUGTGDT6OEXPUL4POZO6", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711521210373917, "job": 1, "event": "recovery_finished"} 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-27T06:33:30.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564bf6bc4000 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: DB pointer 0x564bf6bae000 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** DB Stats ** 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-03-27T06:33:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: L0 2/0 101.17 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: Sum 2/0 101.17 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-03-27T06:33:30.435 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative compaction: 0.00 GB write, 32.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval compaction: 0.00 GB write, 32.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache BinnedLRUCache@0x564bf6ad5090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: starting mon.a rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:33:30.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: mon.a@-1(???) e1 preinit fsid c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: mon.a@-1(???).mds e1 new map 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: mon.a@-1(???).mds e1 print_map 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout: e1 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout: legacy client fscid: -1 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout: No filesystems configured 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 bash[31889]: 06aedba2800884d1ea2407fd87a86eadf06a7409b67b48ac2dffa27527891498 2024-03-27T06:33:30.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 systemd[1]: Started Ceph mon.a for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:33:30.438 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-27T06:33:30.712 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-27T06:33:30.712 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: monmap epoch 1 2024-03-27T06:33:30.712 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: fsid c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:33:30.712 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: last_changed 2024-03-27T06:33:22.632787+0000 2024-03-27T06:33:30.712 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: created 2024-03-27T06:33:22.632787+0000 2024-03-27T06:33:30.712 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: min_mon_release 19 (squid) 2024-03-27T06:33:30.712 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: election_strategy: 1 2024-03-27T06:33:30.712 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-03-27T06:33:30.713 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: fsmap 2024-03-27T06:33:30.713 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: osdmap e1: 0 total, 0 up, 0 in 2024-03-27T06:33:30.713 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:30 smithi016 ceph-mon[31908]: mgrmap e1: no daemons active 2024-03-27T06:33:31.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.822+0000 7f5cf7bda700 1 Processor -- start 2024-03-27T06:33:31.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.822+0000 7f5cf7bda700 1 -- start start 2024-03-27T06:33:31.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.822+0000 7f5cf7bda700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf0108710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.822+0000 7f5cf7bda700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf0108c50 con 0x7f5cf01082f0 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.822+0000 7f5cf5976700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf0108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.823+0000 7f5cf5976700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf0108710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44472/0 (socket says 172.21.15.16:44472) 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.823+0000 7f5cf5976700 1 -- 172.21.15.16:0/942973615 learned_addr learned my addr 172.21.15.16:0/942973615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.823+0000 7f5cf5976700 1 -- 172.21.15.16:0/942973615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf0108dd0 con 0x7f5cf01082f0 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.823+0000 7f5cf5976700 1 --2- 172.21.15.16:0/942973615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf0108710 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098a0 tx=0x7f5cec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96864cca3c2037fb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.823+0000 7f5cf4974700 1 -- 172.21.15.16:0/942973615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec004160 con 0x7f5cf01082f0 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.823+0000 7f5cf4974700 1 -- 172.21.15.16:0/942973615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f5cec004300 con 0x7f5cf01082f0 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.823+0000 7f5cf4974700 1 -- 172.21.15.16:0/942973615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec011c00 con 0x7f5cf01082f0 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.824+0000 7f5cf7bda700 1 -- 172.21.15.16:0/942973615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 msgr2=0x7f5cf0108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.824+0000 7f5cf7bda700 1 --2- 172.21.15.16:0/942973615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf0108710 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098a0 tx=0x7f5cec009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.824+0000 7f5cf7bda700 1 -- 172.21.15.16:0/942973615 shutdown_connections 2024-03-27T06:33:31.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.824+0000 7f5cf7bda700 1 --2- 172.21.15.16:0/942973615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf0108710 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.824+0000 7f5cf7bda700 1 -- 172.21.15.16:0/942973615 >> 172.21.15.16:0/942973615 conn(0x7f5cf0103930 msgr2=0x7f5cf0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.824+0000 7f5cf7bda700 1 -- 172.21.15.16:0/942973615 shutdown_connections 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.824+0000 7f5cf7bda700 1 -- 172.21.15.16:0/942973615 wait complete. 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.824+0000 7f5cf7bda700 1 Processor -- start 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.824+0000 7f5cf7bda700 1 -- start start 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5cf7bda700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5cf7bda700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010c760 con 0x7f5cf01082f0 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5cf5976700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5cf5976700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44482/0 (socket says 172.21.15.16:44482) 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5cf5976700 1 -- 172.21.15.16:0/1920393627 learned_addr learned my addr 172.21.15.16:0/1920393627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5cf5976700 1 -- 172.21.15.16:0/1920393627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf019c7d0 con 0x7f5cf01082f0 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5cf5976700 1 --2- 172.21.15.16:0/1920393627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf019c290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5cec011bd0 tx=0x7f5cec011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5ce6ffd700 1 -- 172.21.15.16:0/1920393627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec003b20 con 0x7f5cf01082f0 2024-03-27T06:33:31.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5cf7bda700 1 -- 172.21.15.16:0/1920393627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf019ca60 con 0x7f5cf01082f0 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5ce6ffd700 1 -- 172.21.15.16:0/1920393627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f5cec003cc0 con 0x7f5cf01082f0 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5ce6ffd700 1 -- 172.21.15.16:0/1920393627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec024930 con 0x7f5cf01082f0 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.825+0000 7f5cf7bda700 1 -- 172.21.15.16:0/1920393627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf019f750 con 0x7f5cf01082f0 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.826+0000 7f5ce6ffd700 1 -- 172.21.15.16:0/1920393627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5cec022020 con 0x7f5cf01082f0 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.826+0000 7f5ce6ffd700 1 -- 172.21.15.16:0/1920393627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5cec0193c0 con 0x7f5cf01082f0 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.826+0000 7f5cf7bda700 1 -- 172.21.15.16:0/1920393627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf010c8e0 con 0x7f5cf01082f0 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.829+0000 7f5ce6ffd700 1 -- 172.21.15.16:0/1920393627 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5cec01aca0 con 0x7f5cf01082f0 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.899+0000 7f5cf7bda700 1 -- 172.21.15.16:0/1920393627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f5cf019f990 con 0x7f5cf01082f0 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.901+0000 7f5ce6ffd700 1 -- 172.21.15.16:0/1920393627 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f5cec02d3f0 con 0x7f5cf01082f0 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.902+0000 7f5cf7bda700 1 -- 172.21.15.16:0/1920393627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 msgr2=0x7f5cf019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.902+0000 7f5cf7bda700 1 --2- 172.21.15.16:0/1920393627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf019c290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5cec011bd0 tx=0x7f5cec011cb0 comp rx=0 tx=0).stop 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.902+0000 7f5cf7bda700 1 -- 172.21.15.16:0/1920393627 shutdown_connections 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.902+0000 7f5cf7bda700 1 --2- 172.21.15.16:0/1920393627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf01082f0 0x7f5cf019c290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:31.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.902+0000 7f5cf7bda700 1 -- 172.21.15.16:0/1920393627 >> 172.21.15.16:0/1920393627 conn(0x7f5cf0103930 msgr2=0x7f5cf01043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:31.502 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.902+0000 7f5cf7bda700 1 -- 172.21.15.16:0/1920393627 shutdown_connections 2024-03-27T06:33:31.502 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:30.902+0000 7f5cf7bda700 1 -- 172.21.15.16:0/1920393627 wait complete. 2024-03-27T06:33:31.502 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-27T06:33:31.524 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-27T06:33:31.524 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-03-27T06:33:31.524 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-27T06:33:31.525 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-27T06:33:31.762 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.x 2024-03-27T06:33:31.763 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.x.service: Unit ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.x.service not loaded. 2024-03-27T06:33:31.908 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c9095bc4-ec03-11ee-95d0-87774f69a715.target.wants/ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@.service. 2024-03-27T06:33:32.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:31 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1920393627' entity='client.admin' 2024-03-27T06:33:32.384 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-27T06:33:32.384 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-03-27T06:33:32.391 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-27T06:33:32.391 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-03-27T06:33:32.391 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-03-27T06:33:32.391 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-03-27T06:33:32.399 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-27T06:33:32.399 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-03-27T06:33:32.407 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-27T06:33:32.407 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-03-27T06:33:32.407 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-03-27T06:33:32.407 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-27T06:33:32.407 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-03-27T06:33:32.407 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-03-27T06:33:32.922 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:32 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:32.872+0000 7f54d44fb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T06:33:33.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:32 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2592836707' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-27T06:33:33.360 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:32 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:32.974+0000 7f54d44fb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T06:33:33.984 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:33.658+0000 7f54d44fb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T06:33:33.985 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:33.752+0000 7f54d44fb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T06:33:33.985 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:33.894+0000 7f54d44fb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T06:33:33.985 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:33.991+0000 7f54d44fb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T06:33:34.336 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:34.075+0000 7f54d44fb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T06:33:34.336 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:34.343+0000 7f54d44fb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T06:33:34.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:34.455+0000 7f54d44fb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T06:33:34.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:34.541+0000 7f54d44fb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T06:33:35.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:34.638+0000 7f54d44fb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T06:33:35.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:34.815+0000 7f54d44fb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T06:33:36.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:35 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:35.813+0000 7f54d44fb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T06:33:36.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:35 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:35.946+0000 7f54d44fb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T06:33:36.570 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:36 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:36.286+0000 7f54d44fb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T06:33:36.570 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:36 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:36.377+0000 7f54d44fb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T06:33:36.570 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:36 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:36.577+0000 7f54d44fb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:33:36.860 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:36 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:36.770+0000 7f54d44fb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T06:33:37.336 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:37 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:37.060+0000 7f54d44fb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:33:37.336 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:37 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:37.322+0000 7f54d44fb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T06:33:37.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:37 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:37.563+0000 7f54d44fb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T06:33:38.712 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:38 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:38.387+0000 7f54d44fb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T06:33:38.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-03-27T06:33:38.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-03-27T06:33:38.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:33:38.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-03-27T06:33:38.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-27T06:33:38.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-27T06:33:38.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-27T06:33:38.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-27T06:33:38.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-03-27T06:33:25.534616+0000", 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.847+0000 7f42e9a27700 1 Processor -- start 2024-03-27T06:33:38.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.847+0000 7f42e9a27700 1 -- start start 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.847+0000 7f42e9a27700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e41075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.847+0000 7f42e9a27700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4107bc0 con 0x7f42e41071d0 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.847+0000 7f42e2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e41075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.847+0000 7f42e2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e41075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44512/0 (socket says 172.21.15.16:44512) 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.847+0000 7f42e2ffd700 1 -- 172.21.15.16:0/3781546312 learned_addr learned my addr 172.21.15.16:0/3781546312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.848+0000 7f42e2ffd700 1 -- 172.21.15.16:0/3781546312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e41083d0 con 0x7f42e41071d0 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.848+0000 7f42e2ffd700 1 --2- 172.21.15.16:0/3781546312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e41075f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42d80098a0 tx=0x7f42d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c182a83dcbe2437 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.848+0000 7f42e1ffb700 1 -- 172.21.15.16:0/3781546312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d8004160 con 0x7f42e41071d0 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.848+0000 7f42e1ffb700 1 -- 172.21.15.16:0/3781546312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f42d8004300 con 0x7f42e41071d0 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.848+0000 7f42e1ffb700 1 -- 172.21.15.16:0/3781546312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d8011c00 con 0x7f42e41071d0 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.849+0000 7f42e9a27700 1 -- 172.21.15.16:0/3781546312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 msgr2=0x7f42e41075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.849+0000 7f42e9a27700 1 --2- 172.21.15.16:0/3781546312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e41075f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42d80098a0 tx=0x7f42d8009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.849+0000 7f42e9a27700 1 -- 172.21.15.16:0/3781546312 shutdown_connections 2024-03-27T06:33:38.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.849+0000 7f42e9a27700 1 --2- 172.21.15.16:0/3781546312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e41075f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.849+0000 7f42e9a27700 1 -- 172.21.15.16:0/3781546312 >> 172.21.15.16:0/3781546312 conn(0x7f42e41027b0 msgr2=0x7f42e4104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.849+0000 7f42e9a27700 1 -- 172.21.15.16:0/3781546312 shutdown_connections 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.849+0000 7f42e9a27700 1 -- 172.21.15.16:0/3781546312 wait complete. 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.849+0000 7f42e9a27700 1 Processor -- start 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.850+0000 7f42e9a27700 1 -- start start 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.850+0000 7f42e9a27700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.850+0000 7f42e9a27700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42e410b780 con 0x7f42e41071d0 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.850+0000 7f42e2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.850+0000 7f42e2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44526/0 (socket says 172.21.15.16:44526) 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.850+0000 7f42e2ffd700 1 -- 172.21.15.16:0/2592836707 learned_addr learned my addr 172.21.15.16:0/2592836707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.850+0000 7f42e2ffd700 1 -- 172.21.15.16:0/2592836707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e419b7f0 con 0x7f42e41071d0 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.850+0000 7f42e2ffd700 1 --2- 172.21.15.16:0/2592836707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e419b2b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009fb0 tx=0x7f42d8011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:38.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.851+0000 7f42e8a25700 1 -- 172.21.15.16:0/2592836707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d80038b0 con 0x7f42e41071d0 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.851+0000 7f42e8a25700 1 -- 172.21.15.16:0/2592836707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f42d801a430 con 0x7f42e41071d0 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.851+0000 7f42e9a27700 1 -- 172.21.15.16:0/2592836707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e419ba80 con 0x7f42e41071d0 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.851+0000 7f42e8a25700 1 -- 172.21.15.16:0/2592836707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d80195e0 con 0x7f42e41071d0 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.851+0000 7f42e9a27700 1 -- 172.21.15.16:0/2592836707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e419e770 con 0x7f42e41071d0 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.851+0000 7f42e8a25700 1 -- 172.21.15.16:0/2592836707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f42d8019c90 con 0x7f42e41071d0 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.851+0000 7f42e8a25700 1 -- 172.21.15.16:0/2592836707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f42d8024730 con 0x7f42e41071d0 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.852+0000 7f42e9a27700 1 -- 172.21.15.16:0/2592836707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d0005360 con 0x7f42e41071d0 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.854+0000 7f42e8a25700 1 -- 172.21.15.16:0/2592836707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f42d8043e50 con 0x7f42e41071d0 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.928+0000 7f42e9a27700 1 -- 172.21.15.16:0/2592836707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f42d0005920 con 0x7f42e41071d0 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.929+0000 7f42e8a25700 1 -- 172.21.15.16:0/2592836707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f42d8032020 con 0x7f42e41071d0 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.930+0000 7f42e9a27700 1 -- 172.21.15.16:0/2592836707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 msgr2=0x7f42e419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.930+0000 7f42e9a27700 1 --2- 172.21.15.16:0/2592836707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e419b2b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009fb0 tx=0x7f42d8011c60 comp rx=0 tx=0).stop 2024-03-27T06:33:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.930+0000 7f42e9a27700 1 -- 172.21.15.16:0/2592836707 shutdown_connections 2024-03-27T06:33:38.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.930+0000 7f42e9a27700 1 --2- 172.21.15.16:0/2592836707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42e41071d0 0x7f42e419b2b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:38.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.930+0000 7f42e9a27700 1 -- 172.21.15.16:0/2592836707 >> 172.21.15.16:0/2592836707 conn(0x7f42e41027b0 msgr2=0x7f42e4103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:38.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.930+0000 7f42e9a27700 1 -- 172.21.15.16:0/2592836707 shutdown_connections 2024-03-27T06:33:38.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:32.930+0000 7f42e9a27700 1 -- 172.21.15.16:0/2592836707 wait complete. 2024-03-27T06:33:38.733 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-03-27T06:33:39.284 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:38 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:38.970+0000 7f54d44fb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T06:33:39.284 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:39.070+0000 7f54d44fb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T06:33:39.284 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:39.291+0000 7f54d44fb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T06:33:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: Activating manager daemon x 2024-03-27T06:33:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: mgrmap e2: x(active, starting, since 0.00315914s) 2024-03-27T06:33:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: from='mgr.14100 172.21.15.16:0/3960903349' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-27T06:33:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: from='mgr.14100 172.21.15.16:0/3960903349' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-27T06:33:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: from='mgr.14100 172.21.15.16:0/3960903349' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-27T06:33:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: from='mgr.14100 172.21.15.16:0/3960903349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:33:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: from='mgr.14100 172.21.15.16:0/3960903349' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-27T06:33:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: Manager daemon x is now available 2024-03-27T06:33:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: from='mgr.14100 172.21.15.16:0/3960903349' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-27T06:33:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: from='mgr.14100 172.21.15.16:0/3960903349' entity='mgr.x' 2024-03-27T06:33:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: from='mgr.14100 172.21.15.16:0/3960903349' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-27T06:33:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: from='mgr.14100 172.21.15.16:0/3960903349' entity='mgr.x' 2024-03-27T06:33:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:39 smithi016 ceph-mon[31908]: from='mgr.14100 172.21.15.16:0/3960903349' entity='mgr.x' 2024-03-27T06:33:41.436 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:41 smithi016 ceph-mon[31908]: mgrmap e3: x(active, since 1.00729s) 2024-03-27T06:33:42.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-03-27T06:33:42.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-03-27T06:33:42.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:33:42.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-03-27T06:33:42.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-27T06:33:42.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-27T06:33:42.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-27T06:33:42.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:42.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T06:33:42.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-27T06:33:42.082 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-27T06:33:42.082 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-27T06:33:42.091 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-27T06:33:42.092 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-03-27T06:33:42.093 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-03-27T06:33:25.534616+0000", 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.140+0000 7f36db838700 1 Processor -- start 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.140+0000 7f36db838700 1 -- start start 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.140+0000 7f36db838700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d41075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.140+0000 7f36db838700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4107bc0 con 0x7f36d41071d0 2024-03-27T06:33:42.094 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.140+0000 7f36d95d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d41075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:42.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.140+0000 7f36d95d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d41075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54528/0 (socket says 172.21.15.16:54528) 2024-03-27T06:33:42.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.140+0000 7f36d95d4700 1 -- 172.21.15.16:0/914349020 learned_addr learned my addr 172.21.15.16:0/914349020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:42.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.141+0000 7f36d95d4700 1 -- 172.21.15.16:0/914349020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d41083d0 con 0x7f36d41071d0 2024-03-27T06:33:42.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.141+0000 7f36d95d4700 1 --2- 172.21.15.16:0/914349020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d41075f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f36d00098a0 tx=0x7f36d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef430d21f91df963 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:42.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.141+0000 7f36cbfff700 1 -- 172.21.15.16:0/914349020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36d0004160 con 0x7f36d41071d0 2024-03-27T06:33:42.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.141+0000 7f36cbfff700 1 -- 172.21.15.16:0/914349020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f36d0004300 con 0x7f36d41071d0 2024-03-27T06:33:42.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.141+0000 7f36cbfff700 1 -- 172.21.15.16:0/914349020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36d0011c00 con 0x7f36d41071d0 2024-03-27T06:33:42.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.142+0000 7f36db838700 1 -- 172.21.15.16:0/914349020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 msgr2=0x7f36d41075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:42.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.142+0000 7f36db838700 1 --2- 172.21.15.16:0/914349020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d41075f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f36d00098a0 tx=0x7f36d0009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.142+0000 7f36db838700 1 -- 172.21.15.16:0/914349020 shutdown_connections 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.142+0000 7f36db838700 1 --2- 172.21.15.16:0/914349020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d41075f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.142+0000 7f36db838700 1 -- 172.21.15.16:0/914349020 >> 172.21.15.16:0/914349020 conn(0x7f36d41027b0 msgr2=0x7f36d4104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.142+0000 7f36db838700 1 -- 172.21.15.16:0/914349020 shutdown_connections 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.142+0000 7f36db838700 1 -- 172.21.15.16:0/914349020 wait complete. 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.142+0000 7f36db838700 1 Processor -- start 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.142+0000 7f36db838700 1 -- start start 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.142+0000 7f36db838700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d407ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.142+0000 7f36db838700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36d410b780 con 0x7f36d41071d0 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.143+0000 7f36d95d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d407ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.143+0000 7f36d95d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d407ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54536/0 (socket says 172.21.15.16:54536) 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.143+0000 7f36d95d4700 1 -- 172.21.15.16:0/2406065360 learned_addr learned my addr 172.21.15.16:0/2406065360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.143+0000 7f36d95d4700 1 -- 172.21.15.16:0/2406065360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d419f100 con 0x7f36d41071d0 2024-03-27T06:33:42.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.143+0000 7f36d95d4700 1 --2- 172.21.15.16:0/2406065360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d407ba50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009fb0 tx=0x7f36d0011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.143+0000 7f36ca7fc700 1 -- 172.21.15.16:0/2406065360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36d00038b0 con 0x7f36d41071d0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.143+0000 7f36ca7fc700 1 -- 172.21.15.16:0/2406065360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f36d001a430 con 0x7f36d41071d0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.143+0000 7f36db838700 1 -- 172.21.15.16:0/2406065360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d419f2d0 con 0x7f36d41071d0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.143+0000 7f36ca7fc700 1 -- 172.21.15.16:0/2406065360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36d00195e0 con 0x7f36d41071d0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.143+0000 7f36db838700 1 -- 172.21.15.16:0/2406065360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d419f670 con 0x7f36d41071d0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.144+0000 7f36ca7fc700 1 -- 172.21.15.16:0/2406065360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 3) ==== 46349+0+0 (secure 0 0 0) 0x7f36d0019890 con 0x7f36d41071d0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.144+0000 7f36ca7fc700 1 --2- 172.21.15.16:0/2406065360 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f36c0039270 0x7f36c003b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.144+0000 7f36ca7fc700 1 -- 172.21.15.16:0/2406065360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f36d004d510 con 0x7f36d41071d0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.144+0000 7f36d8dd3700 1 --2- 172.21.15.16:0/2406065360 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f36c0039270 0x7f36c003b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.144+0000 7f36db838700 1 -- 172.21.15.16:0/2406065360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d410b900 con 0x7f36d41071d0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.145+0000 7f36d8dd3700 1 --2- 172.21.15.16:0/2406065360 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f36c0039270 0x7f36c003b730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f36c4006fd0 tx=0x7f36c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.150+0000 7f36ca7fc700 1 -- 172.21.15.16:0/2406065360 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f36d000bc40 con 0x7f36d41071d0 2024-03-27T06:33:42.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.443+0000 7f36db838700 1 -- 172.21.15.16:0/2406065360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f36d419f8f0 con 0x7f36d41071d0 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.443+0000 7f36ca7fc700 1 -- 172.21.15.16:0/2406065360 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f36d000fda0 con 0x7f36d41071d0 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.445+0000 7f36db838700 1 -- 172.21.15.16:0/2406065360 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f36c0039270 msgr2=0x7f36c003b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.445+0000 7f36db838700 1 --2- 172.21.15.16:0/2406065360 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f36c0039270 0x7f36c003b730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f36c4006fd0 tx=0x7f36c4006e40 comp rx=0 tx=0).stop 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.445+0000 7f36db838700 1 -- 172.21.15.16:0/2406065360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 msgr2=0x7f36d407ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.445+0000 7f36db838700 1 --2- 172.21.15.16:0/2406065360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d407ba50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009fb0 tx=0x7f36d0011c60 comp rx=0 tx=0).stop 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.445+0000 7f36db838700 1 -- 172.21.15.16:0/2406065360 shutdown_connections 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.445+0000 7f36db838700 1 --2- 172.21.15.16:0/2406065360 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f36c0039270 0x7f36c003b730 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.445+0000 7f36db838700 1 --2- 172.21.15.16:0/2406065360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d41071d0 0x7f36d407ba50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.445+0000 7f36db838700 1 -- 172.21.15.16:0/2406065360 >> 172.21.15.16:0/2406065360 conn(0x7f36d41027b0 msgr2=0x7f36d4103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.445+0000 7f36db838700 1 -- 172.21.15.16:0/2406065360 shutdown_connections 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:41.446+0000 7f36db838700 1 -- 172.21.15.16:0/2406065360 wait complete. 2024-03-27T06:33:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-03-27T06:33:42.325 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:42 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2406065360' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-27T06:33:43.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-03-27T06:33:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.547+0000 7fc3839a0700 1 Processor -- start 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.547+0000 7fc3839a0700 1 -- start start 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.547+0000 7fc3839a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c10b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.547+0000 7fc3839a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c07beb0 con 0x7fc37c108d70 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.548+0000 7fc38173c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c10b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.548+0000 7fc38173c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c10b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54546/0 (socket says 172.21.15.16:54546) 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.548+0000 7fc38173c700 1 -- 172.21.15.16:0/4151898643 learned_addr learned my addr 172.21.15.16:0/4151898643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.548+0000 7fc38173c700 1 -- 172.21.15.16:0/4151898643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c10b6a0 con 0x7fc37c108d70 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.548+0000 7fc38173c700 1 --2- 172.21.15.16:0/4151898643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c10b160 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3740098a0 tx=0x7fc374009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8e75bcdd713155 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.548+0000 7fc37bfff700 1 -- 172.21.15.16:0/4151898643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc374004160 con 0x7fc37c108d70 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.548+0000 7fc37bfff700 1 -- 172.21.15.16:0/4151898643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fc374004300 con 0x7fc37c108d70 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.548+0000 7fc37bfff700 1 -- 172.21.15.16:0/4151898643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc374011c00 con 0x7fc37c108d70 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.549+0000 7fc3839a0700 1 -- 172.21.15.16:0/4151898643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 msgr2=0x7fc37c10b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.549+0000 7fc3839a0700 1 --2- 172.21.15.16:0/4151898643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c10b160 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3740098a0 tx=0x7fc374009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.549+0000 7fc3839a0700 1 -- 172.21.15.16:0/4151898643 shutdown_connections 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.549+0000 7fc3839a0700 1 --2- 172.21.15.16:0/4151898643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c10b160 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.549+0000 7fc3839a0700 1 -- 172.21.15.16:0/4151898643 >> 172.21.15.16:0/4151898643 conn(0x7fc37c102780 msgr2=0x7fc37c104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.549+0000 7fc3839a0700 1 -- 172.21.15.16:0/4151898643 shutdown_connections 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.549+0000 7fc3839a0700 1 -- 172.21.15.16:0/4151898643 wait complete. 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.550+0000 7fc3839a0700 1 Processor -- start 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.550+0000 7fc3839a0700 1 -- start start 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.550+0000 7fc3839a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.550+0000 7fc3839a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c10c730 con 0x7fc37c108d70 2024-03-27T06:33:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.550+0000 7fc38173c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.550+0000 7fc38173c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c19e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54560/0 (socket says 172.21.15.16:54560) 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.550+0000 7fc38173c700 1 -- 172.21.15.16:0/3384217296 learned_addr learned my addr 172.21.15.16:0/3384217296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.550+0000 7fc38173c700 1 -- 172.21.15.16:0/3384217296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c19eaa0 con 0x7fc37c108d70 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.550+0000 7fc38173c700 1 --2- 172.21.15.16:0/3384217296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c19e560 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc374011bd0 tx=0x7fc374011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.550+0000 7fc37a7fc700 1 -- 172.21.15.16:0/3384217296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc374003970 con 0x7fc37c108d70 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.550+0000 7fc37a7fc700 1 -- 172.21.15.16:0/3384217296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fc374003b10 con 0x7fc37c108d70 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.551+0000 7fc37a7fc700 1 -- 172.21.15.16:0/3384217296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3740245a0 con 0x7fc37c108d70 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.551+0000 7fc3839a0700 1 -- 172.21.15.16:0/3384217296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c19ed30 con 0x7fc37c108d70 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.551+0000 7fc3839a0700 1 -- 172.21.15.16:0/3384217296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c1a1a20 con 0x7fc37c108d70 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.551+0000 7fc37a7fc700 1 -- 172.21.15.16:0/3384217296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 46455+0+0 (secure 0 0 0) 0x7fc374024850 con 0x7fc37c108d70 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.552+0000 7fc3839a0700 1 -- 172.21.15.16:0/3384217296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc37c063c70 con 0x7fc37c108d70 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.552+0000 7fc37a7fc700 1 --2- 172.21.15.16:0/3384217296 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fc3640393d0 0x7fc36403b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.552+0000 7fc37a7fc700 1 -- 172.21.15.16:0/3384217296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc37404d7b0 con 0x7fc37c108d70 2024-03-27T06:33:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.552+0000 7fc380f3b700 1 --2- 172.21.15.16:0/3384217296 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fc3640393d0 0x7fc36403b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.552+0000 7fc380f3b700 1 --2- 172.21.15.16:0/3384217296 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fc3640393d0 0x7fc36403b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc370006fd0 tx=0x7fc370006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.557+0000 7fc37a7fc700 1 -- 172.21.15.16:0/3384217296 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc374003cc0 con 0x7fc37c108d70 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.764+0000 7fc3839a0700 1 -- 172.21.15.16:0/3384217296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc37c1a1bf0 con 0x7fc37c108d70 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.765+0000 7fc37a7fc700 1 -- 172.21.15.16:0/3384217296 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fc37c1a1bf0 con 0x7fc37c108d70 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.767+0000 7fc3839a0700 1 -- 172.21.15.16:0/3384217296 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fc3640393d0 msgr2=0x7fc36403b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.767+0000 7fc3839a0700 1 --2- 172.21.15.16:0/3384217296 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fc3640393d0 0x7fc36403b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc370006fd0 tx=0x7fc370006e40 comp rx=0 tx=0).stop 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.767+0000 7fc3839a0700 1 -- 172.21.15.16:0/3384217296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 msgr2=0x7fc37c19e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.767+0000 7fc3839a0700 1 --2- 172.21.15.16:0/3384217296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c19e560 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc374011bd0 tx=0x7fc374011cb0 comp rx=0 tx=0).stop 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.768+0000 7fc3839a0700 1 -- 172.21.15.16:0/3384217296 shutdown_connections 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.768+0000 7fc3839a0700 1 --2- 172.21.15.16:0/3384217296 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fc3640393d0 0x7fc36403b890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.768+0000 7fc3839a0700 1 --2- 172.21.15.16:0/3384217296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc37c108d70 0x7fc37c19e560 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.768+0000 7fc3839a0700 1 -- 172.21.15.16:0/3384217296 >> 172.21.15.16:0/3384217296 conn(0x7fc37c102780 msgr2=0x7fc37c1032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.768+0000 7fc3839a0700 1 -- 172.21.15.16:0/3384217296 shutdown_connections 2024-03-27T06:33:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:42.768+0000 7fc3839a0700 1 -- 172.21.15.16:0/3384217296 wait complete. 2024-03-27T06:33:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-03-27T06:33:43.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:43 smithi016 ceph-mon[31908]: mgrmap e4: x(active, since 3s) 2024-03-27T06:33:43.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:43 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3384217296' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-27T06:33:44.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:44 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3868310683' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-27T06:33:44.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:44 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: ignoring --setuser ceph since I am not root 2024-03-27T06:33:44.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:44 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: ignoring --setgroup ceph since I am not root 2024-03-27T06:33:44.981 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:44 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:44.891+0000 7f048986d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T06:33:44.981 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:44 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:44.987+0000 7f048986d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T06:33:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:45 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3868310683' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-27T06:33:45.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:45 smithi016 ceph-mon[31908]: mgrmap e5: x(active, since 5s) 2024-03-27T06:33:45.882 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:45 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:45.649+0000 7f048986d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T06:33:45.882 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:45 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:45.745+0000 7f048986d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T06:33:45.882 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:45 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:45.888+0000 7f048986d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T06:33:46.335 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:45 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:45.987+0000 7f048986d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T06:33:46.335 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:46.072+0000 7f048986d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T06:33:46.335 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:46.342+0000 7f048986d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T06:33:46.511 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.716+0000 7fbffdbdf700 1 Processor -- start 2024-03-27T06:33:46.511 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.716+0000 7fbffdbdf700 1 -- start start 2024-03-27T06:33:46.511 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.716+0000 7fbffdbdf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:46.511 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.716+0000 7fbffdbdf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbff807c840 con 0x7fbff807dd90 2024-03-27T06:33:46.511 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.716+0000 7fbff77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:46.511 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.716+0000 7fbff77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54568/0 (socket says 172.21.15.16:54568) 2024-03-27T06:33:46.511 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.716+0000 7fbff77fe700 1 -- 172.21.15.16:0/1770791363 learned_addr learned my addr 172.21.15.16:0/1770791363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.717+0000 7fbff77fe700 1 -- 172.21.15.16:0/1770791363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff807c9c0 con 0x7fbff807dd90 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.717+0000 7fbff77fe700 1 --2- 172.21.15.16:0/1770791363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff807c300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80098a0 tx=0x7fbfe8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de478a76dc8e6d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.717+0000 7fbff67fc700 1 -- 172.21.15.16:0/1770791363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfe8004160 con 0x7fbff807dd90 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.717+0000 7fbff67fc700 1 -- 172.21.15.16:0/1770791363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fbfe8004300 con 0x7fbff807dd90 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.717+0000 7fbff67fc700 1 -- 172.21.15.16:0/1770791363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfe8011c00 con 0x7fbff807dd90 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.717+0000 7fbffdbdf700 1 -- 172.21.15.16:0/1770791363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 msgr2=0x7fbff807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.717+0000 7fbffdbdf700 1 --2- 172.21.15.16:0/1770791363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff807c300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80098a0 tx=0x7fbfe8009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.717+0000 7fbffdbdf700 1 -- 172.21.15.16:0/1770791363 shutdown_connections 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.717+0000 7fbffdbdf700 1 --2- 172.21.15.16:0/1770791363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff807c300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.717+0000 7fbffdbdf700 1 -- 172.21.15.16:0/1770791363 >> 172.21.15.16:0/1770791363 conn(0x7fbff8103990 msgr2=0x7fbff8105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.718+0000 7fbffdbdf700 1 -- 172.21.15.16:0/1770791363 shutdown_connections 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.718+0000 7fbffdbdf700 1 -- 172.21.15.16:0/1770791363 wait complete. 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.718+0000 7fbffdbdf700 1 Processor -- start 2024-03-27T06:33:46.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.718+0000 7fbffdbdf700 1 -- start start 2024-03-27T06:33:46.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.718+0000 7fbffdbdf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:46.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.718+0000 7fbffdbdf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbff810e840 con 0x7fbff807dd90 2024-03-27T06:33:46.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.719+0000 7fbff77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:46.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.719+0000 7fbff77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff819e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54570/0 (socket says 172.21.15.16:54570) 2024-03-27T06:33:46.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.719+0000 7fbff77fe700 1 -- 172.21.15.16:0/3868310683 learned_addr learned my addr 172.21.15.16:0/3868310683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:46.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.719+0000 7fbff77fe700 1 -- 172.21.15.16:0/3868310683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff819ea10 con 0x7fbff807dd90 2024-03-27T06:33:46.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.719+0000 7fbff77fe700 1 --2- 172.21.15.16:0/3868310683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff819e4d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8011bd0 tx=0x7fbfe8011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:46.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.719+0000 7fbff4ff9700 1 -- 172.21.15.16:0/3868310683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfe8003b20 con 0x7fbff807dd90 2024-03-27T06:33:46.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.719+0000 7fbff4ff9700 1 -- 172.21.15.16:0/3868310683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fbfe8003cc0 con 0x7fbff807dd90 2024-03-27T06:33:46.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.719+0000 7fbff4ff9700 1 -- 172.21.15.16:0/3868310683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfe8024930 con 0x7fbff807dd90 2024-03-27T06:33:46.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.719+0000 7fbffdbdf700 1 -- 172.21.15.16:0/3868310683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff819eca0 con 0x7fbff807dd90 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.719+0000 7fbffdbdf700 1 -- 172.21.15.16:0/3868310683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff81a1990 con 0x7fbff807dd90 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.720+0000 7fbff4ff9700 1 -- 172.21.15.16:0/3868310683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 46455+0+0 (secure 0 0 0) 0x7fbfe8022020 con 0x7fbff807dd90 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.720+0000 7fbffdbdf700 1 -- 172.21.15.16:0/3868310683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff8063c70 con 0x7fbff807dd90 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.720+0000 7fbff4ff9700 1 --2- 172.21.15.16:0/3868310683 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fbfe00393d0 0x7fbfe003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.720+0000 7fbff4ff9700 1 -- 172.21.15.16:0/3868310683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbfe804d1c0 con 0x7fbff807dd90 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.720+0000 7fbff6ffd700 1 --2- 172.21.15.16:0/3868310683 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fbfe00393d0 0x7fbfe003b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.721+0000 7fbff6ffd700 1 --2- 172.21.15.16:0/3868310683 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fbfe00393d0 0x7fbfe003b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbfec006fd0 tx=0x7fbfec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.725+0000 7fbff4ff9700 1 -- 172.21.15.16:0/3868310683 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fbfe804cf20 con 0x7fbff807dd90 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:43.994+0000 7fbffdbdf700 1 -- 172.21.15.16:0/3868310683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fbff81a1b60 con 0x7fbff807dd90 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.315+0000 7fbff4ff9700 1 -- 172.21.15.16:0/3868310683 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fbff81a1b60 con 0x7fbff807dd90 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.317+0000 7fbff4ff9700 1 -- 172.21.15.16:0/3868310683 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 5) ==== 46466+0+0 (secure 0 0 0) 0x7fbfe801a950 con 0x7fbff807dd90 2024-03-27T06:33:46.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.318+0000 7fbffdbdf700 1 -- 172.21.15.16:0/3868310683 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fbfe00393d0 msgr2=0x7fbfe003b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:46.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.318+0000 7fbffdbdf700 1 --2- 172.21.15.16:0/3868310683 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fbfe00393d0 0x7fbfe003b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbfec006fd0 tx=0x7fbfec006e40 comp rx=0 tx=0).stop 2024-03-27T06:33:46.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.318+0000 7fbffdbdf700 1 -- 172.21.15.16:0/3868310683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 msgr2=0x7fbff819e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:46.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.318+0000 7fbffdbdf700 1 --2- 172.21.15.16:0/3868310683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff819e4d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8011bd0 tx=0x7fbfe8011cb0 comp rx=0 tx=0).stop 2024-03-27T06:33:46.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.318+0000 7fbffdbdf700 1 -- 172.21.15.16:0/3868310683 shutdown_connections 2024-03-27T06:33:46.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.318+0000 7fbffdbdf700 1 --2- 172.21.15.16:0/3868310683 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fbfe00393d0 0x7fbfe003b890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:46.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.318+0000 7fbffdbdf700 1 --2- 172.21.15.16:0/3868310683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff807dd90 0x7fbff819e4d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:46.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.318+0000 7fbffdbdf700 1 -- 172.21.15.16:0/3868310683 >> 172.21.15.16:0/3868310683 conn(0x7fbff8103990 msgr2=0x7fbff8104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:46.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.319+0000 7fbffdbdf700 1 -- 172.21.15.16:0/3868310683 shutdown_connections 2024-03-27T06:33:46.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:44.319+0000 7fbffdbdf700 1 -- 172.21.15.16:0/3868310683 wait complete. 2024-03-27T06:33:46.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:46.455+0000 7f048986d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T06:33:46.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:46.546+0000 7f048986d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T06:33:46.888 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:46.646+0000 7f048986d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T06:33:46.888 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:46.827+0000 7f048986d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T06:33:48.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:47 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:47.834+0000 7f048986d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T06:33:48.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:47 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:47.969+0000 7f048986d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T06:33:48.598 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:48 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:48.309+0000 7f048986d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T06:33:48.599 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:48 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:48.403+0000 7f048986d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T06:33:48.599 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:48 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:48.605+0000 7f048986d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:33:48.860 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:48 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:48.797+0000 7f048986d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T06:33:49.360 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:49 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:49.091+0000 7f048986d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:33:49.616 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:49 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:49.371+0000 7f048986d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T06:33:49.616 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:49 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:49.617+0000 7f048986d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T06:33:49.973 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:49 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/4075066375' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-27T06:33:50.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-03-27T06:33:50.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-27T06:33:50.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-27T06:33:50.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-27T06:33:50.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-27T06:33:50.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-03-27T06:33:50.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.326+0000 7fd3d9ca2700 1 Processor -- start 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.326+0000 7fd3d9ca2700 1 -- start start 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.326+0000 7fd3d9ca2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d410b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.326+0000 7fd3d9ca2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d407beb0 con 0x7fd3d4108d40 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.326+0000 7fd3d37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d410b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.326+0000 7fd3d37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d410b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57678/0 (socket says 172.21.15.16:57678) 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.326+0000 7fd3d37fe700 1 -- 172.21.15.16:0/2911814232 learned_addr learned my addr 172.21.15.16:0/2911814232 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.327+0000 7fd3d37fe700 1 -- 172.21.15.16:0/2911814232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d410b670 con 0x7fd3d4108d40 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.327+0000 7fd3d37fe700 1 --2- 172.21.15.16:0/2911814232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d410b130 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40098a0 tx=0x7fd3c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215cf89cb8189842 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.327+0000 7fd3d27fc700 1 -- 172.21.15.16:0/2911814232 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3c4004160 con 0x7fd3d4108d40 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.327+0000 7fd3d27fc700 1 -- 172.21.15.16:0/2911814232 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fd3c4004300 con 0x7fd3d4108d40 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.327+0000 7fd3d27fc700 1 -- 172.21.15.16:0/2911814232 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3c4011c00 con 0x7fd3d4108d40 2024-03-27T06:33:50.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.328+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/2911814232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 msgr2=0x7fd3d410b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.328+0000 7fd3d9ca2700 1 --2- 172.21.15.16:0/2911814232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d410b130 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40098a0 tx=0x7fd3c4009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.328+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/2911814232 shutdown_connections 2024-03-27T06:33:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.328+0000 7fd3d9ca2700 1 --2- 172.21.15.16:0/2911814232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d410b130 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.328+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/2911814232 >> 172.21.15.16:0/2911814232 conn(0x7fd3d41027b0 msgr2=0x7fd3d4104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.328+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/2911814232 shutdown_connections 2024-03-27T06:33:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.328+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/2911814232 wait complete. 2024-03-27T06:33:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.329+0000 7fd3d9ca2700 1 Processor -- start 2024-03-27T06:33:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.329+0000 7fd3d9ca2700 1 -- start start 2024-03-27T06:33:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.329+0000 7fd3d9ca2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.329+0000 7fd3d9ca2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d410c700 con 0x7fd3d4108d40 2024-03-27T06:33:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.329+0000 7fd3d37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.329+0000 7fd3d37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d419c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57694/0 (socket says 172.21.15.16:57694) 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.329+0000 7fd3d37fe700 1 -- 172.21.15.16:0/4075066375 learned_addr learned my addr 172.21.15.16:0/4075066375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.329+0000 7fd3d37fe700 1 -- 172.21.15.16:0/4075066375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d419c8d0 con 0x7fd3d4108d40 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.329+0000 7fd3d37fe700 1 --2- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d419c390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4011bd0 tx=0x7fd3c4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.330+0000 7fd3d0ff9700 1 -- 172.21.15.16:0/4075066375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3c4003970 con 0x7fd3d4108d40 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.330+0000 7fd3d0ff9700 1 -- 172.21.15.16:0/4075066375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fd3c4003b10 con 0x7fd3d4108d40 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.330+0000 7fd3d0ff9700 1 -- 172.21.15.16:0/4075066375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3c40245a0 con 0x7fd3d4108d40 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.330+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/4075066375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d419cb60 con 0x7fd3d4108d40 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.330+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/4075066375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d419f850 con 0x7fd3d4108d40 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.331+0000 7fd3d0ff9700 1 -- 172.21.15.16:0/4075066375 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 46466+0+0 (secure 0 0 0) 0x7fd3c4024850 con 0x7fd3d4108d40 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.331+0000 7fd3d0ff9700 1 --2- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fd3bc039420 0x7fd3bc03b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:50.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.331+0000 7fd3d0ff9700 1 -- 172.21.15.16:0/4075066375 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd3c404d660 con 0x7fd3d4108d40 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.331+0000 7fd3d2ffd700 1 -- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fd3bc039420 msgr2=0x7fd3bc03b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1221908125 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.331+0000 7fd3d2ffd700 1 --2- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fd3bc039420 0x7fd3bc03b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.331+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/4075066375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d4063c70 con 0x7fd3d4108d40 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.336+0000 7fd3d0ff9700 1 -- 172.21.15.16:0/4075066375 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd3c401a430 con 0x7fd3d4108d40 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.532+0000 7fd3d2ffd700 1 -- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fd3bc039420 msgr2=0x7fd3bc03b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1221908125 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.532+0000 7fd3d2ffd700 1 --2- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fd3bc039420 0x7fd3bc03b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.620+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/4075066375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd3d419fe20 con 0x7fd3d4108d40 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.620+0000 7fd3d0ff9700 1 -- 172.21.15.16:0/4075066375 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fd3c4019c70 con 0x7fd3d4108d40 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.622+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fd3bc039420 msgr2=0x7fd3bc03b8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.622+0000 7fd3d9ca2700 1 --2- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fd3bc039420 0x7fd3bc03b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.622+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 msgr2=0x7fd3d419c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:50.257 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.622+0000 7fd3d9ca2700 1 --2- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d419c390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4011bd0 tx=0x7fd3c4011cb0 comp rx=0 tx=0).stop 2024-03-27T06:33:50.258 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.622+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/4075066375 shutdown_connections 2024-03-27T06:33:50.258 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.622+0000 7fd3d9ca2700 1 --2- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7fd3bc039420 0x7fd3bc03b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:50.258 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.622+0000 7fd3d9ca2700 1 --2- 172.21.15.16:0/4075066375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d4108d40 0x7fd3d419c390 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:50.258 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.622+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/4075066375 >> 172.21.15.16:0/4075066375 conn(0x7fd3d41027b0 msgr2=0x7fd3d4103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:50.258 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.623+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/4075066375 shutdown_connections 2024-03-27T06:33:50.258 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:49.623+0000 7fd3d9ca2700 1 -- 172.21.15.16:0/4075066375 wait complete. 2024-03-27T06:33:50.258 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-03-27T06:33:50.258 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-03-27T06:33:50.492 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:50 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:50.472+0000 7f048986d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T06:33:51.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:51.059+0000 7f048986d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T06:33:51.439 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:51.159+0000 7f048986d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T06:33:51.440 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:33:51.381+0000 7f048986d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T06:33:51.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: Active manager daemon x restarted 2024-03-27T06:33:51.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: Activating manager daemon x 2024-03-27T06:33:51.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: osdmap e2: 0 total, 0 up, 0 in 2024-03-27T06:33:51.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: mgrmap e6: x(active, starting, since 0.00347152s) 2024-03-27T06:33:51.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:33:51.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-27T06:33:51.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-27T06:33:51.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-27T06:33:51.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-27T06:33:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: Manager daemon x is now available 2024-03-27T06:33:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:33:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:33:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:33:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:51 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:33:52.704 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:52 smithi016 ceph-mon[31908]: Found migration_current of "None". Setting to last migration. 2024-03-27T06:33:52.704 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:52 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-27T06:33:52.704 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:52 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-27T06:33:52.704 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:52 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:33:52.704 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:52 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:33:52.704 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:52 smithi016 ceph-mon[31908]: [27/Mar/2024:06:33:52] ENGINE Bus STARTING 2024-03-27T06:33:52.704 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:52 smithi016 ceph-mon[31908]: [27/Mar/2024:06:33:52] ENGINE Serving on https://172.21.15.16:7150 2024-03-27T06:33:52.705 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:52 smithi016 ceph-mon[31908]: mgrmap e7: x(active, since 1.00565s) 2024-03-27T06:33:53.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-03-27T06:33:53.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-27T06:33:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-27T06:33:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-03-27T06:33:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.606+0000 7f57bdbff700 1 Processor -- start 2024-03-27T06:33:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.606+0000 7f57bdbff700 1 -- start start 2024-03-27T06:33:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.606+0000 7f57bdbff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b81075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.606+0000 7f57bdbff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8107ba0 con 0x7f57b81071b0 2024-03-27T06:33:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.606+0000 7f57b77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b81075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.606+0000 7f57b77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b81075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57704/0 (socket says 172.21.15.16:57704) 2024-03-27T06:33:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.606+0000 7f57b77fe700 1 -- 172.21.15.16:0/1040380582 learned_addr learned my addr 172.21.15.16:0/1040380582 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.607+0000 7f57b77fe700 1 -- 172.21.15.16:0/1040380582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b81083b0 con 0x7f57b81071b0 2024-03-27T06:33:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.607+0000 7f57b77fe700 1 --2- 172.21.15.16:0/1040380582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b81075d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f57a80098a0 tx=0x7f57a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d332be88042e1470 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.607+0000 7f57b67fc700 1 -- 172.21.15.16:0/1040380582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57a8004160 con 0x7f57b81071b0 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.607+0000 7f57b67fc700 1 -- 172.21.15.16:0/1040380582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f57a8004300 con 0x7f57b81071b0 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.607+0000 7f57b67fc700 1 -- 172.21.15.16:0/1040380582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57a8011c00 con 0x7f57b81071b0 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.608+0000 7f57bdbff700 1 -- 172.21.15.16:0/1040380582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 msgr2=0x7f57b81075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.608+0000 7f57bdbff700 1 --2- 172.21.15.16:0/1040380582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b81075d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f57a80098a0 tx=0x7f57a8009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.608+0000 7f57bdbff700 1 -- 172.21.15.16:0/1040380582 shutdown_connections 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.608+0000 7f57bdbff700 1 --2- 172.21.15.16:0/1040380582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b81075d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.608+0000 7f57bdbff700 1 -- 172.21.15.16:0/1040380582 >> 172.21.15.16:0/1040380582 conn(0x7f57b8102790 msgr2=0x7f57b8104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.608+0000 7f57bdbff700 1 -- 172.21.15.16:0/1040380582 shutdown_connections 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.608+0000 7f57bdbff700 1 -- 172.21.15.16:0/1040380582 wait complete. 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.609+0000 7f57bdbff700 1 Processor -- start 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.609+0000 7f57bdbff700 1 -- start start 2024-03-27T06:33:53.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.609+0000 7f57bdbff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b807ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.609+0000 7f57bdbff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57b810b760 con 0x7f57b81071b0 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.609+0000 7f57b77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b807ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.609+0000 7f57b77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b807ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57718/0 (socket says 172.21.15.16:57718) 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.609+0000 7f57b77fe700 1 -- 172.21.15.16:0/2387917778 learned_addr learned my addr 172.21.15.16:0/2387917778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.609+0000 7f57b77fe700 1 -- 172.21.15.16:0/2387917778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b819efc0 con 0x7f57b81071b0 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.610+0000 7f57b77fe700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b807ba70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f57a8000c00 tx=0x7f57a8011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.610+0000 7f57b4ff9700 1 -- 172.21.15.16:0/2387917778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57a8003b20 con 0x7f57b81071b0 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.610+0000 7f57b4ff9700 1 -- 172.21.15.16:0/2387917778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f57a8003cc0 con 0x7f57b81071b0 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.610+0000 7f57bdbff700 1 -- 172.21.15.16:0/2387917778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b819f190 con 0x7f57b81071b0 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.610+0000 7f57b4ff9700 1 -- 172.21.15.16:0/2387917778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57a80249a0 con 0x7f57b81071b0 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.610+0000 7f57bdbff700 1 -- 172.21.15.16:0/2387917778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b819f5b0 con 0x7f57b81071b0 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.611+0000 7f57b4ff9700 1 -- 172.21.15.16:0/2387917778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 46466+0+0 (secure 0 0 0) 0x7f57a8022020 con 0x7f57b81071b0 2024-03-27T06:33:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.611+0000 7f57b4ff9700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f57a00393d0 0x7f57a003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.611+0000 7f57bdbff700 1 -- 172.21.15.16:0/2387917778 --> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f57b8063c70 con 0x7f57a00393d0 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.611+0000 7f57b4ff9700 1 -- 172.21.15.16:0/2387917778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f57a804e2c0 con 0x7f57b81071b0 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.611+0000 7f57b6ffd700 1 -- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f57a00393d0 msgr2=0x7f57a003b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1221908125 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.611+0000 7f57b6ffd700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f57a00393d0 0x7f57a003b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.812+0000 7f57b6ffd700 1 -- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f57a00393d0 msgr2=0x7f57a003b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1221908125 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:50.812+0000 7f57b6ffd700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f57a00393d0 0x7f57a003b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:51.213+0000 7f57b6ffd700 1 -- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f57a00393d0 msgr2=0x7f57a003b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1221908125 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:51.213+0000 7f57b6ffd700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f57a00393d0 0x7f57a003b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:51.397+0000 7f57b4ff9700 1 -- 172.21.15.16:0/2387917778 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 6) ==== 46233+0+0 (secure 0 0 0) 0x7f57a804d0c0 con 0x7f57b81071b0 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:51.397+0000 7f57b4ff9700 1 -- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f57a00393d0 msgr2=0x7f57a003b890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:51.397+0000 7f57b4ff9700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f57a00393d0 0x7f57a003b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.398+0000 7f57b4ff9700 1 -- 172.21.15.16:0/2387917778 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 7) ==== 46360+0+0 (secure 0 0 0) 0x7f57a802dc90 con 0x7f57b81071b0 2024-03-27T06:33:53.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.399+0000 7f57b4ff9700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f57a003cad0 0x7f57a003eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.399+0000 7f57b4ff9700 1 -- 172.21.15.16:0/2387917778 --> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f57a80068a0 con 0x7f57a003cad0 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.399+0000 7f57b6ffd700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f57a003cad0 0x7f57a003eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.400+0000 7f57b6ffd700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f57a003cad0 0x7f57a003eec0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f57ac003a10 tx=0x7f57ac009290 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.400+0000 7f57b4ff9700 1 -- 172.21.15.16:0/2387917778 <== mgr.14116 v2:172.21.15.16:6800/1685933023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f57a80068a0 con 0x7f57a003cad0 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.407+0000 7f57bdbff700 1 -- 172.21.15.16:0/2387917778 --> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f57b810b8e0 con 0x7f57a003cad0 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.407+0000 7f57b4ff9700 1 -- 172.21.15.16:0/2387917778 <== mgr.14116 v2:172.21.15.16:6800/1685933023 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f57b810b8e0 con 0x7f57a003cad0 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.407+0000 7f57bdbff700 1 -- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f57a003cad0 msgr2=0x7f57a003eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.407+0000 7f57bdbff700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f57a003cad0 0x7f57a003eec0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f57ac003a10 tx=0x7f57ac009290 comp rx=0 tx=0).stop 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.407+0000 7f57bdbff700 1 -- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 msgr2=0x7f57b807ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.407+0000 7f57bdbff700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b807ba70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f57a8000c00 tx=0x7f57a8011c80 comp rx=0 tx=0).stop 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.407+0000 7f57bdbff700 1 -- 172.21.15.16:0/2387917778 shutdown_connections 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.407+0000 7f57bdbff700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f57a003cad0 0x7f57a003eec0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.407+0000 7f57bdbff700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b81071b0 0x7f57b807ba70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.407+0000 7f57bdbff700 1 --2- 172.21.15.16:0/2387917778 >> [v2:172.21.15.16:6800/1221908125,v1:172.21.15.16:6801/1221908125] conn(0x7f57a00393d0 0x7f57a003b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:53.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.407+0000 7f57bdbff700 1 -- 172.21.15.16:0/2387917778 >> 172.21.15.16:0/2387917778 conn(0x7f57b8102790 msgr2=0x7f57b8103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:53.038 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.408+0000 7f57bdbff700 1 -- 172.21.15.16:0/2387917778 shutdown_connections 2024-03-27T06:33:53.038 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:52.408+0000 7f57bdbff700 1 -- 172.21.15.16:0/2387917778 wait complete. 2024-03-27T06:33:53.038 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-03-27T06:33:53.038 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-03-27T06:33:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:53 smithi016 ceph-mon[31908]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-27T06:33:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:53 smithi016 ceph-mon[31908]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-27T06:33:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:53 smithi016 ceph-mon[31908]: [27/Mar/2024:06:33:52] ENGINE Serving on http://172.21.15.16:8765 2024-03-27T06:33:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:53 smithi016 ceph-mon[31908]: [27/Mar/2024:06:33:52] ENGINE Bus STARTED 2024-03-27T06:33:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:53 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:33:54.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.446+0000 7ff7bfc13700 1 Processor -- start 2024-03-27T06:33:54.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.446+0000 7ff7bfc13700 1 -- start start 2024-03-27T06:33:54.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.446+0000 7ff7bfc13700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b810b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:54.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.446+0000 7ff7bfc13700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b807be50 con 0x7ff7b8108d80 2024-03-27T06:33:54.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.447+0000 7ff7bd9af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b810b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:54.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.447+0000 7ff7bd9af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b810b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57788/0 (socket says 172.21.15.16:57788) 2024-03-27T06:33:54.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.447+0000 7ff7bd9af700 1 -- 172.21.15.16:0/3765620770 learned_addr learned my addr 172.21.15.16:0/3765620770 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:54.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.447+0000 7ff7bd9af700 1 -- 172.21.15.16:0/3765620770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b810b6b0 con 0x7ff7b8108d80 2024-03-27T06:33:54.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.447+0000 7ff7bd9af700 1 --2- 172.21.15.16:0/3765620770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b810b170 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009e80 tx=0x7ff7a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efd093490dc40c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:54.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.447+0000 7ff7bc9ad700 1 -- 172.21.15.16:0/3765620770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a8017070 con 0x7ff7b8108d80 2024-03-27T06:33:54.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.447+0000 7ff7bc9ad700 1 -- 172.21.15.16:0/3765620770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7ff7a800b8c0 con 0x7ff7b8108d80 2024-03-27T06:33:54.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.447+0000 7ff7bc9ad700 1 -- 172.21.15.16:0/3765620770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a8004620 con 0x7ff7b8108d80 2024-03-27T06:33:54.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.448+0000 7ff7bfc13700 1 -- 172.21.15.16:0/3765620770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 msgr2=0x7ff7b810b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:54.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.448+0000 7ff7bfc13700 1 --2- 172.21.15.16:0/3765620770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b810b170 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009e80 tx=0x7ff7a800b330 comp rx=0 tx=0).stop 2024-03-27T06:33:54.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.448+0000 7ff7bfc13700 1 -- 172.21.15.16:0/3765620770 shutdown_connections 2024-03-27T06:33:54.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.448+0000 7ff7bfc13700 1 --2- 172.21.15.16:0/3765620770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b810b170 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:54.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.448+0000 7ff7bfc13700 1 -- 172.21.15.16:0/3765620770 >> 172.21.15.16:0/3765620770 conn(0x7ff7b8102790 msgr2=0x7ff7b8104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:54.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.448+0000 7ff7bfc13700 1 -- 172.21.15.16:0/3765620770 shutdown_connections 2024-03-27T06:33:54.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.448+0000 7ff7bfc13700 1 -- 172.21.15.16:0/3765620770 wait complete. 2024-03-27T06:33:54.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.449+0000 7ff7bfc13700 1 Processor -- start 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.449+0000 7ff7bfc13700 1 -- start start 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.449+0000 7ff7bfc13700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.449+0000 7ff7bfc13700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b810c6e0 con 0x7ff7b8108d80 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.449+0000 7ff7bd9af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.449+0000 7ff7bd9af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b819c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57800/0 (socket says 172.21.15.16:57800) 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.449+0000 7ff7bd9af700 1 -- 172.21.15.16:0/338739399 learned_addr learned my addr 172.21.15.16:0/338739399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.449+0000 7ff7bd9af700 1 -- 172.21.15.16:0/338739399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b819c790 con 0x7ff7b8108d80 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.450+0000 7ff7bd9af700 1 --2- 172.21.15.16:0/338739399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b819c250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80043d0 tx=0x7ff7a8003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.451+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/338739399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a8021020 con 0x7ff7b8108d80 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.451+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/338739399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7ff7a80094d0 con 0x7ff7b8108d80 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.451+0000 7ff7bfc13700 1 -- 172.21.15.16:0/338739399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b819ca20 con 0x7ff7b8108d80 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.451+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/338739399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7a8017040 con 0x7ff7b8108d80 2024-03-27T06:33:54.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.451+0000 7ff7bfc13700 1 -- 172.21.15.16:0/338739399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b819f710 con 0x7ff7b8108d80 2024-03-27T06:33:54.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.452+0000 7ff7bfc13700 1 -- 172.21.15.16:0/338739399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b8063c70 con 0x7ff7b8108d80 2024-03-27T06:33:54.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.453+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/338739399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7ff7a8024020 con 0x7ff7b8108d80 2024-03-27T06:33:54.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.453+0000 7ff7b6ffd700 1 --2- 172.21.15.16:0/338739399 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7ff7a00393d0 0x7ff7a003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:54.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.453+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/338739399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff7a800f050 con 0x7ff7b8108d80 2024-03-27T06:33:54.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.454+0000 7ff7bd1ae700 1 --2- 172.21.15.16:0/338739399 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7ff7a00393d0 0x7ff7a003b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:54.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.454+0000 7ff7bd1ae700 1 --2- 172.21.15.16:0/338739399 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7ff7a00393d0 0x7ff7a003b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac006fd0 tx=0x7ff7ac006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:54.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.458+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/338739399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff7a802b020 con 0x7ff7b8108d80 2024-03-27T06:33:54.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.683+0000 7ff7bfc13700 1 -- 172.21.15.16:0/338739399 --> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7ff7b8103940 con 0x7ff7a00393d0 2024-03-27T06:33:54.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.688+0000 7ff7b6ffd700 1 -- 172.21.15.16:0/338739399 <== mgr.14116 v2:172.21.15.16:6800/1685933023 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff7b8103940 con 0x7ff7a00393d0 2024-03-27T06:33:54.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.691+0000 7ff7bfc13700 1 -- 172.21.15.16:0/338739399 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7ff7a00393d0 msgr2=0x7ff7a003b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:54.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.691+0000 7ff7bfc13700 1 --2- 172.21.15.16:0/338739399 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7ff7a00393d0 0x7ff7a003b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac006fd0 tx=0x7ff7ac006e40 comp rx=0 tx=0).stop 2024-03-27T06:33:54.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.691+0000 7ff7bfc13700 1 -- 172.21.15.16:0/338739399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 msgr2=0x7ff7b819c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:54.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.691+0000 7ff7bfc13700 1 --2- 172.21.15.16:0/338739399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b819c250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80043d0 tx=0x7ff7a8003d00 comp rx=0 tx=0).stop 2024-03-27T06:33:54.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.691+0000 7ff7bfc13700 1 -- 172.21.15.16:0/338739399 shutdown_connections 2024-03-27T06:33:54.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.691+0000 7ff7bfc13700 1 --2- 172.21.15.16:0/338739399 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7ff7a00393d0 0x7ff7a003b890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:54.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.691+0000 7ff7bfc13700 1 --2- 172.21.15.16:0/338739399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7b8108d80 0x7ff7b819c250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:54.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.691+0000 7ff7bfc13700 1 -- 172.21.15.16:0/338739399 >> 172.21.15.16:0/338739399 conn(0x7ff7b8102790 msgr2=0x7ff7b8103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:54.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.691+0000 7ff7bfc13700 1 -- 172.21.15.16:0/338739399 shutdown_connections 2024-03-27T06:33:54.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:53.691+0000 7ff7bfc13700 1 -- 172.21.15.16:0/338739399 wait complete. 2024-03-27T06:33:54.772 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:54 smithi016 ceph-mon[31908]: mgrmap e8: x(active, since 2s) 2024-03-27T06:33:54.772 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:54 smithi016 ceph-mon[31908]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:33:54.772 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:54 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:33:54.773 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:54 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:33:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:55 smithi016 ceph-mon[31908]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:33:55.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-27T06:33:55.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.799+0000 7fe526a3c700 1 Processor -- start 2024-03-27T06:33:55.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.799+0000 7fe526a3c700 1 -- start start 2024-03-27T06:33:55.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.800+0000 7fe526a3c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52010b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:55.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.800+0000 7fe526a3c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe52007be70 con 0x7fe520108d60 2024-03-27T06:33:55.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.800+0000 7fe51ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52010b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:55.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.800+0000 7fe51ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52010b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57816/0 (socket says 172.21.15.16:57816) 2024-03-27T06:33:55.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.800+0000 7fe51ffff700 1 -- 172.21.15.16:0/3112580125 learned_addr learned my addr 172.21.15.16:0/3112580125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:55.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.800+0000 7fe51ffff700 1 -- 172.21.15.16:0/3112580125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52010b690 con 0x7fe520108d60 2024-03-27T06:33:55.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.800+0000 7fe51ffff700 1 --2- 172.21.15.16:0/3112580125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52010b150 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe5140098a0 tx=0x7fe514009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d927a0e80e4281 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.800+0000 7fe51effd700 1 -- 172.21.15.16:0/3112580125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe514004160 con 0x7fe520108d60 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.801+0000 7fe51effd700 1 -- 172.21.15.16:0/3112580125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fe514004300 con 0x7fe520108d60 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.801+0000 7fe51effd700 1 -- 172.21.15.16:0/3112580125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe514011c00 con 0x7fe520108d60 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.801+0000 7fe526a3c700 1 -- 172.21.15.16:0/3112580125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 msgr2=0x7fe52010b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.801+0000 7fe526a3c700 1 --2- 172.21.15.16:0/3112580125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52010b150 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe5140098a0 tx=0x7fe514009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.801+0000 7fe526a3c700 1 -- 172.21.15.16:0/3112580125 shutdown_connections 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.801+0000 7fe526a3c700 1 --2- 172.21.15.16:0/3112580125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52010b150 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.801+0000 7fe526a3c700 1 -- 172.21.15.16:0/3112580125 >> 172.21.15.16:0/3112580125 conn(0x7fe520102790 msgr2=0x7fe520104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.801+0000 7fe526a3c700 1 -- 172.21.15.16:0/3112580125 shutdown_connections 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.801+0000 7fe526a3c700 1 -- 172.21.15.16:0/3112580125 wait complete. 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.802+0000 7fe526a3c700 1 Processor -- start 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.802+0000 7fe526a3c700 1 -- start start 2024-03-27T06:33:55.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.802+0000 7fe526a3c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.802+0000 7fe526a3c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe52010c710 con 0x7fe520108d60 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.802+0000 7fe51ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.802+0000 7fe51ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52019c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57820/0 (socket says 172.21.15.16:57820) 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.802+0000 7fe51ffff700 1 -- 172.21.15.16:0/4121546278 learned_addr learned my addr 172.21.15.16:0/4121546278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.803+0000 7fe51ffff700 1 -- 172.21.15.16:0/4121546278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52019c800 con 0x7fe520108d60 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.803+0000 7fe51ffff700 1 --2- 172.21.15.16:0/4121546278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52019c2c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe514003fb0 tx=0x7fe514011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.803+0000 7fe51d7fa700 1 -- 172.21.15.16:0/4121546278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5140039c0 con 0x7fe520108d60 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.803+0000 7fe526a3c700 1 -- 172.21.15.16:0/4121546278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52019ca90 con 0x7fe520108d60 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.803+0000 7fe51d7fa700 1 -- 172.21.15.16:0/4121546278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fe514003b60 con 0x7fe520108d60 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.803+0000 7fe51d7fa700 1 -- 172.21.15.16:0/4121546278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe51401a6b0 con 0x7fe520108d60 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.803+0000 7fe526a3c700 1 -- 172.21.15.16:0/4121546278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52019f780 con 0x7fe520108d60 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.804+0000 7fe51d7fa700 1 -- 172.21.15.16:0/4121546278 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7fe51401a9d0 con 0x7fe520108d60 2024-03-27T06:33:55.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.804+0000 7fe526a3c700 1 -- 172.21.15.16:0/4121546278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe520063c70 con 0x7fe520108d60 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.804+0000 7fe51d7fa700 1 --2- 172.21.15.16:0/4121546278 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fe508039380 0x7fe50803b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.804+0000 7fe51d7fa700 1 -- 172.21.15.16:0/4121546278 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe51404da00 con 0x7fe520108d60 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.804+0000 7fe51f7fe700 1 --2- 172.21.15.16:0/4121546278 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fe508039380 0x7fe50803b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.805+0000 7fe51f7fe700 1 --2- 172.21.15.16:0/4121546278 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fe508039380 0x7fe50803b840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe510006fd0 tx=0x7fe510006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:54.810+0000 7fe51d7fa700 1 -- 172.21.15.16:0/4121546278 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fe5140249a0 con 0x7fe520108d60 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.016+0000 7fe526a3c700 1 -- 172.21.15.16:0/4121546278 --> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fe5201039b0 con 0x7fe508039380 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.017+0000 7fe51d7fa700 1 -- 172.21.15.16:0/4121546278 <== mgr.14116 v2:172.21.15.16:6800/1685933023 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fe5201039b0 con 0x7fe508039380 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.018+0000 7fe526a3c700 1 -- 172.21.15.16:0/4121546278 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fe508039380 msgr2=0x7fe50803b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.018+0000 7fe526a3c700 1 --2- 172.21.15.16:0/4121546278 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fe508039380 0x7fe50803b840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe510006fd0 tx=0x7fe510006e40 comp rx=0 tx=0).stop 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.019+0000 7fe526a3c700 1 -- 172.21.15.16:0/4121546278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 msgr2=0x7fe52019c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.019+0000 7fe526a3c700 1 --2- 172.21.15.16:0/4121546278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52019c2c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe514003fb0 tx=0x7fe514011bd0 comp rx=0 tx=0).stop 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.019+0000 7fe526a3c700 1 -- 172.21.15.16:0/4121546278 shutdown_connections 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.019+0000 7fe526a3c700 1 --2- 172.21.15.16:0/4121546278 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fe508039380 0x7fe50803b840 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.019+0000 7fe526a3c700 1 --2- 172.21.15.16:0/4121546278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe520108d60 0x7fe52019c2c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:55.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.019+0000 7fe526a3c700 1 -- 172.21.15.16:0/4121546278 >> 172.21.15.16:0/4121546278 conn(0x7fe520102790 msgr2=0x7fe5201032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:55.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.019+0000 7fe526a3c700 1 -- 172.21.15.16:0/4121546278 shutdown_connections 2024-03-27T06:33:55.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:55.019+0000 7fe526a3c700 1 -- 172.21.15.16:0/4121546278 wait complete. 2024-03-27T06:33:55.630 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-03-27T06:33:56.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-mon[31908]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:33:56.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-mon[31908]: Generating ssh key... 2024-03-27T06:33:56.824 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: Generating public/private rsa key pair. 2024-03-27T06:33:56.824 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: Your identification has been saved in /tmp/tmp_lyevfpy/key. 2024-03-27T06:33:56.824 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: Your public key has been saved in /tmp/tmp_lyevfpy/key.pub. 2024-03-27T06:33:56.824 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: The key fingerprint is: 2024-03-27T06:33:56.824 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: SHA256:53GgWZu21rjE8kBq9Y1sannIulR8wsT6ciiOY5K6EAY ceph-c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:33:56.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: The key's randomart image is: 2024-03-27T06:33:56.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: +---[RSA 3072]----+ 2024-03-27T06:33:56.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: | | 2024-03-27T06:33:56.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: | . | 2024-03-27T06:33:56.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: |E o o | 2024-03-27T06:33:56.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: |. = + + | 2024-03-27T06:33:56.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: |.. . S B . | 2024-03-27T06:33:56.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: |.. B X O | 2024-03-27T06:33:56.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: |.. . *.=o% o | 2024-03-27T06:33:56.825 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: |+ oo + o=O.. | 2024-03-27T06:33:56.826 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: |++... o+..o | 2024-03-27T06:33:56.826 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:33:56 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: +----[SHA256]-----+ 2024-03-27T06:33:57.065 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.052+0000 7fc066e9e700 1 Processor -- start 2024-03-27T06:33:57.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.052+0000 7fc066e9e700 1 -- start start 2024-03-27T06:33:57.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.052+0000 7fc066e9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc060108730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:57.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.052+0000 7fc066e9e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc060108c70 con 0x7fc060108310 2024-03-27T06:33:57.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.052+0000 7fc064c3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc060108730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:57.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.052+0000 7fc064c3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc060108730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40398/0 (socket says 172.21.15.16:40398) 2024-03-27T06:33:57.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.052+0000 7fc064c3a700 1 -- 172.21.15.16:0/3780302668 learned_addr learned my addr 172.21.15.16:0/3780302668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:57.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.052+0000 7fc064c3a700 1 -- 172.21.15.16:0/3780302668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc060108df0 con 0x7fc060108310 2024-03-27T06:33:57.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.053+0000 7fc064c3a700 1 --2- 172.21.15.16:0/3780302668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc060108730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc0540098a0 tx=0x7fc054009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771eb821271c209e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:57.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.053+0000 7fc05f7fe700 1 -- 172.21.15.16:0/3780302668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc054004160 con 0x7fc060108310 2024-03-27T06:33:57.066 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.053+0000 7fc05f7fe700 1 -- 172.21.15.16:0/3780302668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fc054004300 con 0x7fc060108310 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.053+0000 7fc05f7fe700 1 -- 172.21.15.16:0/3780302668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc054011c00 con 0x7fc060108310 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.053+0000 7fc066e9e700 1 -- 172.21.15.16:0/3780302668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 msgr2=0x7fc060108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.053+0000 7fc066e9e700 1 --2- 172.21.15.16:0/3780302668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc060108730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc0540098a0 tx=0x7fc054009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.053+0000 7fc066e9e700 1 -- 172.21.15.16:0/3780302668 shutdown_connections 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.053+0000 7fc066e9e700 1 --2- 172.21.15.16:0/3780302668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc060108730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.053+0000 7fc066e9e700 1 -- 172.21.15.16:0/3780302668 >> 172.21.15.16:0/3780302668 conn(0x7fc060103990 msgr2=0x7fc060105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.053+0000 7fc066e9e700 1 -- 172.21.15.16:0/3780302668 shutdown_connections 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.053+0000 7fc066e9e700 1 -- 172.21.15.16:0/3780302668 wait complete. 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.054+0000 7fc066e9e700 1 Processor -- start 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.054+0000 7fc066e9e700 1 -- start start 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.054+0000 7fc066e9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc06019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.054+0000 7fc066e9e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc06010c780 con 0x7fc060108310 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.054+0000 7fc064c3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc06019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:57.067 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.054+0000 7fc064c3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc06019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40404/0 (socket says 172.21.15.16:40404) 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.054+0000 7fc064c3a700 1 -- 172.21.15.16:0/867671759 learned_addr learned my addr 172.21.15.16:0/867671759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.055+0000 7fc064c3a700 1 -- 172.21.15.16:0/867671759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06019c7f0 con 0x7fc060108310 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.055+0000 7fc064c3a700 1 --2- 172.21.15.16:0/867671759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc06019c2b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc054011bd0 tx=0x7fc054011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.055+0000 7fc05dffb700 1 -- 172.21.15.16:0/867671759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc054003b20 con 0x7fc060108310 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.055+0000 7fc05dffb700 1 -- 172.21.15.16:0/867671759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fc054003cc0 con 0x7fc060108310 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.055+0000 7fc066e9e700 1 -- 172.21.15.16:0/867671759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06019ca80 con 0x7fc060108310 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.055+0000 7fc05dffb700 1 -- 172.21.15.16:0/867671759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc054024930 con 0x7fc060108310 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.055+0000 7fc066e9e700 1 -- 172.21.15.16:0/867671759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06019f770 con 0x7fc060108310 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.056+0000 7fc05dffb700 1 -- 172.21.15.16:0/867671759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7fc054022020 con 0x7fc060108310 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.056+0000 7fc05dffb700 1 --2- 172.21.15.16:0/867671759 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc050039420 0x7fc05003b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.056+0000 7fc05dffb700 1 -- 172.21.15.16:0/867671759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc05404d540 con 0x7fc060108310 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.056+0000 7fc05ffff700 1 --2- 172.21.15.16:0/867671759 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc050039420 0x7fc05003b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:57.068 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.056+0000 7fc066e9e700 1 -- 172.21.15.16:0/867671759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc06010c900 con 0x7fc060108310 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.056+0000 7fc05ffff700 1 --2- 172.21.15.16:0/867671759 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc050039420 0x7fc05003b8e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc04c006fd0 tx=0x7fc04c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.061+0000 7fc05dffb700 1 -- 172.21.15.16:0/867671759 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc05401a950 con 0x7fc060108310 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.270+0000 7fc066e9e700 1 -- 172.21.15.16:0/867671759 --> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fc060104b40 con 0x7fc050039420 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.538+0000 7fc05dffb700 1 -- 172.21.15.16:0/867671759 <== mgr.14116 v2:172.21.15.16:6800/1685933023 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc060104b40 con 0x7fc050039420 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.540+0000 7fc066e9e700 1 -- 172.21.15.16:0/867671759 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc050039420 msgr2=0x7fc05003b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.540+0000 7fc066e9e700 1 --2- 172.21.15.16:0/867671759 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc050039420 0x7fc05003b8e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc04c006fd0 tx=0x7fc04c006e40 comp rx=0 tx=0).stop 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.540+0000 7fc066e9e700 1 -- 172.21.15.16:0/867671759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 msgr2=0x7fc06019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.540+0000 7fc066e9e700 1 --2- 172.21.15.16:0/867671759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc06019c2b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc054011bd0 tx=0x7fc054011cb0 comp rx=0 tx=0).stop 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.540+0000 7fc066e9e700 1 -- 172.21.15.16:0/867671759 shutdown_connections 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.540+0000 7fc066e9e700 1 --2- 172.21.15.16:0/867671759 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc050039420 0x7fc05003b8e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.540+0000 7fc066e9e700 1 --2- 172.21.15.16:0/867671759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc060108310 0x7fc06019c2b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.540+0000 7fc066e9e700 1 -- 172.21.15.16:0/867671759 >> 172.21.15.16:0/867671759 conn(0x7fc060103990 msgr2=0x7fc060104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:57.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.540+0000 7fc066e9e700 1 -- 172.21.15.16:0/867671759 shutdown_connections 2024-03-27T06:33:57.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:56.540+0000 7fc066e9e700 1 -- 172.21.15.16:0/867671759 wait complete. 2024-03-27T06:33:57.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:57 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:33:57.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:57 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:33:58.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI4CrxITdb3xZjmn8nh7O0RL5aLUEGF6QHCFBJi304Ok6t9YNvf3ifmo1ibUgyaw2WYwABPn2BSRLzeKvEtT5psis1a8pctXgE8ZpVW4BMSvigNUjU2WHWbaaZZ4LTxna8dQPhUPyYtB0Xq2gZX2CudBKdjxUrBnuDOdbSI+IV08+SjvjumEutuz1swhMfuSpqnxMUpLHFuMaxcAR7xGRjmsuhYNoYIGfLTvZTg3pgQJMn3iv0SlEIDrV6rfLIP1I6Id7VnGq+sOYMITZGKsETDwm7FdOl3S7Uh/GyCZHAN49P+kNfC7dA+mqG94Wv2BuJ8ab+1ufxrB/KK4IwbjOeIaiLydlvK8VznzTSm3G2ZOh3E22HFDVffeCDO4LxhyOo2BxcOiZoDCf1HkrW+2go+jB3QzRJ8qnEr3REv27wAVfTaPds54WOTNJsCoNAmf71bJbxLYZfQH1vkxp/njaPQ8zpOtUQIedcSXsLX+pgd5jsqkYqTdvZv3FyZxryGm0= ceph-c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:33:58.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.528+0000 7f7a47e3c700 1 Processor -- start 2024-03-27T06:33:58.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.528+0000 7f7a47e3c700 1 -- start start 2024-03-27T06:33:58.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.528+0000 7f7a47e3c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4010b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:58.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.528+0000 7f7a47e3c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4007beb0 con 0x7f7a40108d40 2024-03-27T06:33:58.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.528+0000 7f7a45bd8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4010b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:58.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.528+0000 7f7a45bd8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4010b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40420/0 (socket says 172.21.15.16:40420) 2024-03-27T06:33:58.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.528+0000 7f7a45bd8700 1 -- 172.21.15.16:0/1160182945 learned_addr learned my addr 172.21.15.16:0/1160182945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:58.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.528+0000 7f7a45bd8700 1 -- 172.21.15.16:0/1160182945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4010b670 con 0x7f7a40108d40 2024-03-27T06:33:58.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.529+0000 7f7a45bd8700 1 --2- 172.21.15.16:0/1160182945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4010b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0098a0 tx=0x7f7a3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bf3646ae0f7fa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.529+0000 7f7a44bd6700 1 -- 172.21.15.16:0/1160182945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c004160 con 0x7f7a40108d40 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.529+0000 7f7a44bd6700 1 -- 172.21.15.16:0/1160182945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f7a3c004300 con 0x7f7a40108d40 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.529+0000 7f7a44bd6700 1 -- 172.21.15.16:0/1160182945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c011c00 con 0x7f7a40108d40 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.529+0000 7f7a47e3c700 1 -- 172.21.15.16:0/1160182945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 msgr2=0x7f7a4010b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.529+0000 7f7a47e3c700 1 --2- 172.21.15.16:0/1160182945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4010b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0098a0 tx=0x7f7a3c009bb0 comp rx=0 tx=0).stop 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.529+0000 7f7a47e3c700 1 -- 172.21.15.16:0/1160182945 shutdown_connections 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.529+0000 7f7a47e3c700 1 --2- 172.21.15.16:0/1160182945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4010b130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.529+0000 7f7a47e3c700 1 -- 172.21.15.16:0/1160182945 >> 172.21.15.16:0/1160182945 conn(0x7f7a401027b0 msgr2=0x7f7a40104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.530+0000 7f7a47e3c700 1 -- 172.21.15.16:0/1160182945 shutdown_connections 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.530+0000 7f7a47e3c700 1 -- 172.21.15.16:0/1160182945 wait complete. 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.530+0000 7f7a47e3c700 1 Processor -- start 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.530+0000 7f7a47e3c700 1 -- start start 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.530+0000 7f7a47e3c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:58.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.530+0000 7f7a47e3c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4010c700 con 0x7f7a40108d40 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.530+0000 7f7a45bd8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.531+0000 7f7a45bd8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4019c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40426/0 (socket says 172.21.15.16:40426) 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.531+0000 7f7a45bd8700 1 -- 172.21.15.16:0/3495733329 learned_addr learned my addr 172.21.15.16:0/3495733329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.531+0000 7f7a45bd8700 1 -- 172.21.15.16:0/3495733329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4019c8d0 con 0x7f7a40108d40 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.531+0000 7f7a45bd8700 1 --2- 172.21.15.16:0/3495733329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4019c390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009fb0 tx=0x7f7a3c011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.531+0000 7f7a36ffd700 1 -- 172.21.15.16:0/3495733329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c0038b0 con 0x7f7a40108d40 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.531+0000 7f7a36ffd700 1 -- 172.21.15.16:0/3495733329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f7a3c01a430 con 0x7f7a40108d40 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.531+0000 7f7a47e3c700 1 -- 172.21.15.16:0/3495733329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4019cb60 con 0x7f7a40108d40 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.531+0000 7f7a36ffd700 1 -- 172.21.15.16:0/3495733329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c019630 con 0x7f7a40108d40 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.531+0000 7f7a47e3c700 1 -- 172.21.15.16:0/3495733329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4019f850 con 0x7f7a40108d40 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.532+0000 7f7a36ffd700 1 -- 172.21.15.16:0/3495733329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f7a3c019930 con 0x7f7a40108d40 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.532+0000 7f7a47e3c700 1 -- 172.21.15.16:0/3495733329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a40063c70 con 0x7f7a40108d40 2024-03-27T06:33:58.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.532+0000 7f7a36ffd700 1 --2- 172.21.15.16:0/3495733329 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f7a2c039380 0x7f7a2c03b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.532+0000 7f7a36ffd700 1 -- 172.21.15.16:0/3495733329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7a3c04d930 con 0x7f7a40108d40 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.532+0000 7f7a453d7700 1 --2- 172.21.15.16:0/3495733329 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f7a2c039380 0x7f7a2c03b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.533+0000 7f7a453d7700 1 --2- 172.21.15.16:0/3495733329 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f7a2c039380 0x7f7a2c03b840 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a30006fd0 tx=0x7f7a30006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.537+0000 7f7a36ffd700 1 -- 172.21.15.16:0/3495733329 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7a3c019d20 con 0x7f7a40108d40 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.747+0000 7f7a47e3c700 1 -- 172.21.15.16:0/3495733329 --> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7a40103a70 con 0x7f7a2c039380 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.748+0000 7f7a36ffd700 1 -- 172.21.15.16:0/3495733329 <== mgr.14116 v2:172.21.15.16:6800/1685933023 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7a40103a70 con 0x7f7a2c039380 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.750+0000 7f7a47e3c700 1 -- 172.21.15.16:0/3495733329 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f7a2c039380 msgr2=0x7f7a2c03b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.750+0000 7f7a47e3c700 1 --2- 172.21.15.16:0/3495733329 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f7a2c039380 0x7f7a2c03b840 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a30006fd0 tx=0x7f7a30006e40 comp rx=0 tx=0).stop 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.750+0000 7f7a47e3c700 1 -- 172.21.15.16:0/3495733329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 msgr2=0x7f7a4019c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.750+0000 7f7a47e3c700 1 --2- 172.21.15.16:0/3495733329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4019c390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009fb0 tx=0x7f7a3c011c60 comp rx=0 tx=0).stop 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.750+0000 7f7a47e3c700 1 -- 172.21.15.16:0/3495733329 shutdown_connections 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.750+0000 7f7a47e3c700 1 --2- 172.21.15.16:0/3495733329 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f7a2c039380 0x7f7a2c03b840 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.750+0000 7f7a47e3c700 1 --2- 172.21.15.16:0/3495733329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a40108d40 0x7f7a4019c390 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:33:58.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.751+0000 7f7a47e3c700 1 -- 172.21.15.16:0/3495733329 >> 172.21.15.16:0/3495733329 conn(0x7f7a401027b0 msgr2=0x7f7a40103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:33:58.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.751+0000 7f7a47e3c700 1 -- 172.21.15.16:0/3495733329 shutdown_connections 2024-03-27T06:33:58.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:57.751+0000 7f7a47e3c700 1 -- 172.21.15.16:0/3495733329 wait complete. 2024-03-27T06:33:58.286 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-27T06:33:58.286 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-03-27T06:33:58.286 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-03-27T06:33:58.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:58 smithi016 ceph-mon[31908]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:33:59.783 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:33:59 smithi016 ceph-mon[31908]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:34:00.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:00 smithi016 ceph-mon[31908]: Deploying cephadm binary to smithi016 2024-03-27T06:34:03.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-03-27T06:34:03.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.664+0000 7fc04dc67700 1 Processor -- start 2024-03-27T06:34:03.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.664+0000 7fc04dc67700 1 -- start start 2024-03-27T06:34:03.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.664+0000 7fc04dc67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc0481075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:03.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.664+0000 7fc04dc67700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc048107bc0 con 0x7fc0481071d0 2024-03-27T06:34:03.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.664+0000 7fc0477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc0481075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:03.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.664+0000 7fc0477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc0481075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40434/0 (socket says 172.21.15.16:40434) 2024-03-27T06:34:03.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.664+0000 7fc0477fe700 1 -- 172.21.15.16:0/1986378869 learned_addr learned my addr 172.21.15.16:0/1986378869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:03.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.664+0000 7fc0477fe700 1 -- 172.21.15.16:0/1986378869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0481083d0 con 0x7fc0481071d0 2024-03-27T06:34:03.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.664+0000 7fc0477fe700 1 --2- 172.21.15.16:0/1986378869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc0481075f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc0380098a0 tx=0x7fc038009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f271d4ee54f3da0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:03.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.665+0000 7fc0467fc700 1 -- 172.21.15.16:0/1986378869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc038004160 con 0x7fc0481071d0 2024-03-27T06:34:03.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.665+0000 7fc0467fc700 1 -- 172.21.15.16:0/1986378869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fc038004300 con 0x7fc0481071d0 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.665+0000 7fc0467fc700 1 -- 172.21.15.16:0/1986378869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc038011c00 con 0x7fc0481071d0 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.665+0000 7fc04dc67700 1 -- 172.21.15.16:0/1986378869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 msgr2=0x7fc0481075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.665+0000 7fc04dc67700 1 --2- 172.21.15.16:0/1986378869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc0481075f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc0380098a0 tx=0x7fc038009bb0 comp rx=0 tx=0).stop 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.665+0000 7fc04dc67700 1 -- 172.21.15.16:0/1986378869 shutdown_connections 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.665+0000 7fc04dc67700 1 --2- 172.21.15.16:0/1986378869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc0481075f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.665+0000 7fc04dc67700 1 -- 172.21.15.16:0/1986378869 >> 172.21.15.16:0/1986378869 conn(0x7fc0481027b0 msgr2=0x7fc048104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.665+0000 7fc04dc67700 1 -- 172.21.15.16:0/1986378869 shutdown_connections 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.665+0000 7fc04dc67700 1 -- 172.21.15.16:0/1986378869 wait complete. 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.666+0000 7fc04dc67700 1 Processor -- start 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.666+0000 7fc04dc67700 1 -- start start 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.666+0000 7fc04dc67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc04819c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:03.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.666+0000 7fc04dc67700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc04810b780 con 0x7fc0481071d0 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.666+0000 7fc0477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc04819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.666+0000 7fc0477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc04819c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40448/0 (socket says 172.21.15.16:40448) 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.666+0000 7fc0477fe700 1 -- 172.21.15.16:0/2244839252 learned_addr learned my addr 172.21.15.16:0/2244839252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.666+0000 7fc0477fe700 1 -- 172.21.15.16:0/2244839252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04819c8a0 con 0x7fc0481071d0 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.667+0000 7fc0477fe700 1 --2- 172.21.15.16:0/2244839252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc04819c360 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc038009fb0 tx=0x7fc038011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.667+0000 7fc044ff9700 1 -- 172.21.15.16:0/2244839252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0380038b0 con 0x7fc0481071d0 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.667+0000 7fc044ff9700 1 -- 172.21.15.16:0/2244839252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fc03801a430 con 0x7fc0481071d0 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.667+0000 7fc04dc67700 1 -- 172.21.15.16:0/2244839252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04819cb30 con 0x7fc0481071d0 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.667+0000 7fc044ff9700 1 -- 172.21.15.16:0/2244839252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0380195e0 con 0x7fc0481071d0 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.667+0000 7fc04dc67700 1 -- 172.21.15.16:0/2244839252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04819f820 con 0x7fc0481071d0 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.668+0000 7fc044ff9700 1 -- 172.21.15.16:0/2244839252 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7fc038003a50 con 0x7fc0481071d0 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.668+0000 7fc04dc67700 1 -- 172.21.15.16:0/2244839252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04810b900 con 0x7fc0481071d0 2024-03-27T06:34:03.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.668+0000 7fc044ff9700 1 --2- 172.21.15.16:0/2244839252 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc0300393d0 0x7fc03003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.668+0000 7fc044ff9700 1 -- 172.21.15.16:0/2244839252 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc03804d7f0 con 0x7fc0481071d0 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.668+0000 7fc046ffd700 1 --2- 172.21.15.16:0/2244839252 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc0300393d0 0x7fc03003b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.669+0000 7fc046ffd700 1 --2- 172.21.15.16:0/2244839252 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc0300393d0 0x7fc03003b890 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc03c006fd0 tx=0x7fc03c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.673+0000 7fc044ff9700 1 -- 172.21.15.16:0/2244839252 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc03801ab40 con 0x7fc0481071d0 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:33:58.884+0000 7fc04dc67700 1 -- 172.21.15.16:0/2244839252 --> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}) -- 0x7fc04802e130 con 0x7fc0300393d0 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:03.295+0000 7fc044ff9700 1 -- 172.21.15.16:0/2244839252 <== mgr.14116 v2:172.21.15.16:6800/1685933023 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc04802e130 con 0x7fc0300393d0 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:03.298+0000 7fc04dc67700 1 -- 172.21.15.16:0/2244839252 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc0300393d0 msgr2=0x7fc03003b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:03.298+0000 7fc04dc67700 1 --2- 172.21.15.16:0/2244839252 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc0300393d0 0x7fc03003b890 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc03c006fd0 tx=0x7fc03c006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:03.298+0000 7fc04dc67700 1 -- 172.21.15.16:0/2244839252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 msgr2=0x7fc04819c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:03.298+0000 7fc04dc67700 1 --2- 172.21.15.16:0/2244839252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc04819c360 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc038009fb0 tx=0x7fc038011c60 comp rx=0 tx=0).stop 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:03.298+0000 7fc04dc67700 1 -- 172.21.15.16:0/2244839252 shutdown_connections 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:03.298+0000 7fc04dc67700 1 --2- 172.21.15.16:0/2244839252 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fc0300393d0 0x7fc03003b890 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:03.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:03.298+0000 7fc04dc67700 1 --2- 172.21.15.16:0/2244839252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0481071d0 0x7fc04819c360 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:03.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:03.298+0000 7fc04dc67700 1 -- 172.21.15.16:0/2244839252 >> 172.21.15.16:0/2244839252 conn(0x7fc0481027b0 msgr2=0x7fc0481036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:03.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:03.298+0000 7fc04dc67700 1 -- 172.21.15.16:0/2244839252 shutdown_connections 2024-03-27T06:34:03.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:03.298+0000 7fc04dc67700 1 -- 172.21.15.16:0/2244839252 wait complete. 2024-03-27T06:34:03.869 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mon service... 2024-03-27T06:34:04.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:04 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:34:04.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:04 smithi016 ceph-mon[31908]: Added host smithi016 2024-03-27T06:34:04.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:04 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:34:05.792 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:05 smithi016 ceph-mon[31908]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:34:05.792 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:05 smithi016 ceph-mon[31908]: Saving service mon spec with placement count:5 2024-03-27T06:34:05.792 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:05 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:34:06.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.345+0000 7fb14ce65700 1 Processor -- start 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.345+0000 7fb14ce65700 1 -- start start 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.345+0000 7fb14ce65700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14810b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.345+0000 7fb14ce65700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb14807be50 con 0x7fb148108da0 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.345+0000 7fb14659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14810b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.345+0000 7fb14659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14810b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40454/0 (socket says 172.21.15.16:40454) 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.345+0000 7fb14659c700 1 -- 172.21.15.16:0/3102470738 learned_addr learned my addr 172.21.15.16:0/3102470738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.346+0000 7fb14659c700 1 -- 172.21.15.16:0/3102470738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14810b6d0 con 0x7fb148108da0 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.346+0000 7fb14659c700 1 --2- 172.21.15.16:0/3102470738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14810b190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1380098a0 tx=0x7fb138009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c71319565350e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.346+0000 7fb14559a700 1 -- 172.21.15.16:0/3102470738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb138004160 con 0x7fb148108da0 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.346+0000 7fb14559a700 1 -- 172.21.15.16:0/3102470738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fb138004300 con 0x7fb148108da0 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.346+0000 7fb14559a700 1 -- 172.21.15.16:0/3102470738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb138011c00 con 0x7fb148108da0 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.346+0000 7fb14ce65700 1 -- 172.21.15.16:0/3102470738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 msgr2=0x7fb14810b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.346+0000 7fb14ce65700 1 --2- 172.21.15.16:0/3102470738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14810b190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1380098a0 tx=0x7fb138009bb0 comp rx=0 tx=0).stop 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.347+0000 7fb14ce65700 1 -- 172.21.15.16:0/3102470738 shutdown_connections 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.347+0000 7fb14ce65700 1 --2- 172.21.15.16:0/3102470738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14810b190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.347+0000 7fb14ce65700 1 -- 172.21.15.16:0/3102470738 >> 172.21.15.16:0/3102470738 conn(0x7fb1481027b0 msgr2=0x7fb148104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.347+0000 7fb14ce65700 1 -- 172.21.15.16:0/3102470738 shutdown_connections 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.347+0000 7fb14ce65700 1 -- 172.21.15.16:0/3102470738 wait complete. 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.347+0000 7fb14ce65700 1 Processor -- start 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.347+0000 7fb14ce65700 1 -- start start 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.347+0000 7fb14ce65700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.348+0000 7fb14ce65700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb14810c700 con 0x7fb148108da0 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.348+0000 7fb14659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.348+0000 7fb14659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14819c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40464/0 (socket says 172.21.15.16:40464) 2024-03-27T06:34:06.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.348+0000 7fb14659c700 1 -- 172.21.15.16:0/1164601735 learned_addr learned my addr 172.21.15.16:0/1164601735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.348+0000 7fb14659c700 1 -- 172.21.15.16:0/1164601735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14819c890 con 0x7fb148108da0 2024-03-27T06:34:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.348+0000 7fb14659c700 1 --2- 172.21.15.16:0/1164601735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14819c350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb138009fb0 tx=0x7fb138011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.348+0000 7fb1377fe700 1 -- 172.21.15.16:0/1164601735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1380038b0 con 0x7fb148108da0 2024-03-27T06:34:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.348+0000 7fb1377fe700 1 -- 172.21.15.16:0/1164601735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fb13801a430 con 0x7fb148108da0 2024-03-27T06:34:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.348+0000 7fb14ce65700 1 -- 172.21.15.16:0/1164601735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14819cb20 con 0x7fb148108da0 2024-03-27T06:34:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.348+0000 7fb1377fe700 1 -- 172.21.15.16:0/1164601735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb138019630 con 0x7fb148108da0 2024-03-27T06:34:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.348+0000 7fb14ce65700 1 -- 172.21.15.16:0/1164601735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14819f810 con 0x7fb148108da0 2024-03-27T06:34:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.349+0000 7fb1377fe700 1 -- 172.21.15.16:0/1164601735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7fb138019930 con 0x7fb148108da0 2024-03-27T06:34:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.349+0000 7fb1377fe700 1 --2- 172.21.15.16:0/1164601735 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fb130039380 0x7fb13003b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.349+0000 7fb14ce65700 1 -- 172.21.15.16:0/1164601735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb148063c70 con 0x7fb148108da0 2024-03-27T06:34:06.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.349+0000 7fb1377fe700 1 -- 172.21.15.16:0/1164601735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb13804d930 con 0x7fb148108da0 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.349+0000 7fb145d9b700 1 --2- 172.21.15.16:0/1164601735 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fb130039380 0x7fb13003b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.350+0000 7fb145d9b700 1 --2- 172.21.15.16:0/1164601735 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fb130039380 0x7fb13003b840 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb13c006fd0 tx=0x7fb13c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.354+0000 7fb1377fe700 1 -- 172.21.15.16:0/1164601735 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb13800bde0 con 0x7fb148108da0 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.593+0000 7fb14ce65700 1 -- 172.21.15.16:0/1164601735 --> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb148103a40 con 0x7fb130039380 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.597+0000 7fb1377fe700 1 -- 172.21.15.16:0/1164601735 <== mgr.14116 v2:172.21.15.16:6800/1685933023 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb148103a40 con 0x7fb130039380 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.599+0000 7fb14ce65700 1 -- 172.21.15.16:0/1164601735 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fb130039380 msgr2=0x7fb13003b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.599+0000 7fb14ce65700 1 --2- 172.21.15.16:0/1164601735 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fb130039380 0x7fb13003b840 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb13c006fd0 tx=0x7fb13c006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.599+0000 7fb14ce65700 1 -- 172.21.15.16:0/1164601735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 msgr2=0x7fb14819c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.599+0000 7fb14ce65700 1 --2- 172.21.15.16:0/1164601735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14819c350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb138009fb0 tx=0x7fb138011c60 comp rx=0 tx=0).stop 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.599+0000 7fb14ce65700 1 -- 172.21.15.16:0/1164601735 shutdown_connections 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.599+0000 7fb14ce65700 1 --2- 172.21.15.16:0/1164601735 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fb130039380 0x7fb13003b840 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.599+0000 7fb14ce65700 1 --2- 172.21.15.16:0/1164601735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb148108da0 0x7fb14819c350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:06.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.599+0000 7fb14ce65700 1 -- 172.21.15.16:0/1164601735 >> 172.21.15.16:0/1164601735 conn(0x7fb1481027b0 msgr2=0x7fb148103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:06.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.599+0000 7fb14ce65700 1 -- 172.21.15.16:0/1164601735 shutdown_connections 2024-03-27T06:34:06.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:04.599+0000 7fb14ce65700 1 -- 172.21.15.16:0/1164601735 wait complete. 2024-03-27T06:34:06.276 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mgr service... 2024-03-27T06:34:07.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:06 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:34:07.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:06 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-27T06:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.752+0000 7fbba42ce700 1 Processor -- start 2024-03-27T06:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.752+0000 7fbba42ce700 1 -- start start 2024-03-27T06:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.752+0000 7fbba42ce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c106990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.752+0000 7fbba42ce700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c106f60 con 0x7fbb9c106570 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.752+0000 7fbba206a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c106990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.752+0000 7fbba206a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c106990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40924/0 (socket says 172.21.15.16:40924) 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.752+0000 7fbba206a700 1 -- 172.21.15.16:0/528533104 learned_addr learned my addr 172.21.15.16:0/528533104 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.752+0000 7fbba206a700 1 -- 172.21.15.16:0/528533104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c109550 con 0x7fbb9c106570 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.753+0000 7fbba206a700 1 --2- 172.21.15.16:0/528533104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c106990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb980098a0 tx=0x7fbb98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff5f9b2bbe2402f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.753+0000 7fbba1068700 1 -- 172.21.15.16:0/528533104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb98004160 con 0x7fbb9c106570 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.753+0000 7fbba1068700 1 -- 172.21.15.16:0/528533104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb98004300 con 0x7fbb9c106570 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.753+0000 7fbba1068700 1 -- 172.21.15.16:0/528533104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb98011bb0 con 0x7fbb9c106570 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.753+0000 7fbba42ce700 1 -- 172.21.15.16:0/528533104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 msgr2=0x7fbb9c106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.753+0000 7fbba42ce700 1 --2- 172.21.15.16:0/528533104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c106990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb980098a0 tx=0x7fbb98009bb0 comp rx=0 tx=0).stop 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.754+0000 7fbba42ce700 1 -- 172.21.15.16:0/528533104 shutdown_connections 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.754+0000 7fbba42ce700 1 --2- 172.21.15.16:0/528533104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c106990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.754+0000 7fbba42ce700 1 -- 172.21.15.16:0/528533104 >> 172.21.15.16:0/528533104 conn(0x7fbb9c101f30 msgr2=0x7fbb9c104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.754+0000 7fbba42ce700 1 -- 172.21.15.16:0/528533104 shutdown_connections 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.754+0000 7fbba42ce700 1 -- 172.21.15.16:0/528533104 wait complete. 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.754+0000 7fbba42ce700 1 Processor -- start 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.754+0000 7fbba42ce700 1 -- start start 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.754+0000 7fbba42ce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.754+0000 7fbba42ce700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c10c900 con 0x7fbb9c106570 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.755+0000 7fbba206a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.755+0000 7fbba206a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c19c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40938/0 (socket says 172.21.15.16:40938) 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.755+0000 7fbba206a700 1 -- 172.21.15.16:0/1776936128 learned_addr learned my addr 172.21.15.16:0/1776936128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.755+0000 7fbba206a700 1 -- 172.21.15.16:0/1776936128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c19c8d0 con 0x7fbb9c106570 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.755+0000 7fbba206a700 1 --2- 172.21.15.16:0/1776936128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c19c390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb98011870 tx=0x7fbb98011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.755+0000 7fbb937fe700 1 -- 172.21.15.16:0/1776936128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb98003970 con 0x7fbb9c106570 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.755+0000 7fbba42ce700 1 -- 172.21.15.16:0/1776936128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c19cb60 con 0x7fbb9c106570 2024-03-27T06:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.755+0000 7fbb937fe700 1 -- 172.21.15.16:0/1776936128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb98003b10 con 0x7fbb9c106570 2024-03-27T06:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.755+0000 7fbb937fe700 1 -- 172.21.15.16:0/1776936128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb98019700 con 0x7fbb9c106570 2024-03-27T06:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.755+0000 7fbba42ce700 1 -- 172.21.15.16:0/1776936128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c19cfc0 con 0x7fbb9c106570 2024-03-27T06:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.756+0000 7fbba42ce700 1 -- 172.21.15.16:0/1776936128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9c063c70 con 0x7fbb9c106570 2024-03-27T06:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.756+0000 7fbb937fe700 1 -- 172.21.15.16:0/1776936128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7fbb98019a00 con 0x7fbb9c106570 2024-03-27T06:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.757+0000 7fbb937fe700 1 --2- 172.21.15.16:0/1776936128 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fbb88039380 0x7fbb8803b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.757+0000 7fbb937fe700 1 -- 172.21.15.16:0/1776936128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbb9804dae0 con 0x7fbb9c106570 2024-03-27T06:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.757+0000 7fbba1869700 1 --2- 172.21.15.16:0/1776936128 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fbb88039380 0x7fbb8803b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.757+0000 7fbba1869700 1 --2- 172.21.15.16:0/1776936128 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fbb88039380 0x7fbb8803b840 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c006fd0 tx=0x7fbb8c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:06.762+0000 7fbb937fe700 1 -- 172.21.15.16:0/1776936128 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fbb98019df0 con 0x7fbb9c106570 2024-03-27T06:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.017+0000 7fbba42ce700 1 -- 172.21.15.16:0/1776936128 --> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbb9c1031d0 con 0x7fbb88039380 2024-03-27T06:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.021+0000 7fbb937fe700 1 -- 172.21.15.16:0/1776936128 <== mgr.14116 v2:172.21.15.16:6800/1685933023 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbb9c1031d0 con 0x7fbb88039380 2024-03-27T06:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.023+0000 7fbba42ce700 1 -- 172.21.15.16:0/1776936128 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fbb88039380 msgr2=0x7fbb8803b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.023+0000 7fbba42ce700 1 --2- 172.21.15.16:0/1776936128 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fbb88039380 0x7fbb8803b840 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c006fd0 tx=0x7fbb8c006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.023+0000 7fbba42ce700 1 -- 172.21.15.16:0/1776936128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 msgr2=0x7fbb9c19c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.023+0000 7fbba42ce700 1 --2- 172.21.15.16:0/1776936128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c19c390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb98011870 tx=0x7fbb98011c30 comp rx=0 tx=0).stop 2024-03-27T06:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.023+0000 7fbba42ce700 1 -- 172.21.15.16:0/1776936128 shutdown_connections 2024-03-27T06:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.023+0000 7fbba42ce700 1 --2- 172.21.15.16:0/1776936128 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7fbb88039380 0x7fbb8803b840 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.023+0000 7fbba42ce700 1 --2- 172.21.15.16:0/1776936128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c106570 0x7fbb9c19c390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.023+0000 7fbba42ce700 1 -- 172.21.15.16:0/1776936128 >> 172.21.15.16:0/1776936128 conn(0x7fbb9c101f30 msgr2=0x7fbb9c102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.024+0000 7fbba42ce700 1 -- 172.21.15.16:0/1776936128 shutdown_connections 2024-03-27T06:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:07.024+0000 7fbba42ce700 1 -- 172.21.15.16:0/1776936128 wait complete. 2024-03-27T06:34:08.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:08 smithi016 ceph-mon[31908]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:34:08.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:08 smithi016 ceph-mon[31908]: Saving service mgr spec with placement count:2 2024-03-27T06:34:08.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:08 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:34:09.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.633+0000 7f06d778b700 1 Processor -- start 2024-03-27T06:34:09.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.633+0000 7f06d778b700 1 -- start start 2024-03-27T06:34:09.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.633+0000 7f06d778b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d0106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:09.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.633+0000 7f06d778b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06d007be30 con 0x7f06d0106b90 2024-03-27T06:34:09.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.633+0000 7f06d5527700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d0106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:09.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.633+0000 7f06d5527700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d0106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40946/0 (socket says 172.21.15.16:40946) 2024-03-27T06:34:09.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.633+0000 7f06d5527700 1 -- 172.21.15.16:0/3792423528 learned_addr learned my addr 172.21.15.16:0/3792423528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:09.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.634+0000 7f06d5527700 1 -- 172.21.15.16:0/3792423528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d01074f0 con 0x7f06d0106b90 2024-03-27T06:34:09.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.634+0000 7f06d5527700 1 --2- 172.21.15.16:0/3792423528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d0106fb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f06c40098a0 tx=0x7f06c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd39f750dc53c64c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:09.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.634+0000 7f06c3fff700 1 -- 172.21.15.16:0/3792423528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06c4004160 con 0x7f06d0106b90 2024-03-27T06:34:09.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.634+0000 7f06c3fff700 1 -- 172.21.15.16:0/3792423528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f06c4004300 con 0x7f06d0106b90 2024-03-27T06:34:09.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.634+0000 7f06c3fff700 1 -- 172.21.15.16:0/3792423528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06c4011bb0 con 0x7f06d0106b90 2024-03-27T06:34:09.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.635+0000 7f06d778b700 1 -- 172.21.15.16:0/3792423528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 msgr2=0x7f06d0106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:09.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.635+0000 7f06d778b700 1 --2- 172.21.15.16:0/3792423528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d0106fb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f06c40098a0 tx=0x7f06c4009bb0 comp rx=0 tx=0).stop 2024-03-27T06:34:09.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.635+0000 7f06d778b700 1 -- 172.21.15.16:0/3792423528 shutdown_connections 2024-03-27T06:34:09.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.635+0000 7f06d778b700 1 --2- 172.21.15.16:0/3792423528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d0106fb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:09.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.635+0000 7f06d778b700 1 -- 172.21.15.16:0/3792423528 >> 172.21.15.16:0/3792423528 conn(0x7f06d0102790 msgr2=0x7f06d0104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:09.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.635+0000 7f06d778b700 1 -- 172.21.15.16:0/3792423528 shutdown_connections 2024-03-27T06:34:09.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.635+0000 7f06d778b700 1 -- 172.21.15.16:0/3792423528 wait complete. 2024-03-27T06:34:09.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06d778b700 1 Processor -- start 2024-03-27T06:34:09.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06d778b700 1 -- start start 2024-03-27T06:34:09.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06d778b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:09.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06d778b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06d0108620 con 0x7f06d0106b90 2024-03-27T06:34:09.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06d5527700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:09.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06d5527700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40954/0 (socket says 172.21.15.16:40954) 2024-03-27T06:34:09.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06d5527700 1 -- 172.21.15.16:0/3189501479 learned_addr learned my addr 172.21.15.16:0/3189501479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:09.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06d5527700 1 -- 172.21.15.16:0/3189501479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d019e990 con 0x7f06d0106b90 2024-03-27T06:34:09.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06d5527700 1 --2- 172.21.15.16:0/3189501479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d019e450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f06c4011870 tx=0x7f06c4011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:09.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06c27fc700 1 -- 172.21.15.16:0/3189501479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06c4003af0 con 0x7f06d0106b90 2024-03-27T06:34:09.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06d778b700 1 -- 172.21.15.16:0/3189501479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d019ec20 con 0x7f06d0106b90 2024-03-27T06:34:09.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06c27fc700 1 -- 172.21.15.16:0/3189501479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f06c4003c90 con 0x7f06d0106b90 2024-03-27T06:34:09.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.636+0000 7f06c27fc700 1 -- 172.21.15.16:0/3189501479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06c4019960 con 0x7f06d0106b90 2024-03-27T06:34:09.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.637+0000 7f06d778b700 1 -- 172.21.15.16:0/3189501479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d019f080 con 0x7f06d0106b90 2024-03-27T06:34:09.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.638+0000 7f06c27fc700 1 -- 172.21.15.16:0/3189501479 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f06c4019c60 con 0x7f06d0106b90 2024-03-27T06:34:09.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.638+0000 7f06d778b700 1 -- 172.21.15.16:0/3189501479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d0063c70 con 0x7f06d0106b90 2024-03-27T06:34:09.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.638+0000 7f06c27fc700 1 --2- 172.21.15.16:0/3189501479 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f06bc039380 0x7f06bc03b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:09.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.638+0000 7f06c27fc700 1 -- 172.21.15.16:0/3189501479 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f06c404d9b0 con 0x7f06d0106b90 2024-03-27T06:34:09.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.638+0000 7f06d4d26700 1 --2- 172.21.15.16:0/3189501479 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f06bc039380 0x7f06bc03b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:09.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.638+0000 7f06d4d26700 1 --2- 172.21.15.16:0/3189501479 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f06bc039380 0x7f06bc03b840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f06cc006fd0 tx=0x7f06cc006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:09.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.643+0000 7f06c27fc700 1 -- 172.21.15.16:0/3189501479 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f06c400fda0 con 0x7f06d0106b90 2024-03-27T06:34:09.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.851+0000 7f06d778b700 1 -- 172.21.15.16:0/3189501479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f06d0063250 con 0x7f06d0106b90 2024-03-27T06:34:09.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.853+0000 7f06c27fc700 1 -- 172.21.15.16:0/3189501479 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f06c401ad10 con 0x7f06d0106b90 2024-03-27T06:34:09.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.856+0000 7f06d778b700 1 -- 172.21.15.16:0/3189501479 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f06bc039380 msgr2=0x7f06bc03b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:09.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.856+0000 7f06d778b700 1 --2- 172.21.15.16:0/3189501479 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f06bc039380 0x7f06bc03b840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f06cc006fd0 tx=0x7f06cc006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:09.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.856+0000 7f06d778b700 1 -- 172.21.15.16:0/3189501479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 msgr2=0x7f06d019e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:09.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.856+0000 7f06d778b700 1 --2- 172.21.15.16:0/3189501479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d019e450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f06c4011870 tx=0x7f06c4011d70 comp rx=0 tx=0).stop 2024-03-27T06:34:09.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.856+0000 7f06d778b700 1 -- 172.21.15.16:0/3189501479 shutdown_connections 2024-03-27T06:34:09.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.856+0000 7f06d778b700 1 --2- 172.21.15.16:0/3189501479 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f06bc039380 0x7f06bc03b840 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:09.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.856+0000 7f06d778b700 1 --2- 172.21.15.16:0/3189501479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d0106b90 0x7f06d019e450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:09.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.856+0000 7f06d778b700 1 -- 172.21.15.16:0/3189501479 >> 172.21.15.16:0/3189501479 conn(0x7f06d0102790 msgr2=0x7f06d0194fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:09.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.856+0000 7f06d778b700 1 -- 172.21.15.16:0/3189501479 shutdown_connections 2024-03-27T06:34:09.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:08.856+0000 7f06d778b700 1 -- 172.21.15.16:0/3189501479 wait complete. 2024-03-27T06:34:09.784 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:09 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:34:09.784 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:09 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3189501479' entity='client.admin' 2024-03-27T06:34:10.678 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.895+0000 7f23421a3700 1 Processor -- start 2024-03-27T06:34:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.895+0000 7f23421a3700 1 -- start start 2024-03-27T06:34:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.895+0000 7f23421a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c108c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.895+0000 7f23421a3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f233c109220 con 0x7f233c108830 2024-03-27T06:34:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.895+0000 7f233b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c108c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.895+0000 7f233b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c108c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40970/0 (socket says 172.21.15.16:40970) 2024-03-27T06:34:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.895+0000 7f233b7fe700 1 -- 172.21.15.16:0/3203929085 learned_addr learned my addr 172.21.15.16:0/3203929085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.896+0000 7f233b7fe700 1 -- 172.21.15.16:0/3203929085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233c107400 con 0x7f233c108830 2024-03-27T06:34:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.896+0000 7f233b7fe700 1 --2- 172.21.15.16:0/3203929085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c108c50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f232c0098a0 tx=0x7f232c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77baff0ecc942a65 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.896+0000 7f233a7fc700 1 -- 172.21.15.16:0/3203929085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f232c004160 con 0x7f233c108830 2024-03-27T06:34:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.896+0000 7f233a7fc700 1 -- 172.21.15.16:0/3203929085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f232c004300 con 0x7f233c108830 2024-03-27T06:34:10.679 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.896+0000 7f233a7fc700 1 -- 172.21.15.16:0/3203929085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f232c011bb0 con 0x7f233c108830 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.897+0000 7f23421a3700 1 -- 172.21.15.16:0/3203929085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 msgr2=0x7f233c108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.897+0000 7f23421a3700 1 --2- 172.21.15.16:0/3203929085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c108c50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f232c0098a0 tx=0x7f232c009bb0 comp rx=0 tx=0).stop 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.897+0000 7f23421a3700 1 -- 172.21.15.16:0/3203929085 shutdown_connections 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.897+0000 7f23421a3700 1 --2- 172.21.15.16:0/3203929085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c108c50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.897+0000 7f23421a3700 1 -- 172.21.15.16:0/3203929085 >> 172.21.15.16:0/3203929085 conn(0x7f233c101f30 msgr2=0x7f233c104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.897+0000 7f23421a3700 1 -- 172.21.15.16:0/3203929085 shutdown_connections 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.897+0000 7f23421a3700 1 -- 172.21.15.16:0/3203929085 wait complete. 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.898+0000 7f23421a3700 1 Processor -- start 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.898+0000 7f23421a3700 1 -- start start 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.898+0000 7f23421a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c19c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.898+0000 7f23421a3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f233c10b7c0 con 0x7f233c108830 2024-03-27T06:34:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.898+0000 7f233b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c19c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.898+0000 7f233b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c19c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40974/0 (socket says 172.21.15.16:40974) 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.898+0000 7f233b7fe700 1 -- 172.21.15.16:0/467745995 learned_addr learned my addr 172.21.15.16:0/467745995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.898+0000 7f233b7fe700 1 -- 172.21.15.16:0/467745995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233c19c8a0 con 0x7f233c108830 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.898+0000 7f233b7fe700 1 --2- 172.21.15.16:0/467745995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c19c360 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f232c000c00 tx=0x7f232c011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.899+0000 7f2338ff9700 1 -- 172.21.15.16:0/467745995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f232c011e60 con 0x7f233c108830 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.899+0000 7f2338ff9700 1 -- 172.21.15.16:0/467745995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f232c020070 con 0x7f233c108830 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.899+0000 7f2338ff9700 1 -- 172.21.15.16:0/467745995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f232c019550 con 0x7f233c108830 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.899+0000 7f23421a3700 1 -- 172.21.15.16:0/467745995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233c19cb30 con 0x7f233c108830 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.899+0000 7f23421a3700 1 -- 172.21.15.16:0/467745995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f233c19cf90 con 0x7f233c108830 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.900+0000 7f2338ff9700 1 -- 172.21.15.16:0/467745995 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f232c0197c0 con 0x7f233c108830 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.900+0000 7f23421a3700 1 -- 172.21.15.16:0/467745995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233c063c70 con 0x7f233c108830 2024-03-27T06:34:10.681 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.900+0000 7f2338ff9700 1 --2- 172.21.15.16:0/467745995 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f2324039420 0x7f232403b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.900+0000 7f2338ff9700 1 -- 172.21.15.16:0/467745995 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f232c04d730 con 0x7f233c108830 2024-03-27T06:34:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.900+0000 7f233affd700 1 --2- 172.21.15.16:0/467745995 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f2324039420 0x7f232403b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.900+0000 7f233affd700 1 --2- 172.21.15.16:0/467745995 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f2324039420 0x7f232403b8e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2330006fd0 tx=0x7f2330006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:09.905+0000 7f2338ff9700 1 -- 172.21.15.16:0/467745995 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f232c023530 con 0x7f233c108830 2024-03-27T06:34:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.114+0000 7f23421a3700 1 -- 172.21.15.16:0/467745995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f233c063250 con 0x7f233c108830 2024-03-27T06:34:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.117+0000 7f2338ff9700 1 -- 172.21.15.16:0/467745995 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f232c023790 con 0x7f233c108830 2024-03-27T06:34:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.119+0000 7f23421a3700 1 -- 172.21.15.16:0/467745995 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f2324039420 msgr2=0x7f232403b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.119+0000 7f23421a3700 1 --2- 172.21.15.16:0/467745995 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f2324039420 0x7f232403b8e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2330006fd0 tx=0x7f2330006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.119+0000 7f23421a3700 1 -- 172.21.15.16:0/467745995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 msgr2=0x7f233c19c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.119+0000 7f23421a3700 1 --2- 172.21.15.16:0/467745995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c19c360 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f232c000c00 tx=0x7f232c011c30 comp rx=0 tx=0).stop 2024-03-27T06:34:10.682 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.120+0000 7f23421a3700 1 -- 172.21.15.16:0/467745995 shutdown_connections 2024-03-27T06:34:10.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.120+0000 7f23421a3700 1 --2- 172.21.15.16:0/467745995 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f2324039420 0x7f232403b8e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:10.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.120+0000 7f23421a3700 1 --2- 172.21.15.16:0/467745995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c108830 0x7f233c19c360 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:10.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.120+0000 7f23421a3700 1 -- 172.21.15.16:0/467745995 >> 172.21.15.16:0/467745995 conn(0x7f233c101f30 msgr2=0x7f233c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:10.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.120+0000 7f23421a3700 1 -- 172.21.15.16:0/467745995 shutdown_connections 2024-03-27T06:34:10.683 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:10.120+0000 7f23421a3700 1 -- 172.21.15.16:0/467745995 wait complete. 2024-03-27T06:34:10.683 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-03-27T06:34:11.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:10 smithi016 ceph-mon[31908]: from='mgr.14116 172.21.15.16:0/1145292089' entity='mgr.x' 2024-03-27T06:34:11.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:10 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/467745995' entity='client.admin' 2024-03-27T06:34:12.708 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:12 smithi016 ceph-mon[31908]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:34:12.708 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:12 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1953519007' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-27T06:34:12.708 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:12 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: ignoring --setuser ceph since I am not root 2024-03-27T06:34:12.708 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:12 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: ignoring --setgroup ceph since I am not root 2024-03-27T06:34:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.170+0000 7f49d1c85700 1 Processor -- start 2024-03-27T06:34:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.170+0000 7f49d1c85700 1 -- start start 2024-03-27T06:34:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.170+0000 7f49d1c85700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.170+0000 7f49d1c85700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc07c820 con 0x7f49cc07dd70 2024-03-27T06:34:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.170+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.170+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40990/0 (socket says 172.21.15.16:40990) 2024-03-27T06:34:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.170+0000 7f49cb7fe700 1 -- 172.21.15.16:0/2023734949 learned_addr learned my addr 172.21.15.16:0/2023734949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.171+0000 7f49cb7fe700 1 -- 172.21.15.16:0/2023734949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc07c9a0 con 0x7f49cc07dd70 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.171+0000 7f49cb7fe700 1 --2- 172.21.15.16:0/2023734949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc07c2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009e80 tx=0x7f49bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d78418c2e81508 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.171+0000 7f49ca7fc700 1 -- 172.21.15.16:0/2023734949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49bc017070 con 0x7f49cc07dd70 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.171+0000 7f49ca7fc700 1 -- 172.21.15.16:0/2023734949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f49bc00b8c0 con 0x7f49cc07dd70 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.171+0000 7f49ca7fc700 1 -- 172.21.15.16:0/2023734949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49bc004580 con 0x7f49cc07dd70 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.171+0000 7f49d1c85700 1 -- 172.21.15.16:0/2023734949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 msgr2=0x7f49cc07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.171+0000 7f49d1c85700 1 --2- 172.21.15.16:0/2023734949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc07c2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009e80 tx=0x7f49bc00b330 comp rx=0 tx=0).stop 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.172+0000 7f49d1c85700 1 -- 172.21.15.16:0/2023734949 shutdown_connections 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.172+0000 7f49d1c85700 1 --2- 172.21.15.16:0/2023734949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc07c2e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.172+0000 7f49d1c85700 1 -- 172.21.15.16:0/2023734949 >> 172.21.15.16:0/2023734949 conn(0x7f49cc103990 msgr2=0x7f49cc105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.172+0000 7f49d1c85700 1 -- 172.21.15.16:0/2023734949 shutdown_connections 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.172+0000 7f49d1c85700 1 -- 172.21.15.16:0/2023734949 wait complete. 2024-03-27T06:34:13.051 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.172+0000 7f49d1c85700 1 Processor -- start 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.172+0000 7f49d1c85700 1 -- start start 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49d1c85700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49d1c85700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc10e8a0 con 0x7f49cc07dd70 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc19e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40994/0 (socket says 172.21.15.16:40994) 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49cb7fe700 1 -- 172.21.15.16:0/1953519007 learned_addr learned my addr 172.21.15.16:0/1953519007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49cb7fe700 1 -- 172.21.15.16:0/1953519007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc19e9b0 con 0x7f49cc07dd70 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49cb7fe700 1 --2- 172.21.15.16:0/1953519007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc19e470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0072a0 tx=0x7f49bc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49c8ff9700 1 -- 172.21.15.16:0/1953519007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49bc021020 con 0x7f49cc07dd70 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49d1c85700 1 -- 172.21.15.16:0/1953519007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc19ec40 con 0x7f49cc07dd70 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49c8ff9700 1 -- 172.21.15.16:0/1953519007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f49bc00bde0 con 0x7f49cc07dd70 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49c8ff9700 1 -- 172.21.15.16:0/1953519007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49bc017040 con 0x7f49cc07dd70 2024-03-27T06:34:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.173+0000 7f49d1c85700 1 -- 172.21.15.16:0/1953519007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc1a1930 con 0x7f49cc07dd70 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.174+0000 7f49c8ff9700 1 -- 172.21.15.16:0/1953519007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f49bc024020 con 0x7f49cc07dd70 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.174+0000 7f49d1c85700 1 -- 172.21.15.16:0/1953519007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49cc063c70 con 0x7f49cc07dd70 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.174+0000 7f49c8ff9700 1 --2- 172.21.15.16:0/1953519007 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f49b8039380 0x7f49b803b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.174+0000 7f49c8ff9700 1 -- 172.21.15.16:0/1953519007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f49bc00f050 con 0x7f49cc07dd70 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.174+0000 7f49caffd700 1 --2- 172.21.15.16:0/1953519007 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f49b8039380 0x7f49b803b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.175+0000 7f49caffd700 1 --2- 172.21.15.16:0/1953519007 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f49b8039380 0x7f49b803b840 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f49b4006fd0 tx=0x7f49b4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.179+0000 7f49c8ff9700 1 -- 172.21.15.16:0/1953519007 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f49bc02b020 con 0x7f49cc07dd70 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:11.461+0000 7f49d1c85700 1 -- 172.21.15.16:0/1953519007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f49cc1a1b00 con 0x7f49cc07dd70 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.407+0000 7f49c8ff9700 1 -- 172.21.15.16:0/1953519007 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f49cc1a1b00 con 0x7f49cc07dd70 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.410+0000 7f49c8ff9700 1 -- 172.21.15.16:0/1953519007 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 9) ==== 46479+0+0 (secure 0 0 0) 0x7f49bc01da70 con 0x7f49cc07dd70 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.411+0000 7f49d1c85700 1 -- 172.21.15.16:0/1953519007 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f49b8039380 msgr2=0x7f49b803b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.411+0000 7f49d1c85700 1 --2- 172.21.15.16:0/1953519007 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f49b8039380 0x7f49b803b840 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f49b4006fd0 tx=0x7f49b4006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.411+0000 7f49d1c85700 1 -- 172.21.15.16:0/1953519007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 msgr2=0x7f49cc19e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:13.054 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.411+0000 7f49d1c85700 1 --2- 172.21.15.16:0/1953519007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc19e470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0072a0 tx=0x7f49bc003d00 comp rx=0 tx=0).stop 2024-03-27T06:34:13.054 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.411+0000 7f49d1c85700 1 -- 172.21.15.16:0/1953519007 shutdown_connections 2024-03-27T06:34:13.054 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.411+0000 7f49d1c85700 1 --2- 172.21.15.16:0/1953519007 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f49b8039380 0x7f49b803b840 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:13.054 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.411+0000 7f49d1c85700 1 --2- 172.21.15.16:0/1953519007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc07dd70 0x7f49cc19e470 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:13.054 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.411+0000 7f49d1c85700 1 -- 172.21.15.16:0/1953519007 >> 172.21.15.16:0/1953519007 conn(0x7f49cc103990 msgr2=0x7f49cc104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:13.054 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.411+0000 7f49d1c85700 1 -- 172.21.15.16:0/1953519007 shutdown_connections 2024-03-27T06:34:13.054 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:12.411+0000 7f49d1c85700 1 -- 172.21.15.16:0/1953519007 wait complete. 2024-03-27T06:34:13.291 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:12 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:12.992+0000 7f9cd1abb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T06:34:13.291 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:13 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:13.091+0000 7f9cd1abb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T06:34:13.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:13 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1953519007' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-27T06:34:13.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:13 smithi016 ceph-mon[31908]: mgrmap e9: x(active, since 21s) 2024-03-27T06:34:13.998 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:13 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:13.761+0000 7f9cd1abb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T06:34:13.998 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:13 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:13.860+0000 7f9cd1abb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T06:34:13.999 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:14.004+0000 7f9cd1abb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T06:34:14.357 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:14.102+0000 7f9cd1abb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T06:34:14.357 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:14.188+0000 7f9cd1abb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T06:34:14.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-03-27T06:34:14.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-27T06:34:14.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-27T06:34:14.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.502+0000 7f795a5be700 1 Processor -- start 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.502+0000 7f795a5be700 1 -- start start 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.502+0000 7f795a5be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f7954108710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.502+0000 7f795a5be700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7954108c50 con 0x7f79541082f0 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.502+0000 7f7953fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f7954108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.503+0000 7f7953fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f7954108710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41026/0 (socket says 172.21.15.16:41026) 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.503+0000 7f7953fff700 1 -- 172.21.15.16:0/1818418334 learned_addr learned my addr 172.21.15.16:0/1818418334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.503+0000 7f7953fff700 1 -- 172.21.15.16:0/1818418334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7954108dd0 con 0x7f79541082f0 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.503+0000 7f7953fff700 1 --2- 172.21.15.16:0/1818418334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f7954108710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f79440098a0 tx=0x7f7944009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2100c174ed3876a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.503+0000 7f7952ffd700 1 -- 172.21.15.16:0/1818418334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7944004160 con 0x7f79541082f0 2024-03-27T06:34:14.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.503+0000 7f7952ffd700 1 -- 172.21.15.16:0/1818418334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7944004300 con 0x7f79541082f0 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.503+0000 7f7952ffd700 1 -- 172.21.15.16:0/1818418334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7944011bb0 con 0x7f79541082f0 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.504+0000 7f795a5be700 1 -- 172.21.15.16:0/1818418334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 msgr2=0x7f7954108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.504+0000 7f795a5be700 1 --2- 172.21.15.16:0/1818418334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f7954108710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f79440098a0 tx=0x7f7944009bb0 comp rx=0 tx=0).stop 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.504+0000 7f795a5be700 1 -- 172.21.15.16:0/1818418334 shutdown_connections 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.504+0000 7f795a5be700 1 --2- 172.21.15.16:0/1818418334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f7954108710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.504+0000 7f795a5be700 1 -- 172.21.15.16:0/1818418334 >> 172.21.15.16:0/1818418334 conn(0x7f7954103930 msgr2=0x7f7954105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.504+0000 7f795a5be700 1 -- 172.21.15.16:0/1818418334 shutdown_connections 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.504+0000 7f795a5be700 1 -- 172.21.15.16:0/1818418334 wait complete. 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.504+0000 7f795a5be700 1 Processor -- start 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.504+0000 7f795a5be700 1 -- start start 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.505+0000 7f795a5be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f795419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:14.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.505+0000 7f795a5be700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f795410c760 con 0x7f79541082f0 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.505+0000 7f7953fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f795419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.505+0000 7f7953fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f795419c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41040/0 (socket says 172.21.15.16:41040) 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.505+0000 7f7953fff700 1 -- 172.21.15.16:0/3894130268 learned_addr learned my addr 172.21.15.16:0/3894130268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.505+0000 7f7953fff700 1 -- 172.21.15.16:0/3894130268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795419c730 con 0x7f79541082f0 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.505+0000 7f7953fff700 1 --2- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f795419c1f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7944011870 tx=0x7f7944011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.505+0000 7f79517fa700 1 -- 172.21.15.16:0/3894130268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7944003af0 con 0x7f79541082f0 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.506+0000 7f795a5be700 1 -- 172.21.15.16:0/3894130268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795419c9c0 con 0x7f79541082f0 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.506+0000 7f79517fa700 1 -- 172.21.15.16:0/3894130268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7944003c90 con 0x7f79541082f0 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.506+0000 7f79517fa700 1 -- 172.21.15.16:0/3894130268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7944019960 con 0x7f79541082f0 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.506+0000 7f795a5be700 1 -- 172.21.15.16:0/3894130268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795419ce20 con 0x7f79541082f0 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.507+0000 7f79517fa700 1 -- 172.21.15.16:0/3894130268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 46479+0+0 (secure 0 0 0) 0x7f7944019c60 con 0x7f79541082f0 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.507+0000 7f795a5be700 1 -- 172.21.15.16:0/3894130268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7954063c70 con 0x7f79541082f0 2024-03-27T06:34:14.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.507+0000 7f79517fa700 1 --2- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f793c0393d0 0x7f793c03b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.507+0000 7f79517fa700 1 -- 172.21.15.16:0/3894130268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f794404da20 con 0x7f79541082f0 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.507+0000 7f79537fe700 1 -- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f793c0393d0 msgr2=0x7f793c03b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1685933023 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.507+0000 7f79537fe700 1 --2- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f793c0393d0 0x7f793c03b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.512+0000 7f79517fa700 1 -- 172.21.15.16:0/3894130268 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f794401ad10 con 0x7f79541082f0 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.708+0000 7f79537fe700 1 -- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f793c0393d0 msgr2=0x7f793c03b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1685933023 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.708+0000 7f79537fe700 1 --2- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f793c0393d0 0x7f793c03b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.781+0000 7f795a5be700 1 -- 172.21.15.16:0/3894130268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f795419f7d0 con 0x7f79541082f0 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.781+0000 7f79517fa700 1 -- 172.21.15.16:0/3894130268 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f794400fda0 con 0x7f79541082f0 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.784+0000 7f795a5be700 1 -- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f793c0393d0 msgr2=0x7f793c03b890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.784+0000 7f795a5be700 1 --2- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f793c0393d0 0x7f793c03b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.784+0000 7f795a5be700 1 -- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 msgr2=0x7f795419c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.784+0000 7f795a5be700 1 --2- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f795419c1f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7944011870 tx=0x7f7944011d70 comp rx=0 tx=0).stop 2024-03-27T06:34:14.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.784+0000 7f795a5be700 1 -- 172.21.15.16:0/3894130268 shutdown_connections 2024-03-27T06:34:14.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.784+0000 7f795a5be700 1 --2- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f793c0393d0 0x7f793c03b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:14.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.784+0000 7f795a5be700 1 --2- 172.21.15.16:0/3894130268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541082f0 0x7f795419c1f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:14.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.784+0000 7f795a5be700 1 -- 172.21.15.16:0/3894130268 >> 172.21.15.16:0/3894130268 conn(0x7f7954103930 msgr2=0x7f7954104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:14.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.784+0000 7f795a5be700 1 -- 172.21.15.16:0/3894130268 shutdown_connections 2024-03-27T06:34:14.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:13.784+0000 7f795a5be700 1 -- 172.21.15.16:0/3894130268 wait complete. 2024-03-27T06:34:14.372 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-03-27T06:34:14.372 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-03-27T06:34:14.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:14 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3894130268' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-27T06:34:14.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:14.463+0000 7f9cd1abb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T06:34:14.610 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:14.577+0000 7f9cd1abb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T06:34:14.942 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:14.668+0000 7f9cd1abb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T06:34:14.943 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:14.767+0000 7f9cd1abb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T06:34:14.943 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:14.948+0000 7f9cd1abb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T06:34:16.226 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:15 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:15.935+0000 7f9cd1abb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T06:34:16.226 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:16 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:16.070+0000 7f9cd1abb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T06:34:16.500 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:16 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:16.412+0000 7f9cd1abb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T06:34:16.500 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:16 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:16.506+0000 7f9cd1abb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T06:34:16.806 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:16 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:16.715+0000 7f9cd1abb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:34:17.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:16 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:16.913+0000 7f9cd1abb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T06:34:17.471 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:17 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:17.213+0000 7f9cd1abb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:34:17.471 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:17 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:17.477+0000 7f9cd1abb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T06:34:17.828 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:17 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:17.724+0000 7f9cd1abb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T06:34:18.860 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:18 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:18.557+0000 7f9cd1abb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T06:34:19.459 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:19.140+0000 7f9cd1abb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T06:34:19.459 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:19.240+0000 7f9cd1abb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T06:34:19.459 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:34:19.464+0000 7f9cd1abb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T06:34:19.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-mon[31908]: Active manager daemon x restarted 2024-03-27T06:34:19.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-mon[31908]: Activating manager daemon x 2024-03-27T06:34:19.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-mon[31908]: osdmap e3: 0 total, 0 up, 0 in 2024-03-27T06:34:19.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-mon[31908]: mgrmap e10: x(active, starting, since 0.00347857s) 2024-03-27T06:34:19.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:34:19.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-27T06:34:19.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-27T06:34:19.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-27T06:34:19.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-27T06:34:19.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-mon[31908]: Manager daemon x is now available 2024-03-27T06:34:19.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:34:20.698 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-27T06:34:20.698 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-27T06:34:20.698 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:20 smithi016 ceph-mon[31908]: [27/Mar/2024:06:34:20] ENGINE Bus STARTING 2024-03-27T06:34:20.698 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:20 smithi016 ceph-mon[31908]: [27/Mar/2024:06:34:20] ENGINE Serving on http://172.21.15.16:8765 2024-03-27T06:34:20.699 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:20 smithi016 ceph-mon[31908]: [27/Mar/2024:06:34:20] ENGINE Serving on https://172.21.15.16:7150 2024-03-27T06:34:20.699 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:20 smithi016 ceph-mon[31908]: [27/Mar/2024:06:34:20] ENGINE Bus STARTED 2024-03-27T06:34:20.699 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:20 smithi016 ceph-mon[31908]: mgrmap e11: x(active, since 1.00632s) 2024-03-27T06:34:21.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-03-27T06:34:21.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-27T06:34:21.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-27T06:34:21.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-03-27T06:34:21.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.824+0000 7f9a8377a700 1 Processor -- start 2024-03-27T06:34:21.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.825+0000 7f9a8377a700 1 -- start start 2024-03-27T06:34:21.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.825+0000 7f9a8377a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:21.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.825+0000 7f9a8377a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c107ba0 con 0x7f9a7c1071b0 2024-03-27T06:34:21.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.825+0000 7f9a81516700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:21.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.825+0000 7f9a81516700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41054/0 (socket says 172.21.15.16:41054) 2024-03-27T06:34:21.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.825+0000 7f9a81516700 1 -- 172.21.15.16:0/2525955919 learned_addr learned my addr 172.21.15.16:0/2525955919 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:21.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.825+0000 7f9a81516700 1 -- 172.21.15.16:0/2525955919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c1083b0 con 0x7f9a7c1071b0 2024-03-27T06:34:21.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.826+0000 7f9a81516700 1 --2- 172.21.15.16:0/2525955919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1075d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9a700098a0 tx=0x7f9a70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a200fe27d65a6148 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:21.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.826+0000 7f9a6ffff700 1 -- 172.21.15.16:0/2525955919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a70004160 con 0x7f9a7c1071b0 2024-03-27T06:34:21.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.826+0000 7f9a6ffff700 1 -- 172.21.15.16:0/2525955919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a70004300 con 0x7f9a7c1071b0 2024-03-27T06:34:21.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.826+0000 7f9a6ffff700 1 -- 172.21.15.16:0/2525955919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a70011bb0 con 0x7f9a7c1071b0 2024-03-27T06:34:21.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.826+0000 7f9a8377a700 1 -- 172.21.15.16:0/2525955919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 msgr2=0x7f9a7c1075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:21.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.826+0000 7f9a8377a700 1 --2- 172.21.15.16:0/2525955919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1075d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9a700098a0 tx=0x7f9a70009bb0 comp rx=0 tx=0).stop 2024-03-27T06:34:21.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.826+0000 7f9a8377a700 1 -- 172.21.15.16:0/2525955919 shutdown_connections 2024-03-27T06:34:21.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.826+0000 7f9a8377a700 1 --2- 172.21.15.16:0/2525955919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1075d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:21.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.826+0000 7f9a8377a700 1 -- 172.21.15.16:0/2525955919 >> 172.21.15.16:0/2525955919 conn(0x7f9a7c102790 msgr2=0x7f9a7c104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:21.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.827+0000 7f9a8377a700 1 -- 172.21.15.16:0/2525955919 shutdown_connections 2024-03-27T06:34:21.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.827+0000 7f9a8377a700 1 -- 172.21.15.16:0/2525955919 wait complete. 2024-03-27T06:34:21.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.827+0000 7f9a8377a700 1 Processor -- start 2024-03-27T06:34:21.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.827+0000 7f9a8377a700 1 -- start start 2024-03-27T06:34:21.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.827+0000 7f9a8377a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1923f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:21.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.827+0000 7f9a8377a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c10b760 con 0x7f9a7c1071b0 2024-03-27T06:34:21.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.827+0000 7f9a81516700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:21.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.827+0000 7f9a81516700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1923f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41070/0 (socket says 172.21.15.16:41070) 2024-03-27T06:34:21.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.827+0000 7f9a81516700 1 -- 172.21.15.16:0/1106460053 learned_addr learned my addr 172.21.15.16:0/1106460053 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:21.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.828+0000 7f9a81516700 1 -- 172.21.15.16:0/1106460053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c192930 con 0x7f9a7c1071b0 2024-03-27T06:34:21.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.828+0000 7f9a81516700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1923f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a70011bf0 tx=0x7f9a70011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:21.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.828+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a70003930 con 0x7f9a7c1071b0 2024-03-27T06:34:21.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.828+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a70003ad0 con 0x7f9a7c1071b0 2024-03-27T06:34:21.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.828+0000 7f9a8377a700 1 -- 172.21.15.16:0/1106460053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7c190b20 con 0x7f9a7c1071b0 2024-03-27T06:34:21.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.828+0000 7f9a8377a700 1 -- 172.21.15.16:0/1106460053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7c190fe0 con 0x7f9a7c1071b0 2024-03-27T06:34:21.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.828+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a70019760 con 0x7f9a7c1071b0 2024-03-27T06:34:21.433 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.829+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 46479+0+0 (secure 0 0 0) 0x7f9a70019a60 con 0x7f9a7c1071b0 2024-03-27T06:34:21.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.829+0000 7f9a6e7fc700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 0x7f9a6803b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:21.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.829+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 --> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9a6803bfa0 con 0x7f9a680393d0 2024-03-27T06:34:21.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.829+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9a7004d9d0 con 0x7f9a7c1071b0 2024-03-27T06:34:21.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.829+0000 7f9a80d15700 1 -- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 msgr2=0x7f9a6803b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1685933023 2024-03-27T06:34:21.434 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:14.829+0000 7f9a80d15700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 0x7f9a6803b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:34:21.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:15.030+0000 7f9a80d15700 1 -- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 msgr2=0x7f9a6803b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1685933023 2024-03-27T06:34:21.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:15.030+0000 7f9a80d15700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 0x7f9a6803b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-27T06:34:21.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:15.431+0000 7f9a80d15700 1 -- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 msgr2=0x7f9a6803b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1685933023 2024-03-27T06:34:21.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:15.431+0000 7f9a80d15700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 0x7f9a6803b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-27T06:34:21.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:16.232+0000 7f9a80d15700 1 -- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 msgr2=0x7f9a6803b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1685933023 2024-03-27T06:34:21.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:16.232+0000 7f9a80d15700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 0x7f9a6803b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-27T06:34:21.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:17.833+0000 7f9a80d15700 1 -- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 msgr2=0x7f9a6803b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1685933023 2024-03-27T06:34:21.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:17.833+0000 7f9a80d15700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 0x7f9a6803b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-27T06:34:21.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:19.479+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 10) ==== 46246+0+0 (secure 0 0 0) 0x7f9a7002d960 con 0x7f9a7c1071b0 2024-03-27T06:34:21.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:19.479+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 msgr2=0x7f9a6803b890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:34:21.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:19.479+0000 7f9a6e7fc700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 0x7f9a6803b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:21.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.480+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 11) ==== 46373+0+0 (secure 0 0 0) 0x7f9a7004f910 con 0x7f9a7c1071b0 2024-03-27T06:34:21.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.481+0000 7f9a6e7fc700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9a6803ce50 0x7f9a6803f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:21.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.481+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9a70008ea0 con 0x7f9a6803ce50 2024-03-27T06:34:21.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.481+0000 7f9a80d15700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9a6803ce50 0x7f9a6803f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:21.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.482+0000 7f9a80d15700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9a6803ce50 0x7f9a6803f240 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9a78003a10 tx=0x7f9a780092b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:21.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.482+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f9a70008ea0 con 0x7f9a6803ce50 2024-03-27T06:34:21.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.489+0000 7f9a8377a700 1 -- 172.21.15.16:0/1106460053 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9a7c10b8e0 con 0x7f9a6803ce50 2024-03-27T06:34:21.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.489+0000 7f9a6e7fc700 1 -- 172.21.15.16:0/1106460053 <== mgr.14148 v2:172.21.15.16:6800/4198806087 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f9a7c10b8e0 con 0x7f9a6803ce50 2024-03-27T06:34:21.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.490+0000 7f9a8377a700 1 -- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9a6803ce50 msgr2=0x7f9a6803f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:21.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.490+0000 7f9a8377a700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9a6803ce50 0x7f9a6803f240 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9a78003a10 tx=0x7f9a780092b0 comp rx=0 tx=0).stop 2024-03-27T06:34:21.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.490+0000 7f9a8377a700 1 -- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 msgr2=0x7f9a7c1923f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:21.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.490+0000 7f9a8377a700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1923f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a70011bf0 tx=0x7f9a70011cd0 comp rx=0 tx=0).stop 2024-03-27T06:34:21.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.490+0000 7f9a8377a700 1 -- 172.21.15.16:0/1106460053 shutdown_connections 2024-03-27T06:34:21.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.490+0000 7f9a8377a700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9a6803ce50 0x7f9a6803f240 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:21.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.490+0000 7f9a8377a700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a7c1071b0 0x7f9a7c1923f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:21.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.490+0000 7f9a8377a700 1 --2- 172.21.15.16:0/1106460053 >> [v2:172.21.15.16:6800/1685933023,v1:172.21.15.16:6801/1685933023] conn(0x7f9a680393d0 0x7f9a6803b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:21.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.490+0000 7f9a8377a700 1 -- 172.21.15.16:0/1106460053 >> 172.21.15.16:0/1106460053 conn(0x7f9a7c102790 msgr2=0x7f9a7c1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:21.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.490+0000 7f9a8377a700 1 -- 172.21.15.16:0/1106460053 shutdown_connections 2024-03-27T06:34:21.440 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:20.490+0000 7f9a8377a700 1 -- 172.21.15.16:0/1106460053 wait complete. 2024-03-27T06:34:21.440 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-03-27T06:34:21.440 INFO:teuthology.orchestra.run.smithi016.stdout:Generating a dashboard self-signed certificate... 2024-03-27T06:34:21.680 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:21 smithi016 ceph-mon[31908]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-27T06:34:21.681 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:21 smithi016 ceph-mon[31908]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-27T06:34:22.799 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:22 smithi016 ceph-mon[31908]: mgrmap e12: x(active, since 2s) 2024-03-27T06:34:22.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:22.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:22 smithi016 ceph-mon[31908]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:34:22.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:22.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-27T06:34:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.881+0000 7f386c9a5700 1 Processor -- start 2024-03-27T06:34:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.881+0000 7f386c9a5700 1 -- start start 2024-03-27T06:34:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.882+0000 7f386c9a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f386810b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.882+0000 7f386c9a5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f386807be70 con 0x7f3868108d80 2024-03-27T06:34:22.810 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.882+0000 7f386659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f386810b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:22.810 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.882+0000 7f386659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f386810b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39504/0 (socket says 172.21.15.16:39504) 2024-03-27T06:34:22.810 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.882+0000 7f386659c700 1 -- 172.21.15.16:0/971115767 learned_addr learned my addr 172.21.15.16:0/971115767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:22.810 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.882+0000 7f386659c700 1 -- 172.21.15.16:0/971115767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386810b6b0 con 0x7f3868108d80 2024-03-27T06:34:22.810 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.883+0000 7f386659c700 1 --2- 172.21.15.16:0/971115767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f386810b170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f385c0098a0 tx=0x7f385c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233b037be1d7dba2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:22.810 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.883+0000 7f386559a700 1 -- 172.21.15.16:0/971115767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f385c004160 con 0x7f3868108d80 2024-03-27T06:34:22.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.883+0000 7f386559a700 1 -- 172.21.15.16:0/971115767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f385c004300 con 0x7f3868108d80 2024-03-27T06:34:22.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.883+0000 7f386559a700 1 -- 172.21.15.16:0/971115767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f385c011bb0 con 0x7f3868108d80 2024-03-27T06:34:22.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.883+0000 7f386c9a5700 1 -- 172.21.15.16:0/971115767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 msgr2=0x7f386810b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:22.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.883+0000 7f386c9a5700 1 --2- 172.21.15.16:0/971115767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f386810b170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f385c0098a0 tx=0x7f385c009bb0 comp rx=0 tx=0).stop 2024-03-27T06:34:22.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.883+0000 7f386c9a5700 1 -- 172.21.15.16:0/971115767 shutdown_connections 2024-03-27T06:34:22.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.883+0000 7f386c9a5700 1 --2- 172.21.15.16:0/971115767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f386810b170 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:22.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.883+0000 7f386c9a5700 1 -- 172.21.15.16:0/971115767 >> 172.21.15.16:0/971115767 conn(0x7f38681027b0 msgr2=0x7f3868104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:22.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.884+0000 7f386c9a5700 1 -- 172.21.15.16:0/971115767 shutdown_connections 2024-03-27T06:34:22.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.884+0000 7f386c9a5700 1 -- 172.21.15.16:0/971115767 wait complete. 2024-03-27T06:34:22.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.884+0000 7f386c9a5700 1 Processor -- start 2024-03-27T06:34:22.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.884+0000 7f386c9a5700 1 -- start start 2024-03-27T06:34:22.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f386c9a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f3868108600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:22.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f386c9a5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f386810c770 con 0x7f3868108d80 2024-03-27T06:34:22.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f386659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f3868108600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:22.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f386659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f3868108600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39518/0 (socket says 172.21.15.16:39518) 2024-03-27T06:34:22.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f386659c700 1 -- 172.21.15.16:0/1563697998 learned_addr learned my addr 172.21.15.16:0/1563697998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:22.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f386659c700 1 -- 172.21.15.16:0/1563697998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3868108b40 con 0x7f3868108d80 2024-03-27T06:34:22.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f386659c700 1 --2- 172.21.15.16:0/1563697998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f3868108600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f385c000c00 tx=0x7f385c011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:22.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f38577fe700 1 -- 172.21.15.16:0/1563697998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f385c011e60 con 0x7f3868108d80 2024-03-27T06:34:22.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f38577fe700 1 -- 172.21.15.16:0/1563697998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f385c020070 con 0x7f3868108d80 2024-03-27T06:34:22.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f386c9a5700 1 -- 172.21.15.16:0/1563697998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3868106d30 con 0x7f3868108d80 2024-03-27T06:34:22.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f38577fe700 1 -- 172.21.15.16:0/1563697998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f385c019550 con 0x7f3868108d80 2024-03-27T06:34:22.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.885+0000 7f386c9a5700 1 -- 172.21.15.16:0/1563697998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38681071f0 con 0x7f3868108d80 2024-03-27T06:34:22.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.886+0000 7f38577fe700 1 -- 172.21.15.16:0/1563697998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46479+0+0 (secure 0 0 0) 0x7f385c019890 con 0x7f3868108d80 2024-03-27T06:34:22.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.886+0000 7f38577fe700 1 --2- 172.21.15.16:0/1563697998 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3850039420 0x7f385003b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:22.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.886+0000 7f38577fe700 1 -- 172.21.15.16:0/1563697998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f385c04e920 con 0x7f3868108d80 2024-03-27T06:34:22.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.887+0000 7f3865d9b700 1 --2- 172.21.15.16:0/1563697998 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3850039420 0x7f385003b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:22.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.887+0000 7f386c9a5700 1 -- 172.21.15.16:0/1563697998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3868063c70 con 0x7f3868108d80 2024-03-27T06:34:22.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.887+0000 7f3865d9b700 1 --2- 172.21.15.16:0/1563697998 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3850039420 0x7f385003b8e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3858006fd0 tx=0x7f3858006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:22.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:21.892+0000 7f38577fe700 1 -- 172.21.15.16:0/1563697998 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f385c023530 con 0x7f3868108d80 2024-03-27T06:34:22.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.106+0000 7f386c9a5700 1 -- 172.21.15.16:0/1563697998 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f38681078e0 con 0x7f3850039420 2024-03-27T06:34:22.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.217+0000 7f38577fe700 1 -- 172.21.15.16:0/1563697998 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f38681078e0 con 0x7f3850039420 2024-03-27T06:34:22.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.219+0000 7f386c9a5700 1 -- 172.21.15.16:0/1563697998 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3850039420 msgr2=0x7f385003b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:22.815 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.219+0000 7f386c9a5700 1 --2- 172.21.15.16:0/1563697998 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3850039420 0x7f385003b8e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3858006fd0 tx=0x7f3858006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:22.815 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.219+0000 7f386c9a5700 1 -- 172.21.15.16:0/1563697998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 msgr2=0x7f3868108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:22.815 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.219+0000 7f386c9a5700 1 --2- 172.21.15.16:0/1563697998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f3868108600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f385c000c00 tx=0x7f385c011c30 comp rx=0 tx=0).stop 2024-03-27T06:34:22.815 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.219+0000 7f386c9a5700 1 -- 172.21.15.16:0/1563697998 shutdown_connections 2024-03-27T06:34:22.815 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.219+0000 7f386c9a5700 1 --2- 172.21.15.16:0/1563697998 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3850039420 0x7f385003b8e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:22.815 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.219+0000 7f386c9a5700 1 --2- 172.21.15.16:0/1563697998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3868108d80 0x7f3868108600 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:22.815 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.219+0000 7f386c9a5700 1 -- 172.21.15.16:0/1563697998 >> 172.21.15.16:0/1563697998 conn(0x7f38681027b0 msgr2=0x7f3868103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:22.815 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.219+0000 7f386c9a5700 1 -- 172.21.15.16:0/1563697998 shutdown_connections 2024-03-27T06:34:22.816 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:22.219+0000 7f386c9a5700 1 -- 172.21.15.16:0/1563697998 wait complete. 2024-03-27T06:34:22.816 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-03-27T06:34:24.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:24.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:24.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$FtI5dlyeBSV84E7uZHNVGemv8WQbP47Kk1nrKyNaZwyVAn1GzG1oa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711521263, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-27T06:34:24.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.288+0000 7f3f42eba700 1 Processor -- start 2024-03-27T06:34:24.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.288+0000 7f3f42eba700 1 -- start start 2024-03-27T06:34:24.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.288+0000 7f3f42eba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c1075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:24.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.288+0000 7f3f42eba700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c107ba0 con 0x7f3f3c1071b0 2024-03-27T06:34:24.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.288+0000 7f3f40c56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c1075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:24.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.288+0000 7f3f40c56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c1075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39530/0 (socket says 172.21.15.16:39530) 2024-03-27T06:34:24.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.288+0000 7f3f40c56700 1 -- 172.21.15.16:0/2857527878 learned_addr learned my addr 172.21.15.16:0/2857527878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:24.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.288+0000 7f3f40c56700 1 -- 172.21.15.16:0/2857527878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c1083b0 con 0x7f3f3c1071b0 2024-03-27T06:34:24.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.289+0000 7f3f40c56700 1 --2- 172.21.15.16:0/2857527878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c1075d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3f300098a0 tx=0x7f3f30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d75fce4a4850b8a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:24.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.289+0000 7f3f3b7fe700 1 -- 172.21.15.16:0/2857527878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f30004160 con 0x7f3f3c1071b0 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.289+0000 7f3f3b7fe700 1 -- 172.21.15.16:0/2857527878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3f30004300 con 0x7f3f3c1071b0 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.289+0000 7f3f3b7fe700 1 -- 172.21.15.16:0/2857527878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f30011bb0 con 0x7f3f3c1071b0 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.289+0000 7f3f42eba700 1 -- 172.21.15.16:0/2857527878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 msgr2=0x7f3f3c1075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.289+0000 7f3f42eba700 1 --2- 172.21.15.16:0/2857527878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c1075d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3f300098a0 tx=0x7f3f30009bb0 comp rx=0 tx=0).stop 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f42eba700 1 -- 172.21.15.16:0/2857527878 shutdown_connections 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f42eba700 1 --2- 172.21.15.16:0/2857527878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c1075d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f42eba700 1 -- 172.21.15.16:0/2857527878 >> 172.21.15.16:0/2857527878 conn(0x7f3f3c102790 msgr2=0x7f3f3c104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f42eba700 1 -- 172.21.15.16:0/2857527878 shutdown_connections 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f42eba700 1 -- 172.21.15.16:0/2857527878 wait complete. 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f42eba700 1 Processor -- start 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f42eba700 1 -- start start 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f42eba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c07ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f42eba700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c10b760 con 0x7f3f3c1071b0 2024-03-27T06:34:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f40c56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c07ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f40c56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c07ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39542/0 (socket says 172.21.15.16:39542) 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.290+0000 7f3f40c56700 1 -- 172.21.15.16:0/1440239199 learned_addr learned my addr 172.21.15.16:0/1440239199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.291+0000 7f3f40c56700 1 -- 172.21.15.16:0/1440239199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c19eff0 con 0x7f3f3c1071b0 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.291+0000 7f3f40c56700 1 --2- 172.21.15.16:0/1440239199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c07ba50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3f30011bf0 tx=0x7f3f30011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.291+0000 7f3f39ffb700 1 -- 172.21.15.16:0/1440239199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f30003930 con 0x7f3f3c1071b0 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.291+0000 7f3f39ffb700 1 -- 172.21.15.16:0/1440239199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3f30003ad0 con 0x7f3f3c1071b0 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.291+0000 7f3f42eba700 1 -- 172.21.15.16:0/1440239199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c19f1c0 con 0x7f3f3c1071b0 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.291+0000 7f3f39ffb700 1 -- 172.21.15.16:0/1440239199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f30019760 con 0x7f3f3c1071b0 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.291+0000 7f3f42eba700 1 -- 172.21.15.16:0/1440239199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c19f560 con 0x7f3f3c1071b0 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.292+0000 7f3f39ffb700 1 -- 172.21.15.16:0/1440239199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46479+0+0 (secure 0 0 0) 0x7f3f3001ab50 con 0x7f3f3c1071b0 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.292+0000 7f3f39ffb700 1 --2- 172.21.15.16:0/1440239199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3f2c0393d0 0x7f3f2c03b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.292+0000 7f3f39ffb700 1 -- 172.21.15.16:0/1440239199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f3004da80 con 0x7f3f3c1071b0 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.292+0000 7f3f42eba700 1 -- 172.21.15.16:0/1440239199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3c063c70 con 0x7f3f3c1071b0 2024-03-27T06:34:24.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.293+0000 7f3f3bfff700 1 --2- 172.21.15.16:0/1440239199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3f2c0393d0 0x7f3f2c03b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.293+0000 7f3f3bfff700 1 --2- 172.21.15.16:0/1440239199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3f2c0393d0 0x7f3f2c03b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3f28006fd0 tx=0x7f3f28006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.298+0000 7f3f39ffb700 1 -- 172.21.15.16:0/1440239199 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f3f30003c80 con 0x7f3f3c1071b0 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.533+0000 7f3f42eba700 1 -- 172.21.15.16:0/1440239199 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f3f3c19f790 con 0x7f3f2c0393d0 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.779+0000 7f3f39ffb700 1 -- 172.21.15.16:0/1440239199 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f3f3c19f790 con 0x7f3f2c0393d0 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.782+0000 7f3f42eba700 1 -- 172.21.15.16:0/1440239199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3f2c0393d0 msgr2=0x7f3f2c03b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.782+0000 7f3f42eba700 1 --2- 172.21.15.16:0/1440239199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3f2c0393d0 0x7f3f2c03b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3f28006fd0 tx=0x7f3f28006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.782+0000 7f3f42eba700 1 -- 172.21.15.16:0/1440239199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 msgr2=0x7f3f3c07ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.782+0000 7f3f42eba700 1 --2- 172.21.15.16:0/1440239199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c07ba50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3f30011bf0 tx=0x7f3f30011cd0 comp rx=0 tx=0).stop 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.782+0000 7f3f42eba700 1 -- 172.21.15.16:0/1440239199 shutdown_connections 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.782+0000 7f3f42eba700 1 --2- 172.21.15.16:0/1440239199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3f2c0393d0 0x7f3f2c03b890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.782+0000 7f3f42eba700 1 --2- 172.21.15.16:0/1440239199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f3c1071b0 0x7f3f3c07ba50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.782+0000 7f3f42eba700 1 -- 172.21.15.16:0/1440239199 >> 172.21.15.16:0/1440239199 conn(0x7f3f3c102790 msgr2=0x7f3f3c103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.782+0000 7f3f42eba700 1 -- 172.21.15.16:0/1440239199 shutdown_connections 2024-03-27T06:34:24.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:23.782+0000 7f3f42eba700 1 -- 172.21.15.16:0/1440239199 wait complete. 2024-03-27T06:34:24.949 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-03-27T06:34:25.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:24 smithi016 ceph-mon[31908]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:34:26.176 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:25 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/653616801' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-27T06:34:26.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-03-27T06:34:26.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.390+0000 7f91e1a7e700 1 Processor -- start 2024-03-27T06:34:26.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.390+0000 7f91e1a7e700 1 -- start start 2024-03-27T06:34:26.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.390+0000 7f91e1a7e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc108730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:26.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.390+0000 7f91e1a7e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc108c70 con 0x7f91dc108310 2024-03-27T06:34:26.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.390+0000 7f91db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc108730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:26.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.390+0000 7f91db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc108730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39550/0 (socket says 172.21.15.16:39550) 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.390+0000 7f91db7fe700 1 -- 172.21.15.16:0/3756214474 learned_addr learned my addr 172.21.15.16:0/3756214474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.391+0000 7f91db7fe700 1 -- 172.21.15.16:0/3756214474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc108df0 con 0x7f91dc108310 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.391+0000 7f91db7fe700 1 --2- 172.21.15.16:0/3756214474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc108730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009e80 tx=0x7f91cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651a3f35a7f055ea server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.391+0000 7f91da7fc700 1 -- 172.21.15.16:0/3756214474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc017070 con 0x7f91dc108310 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.391+0000 7f91da7fc700 1 -- 172.21.15.16:0/3756214474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f91cc00b8c0 con 0x7f91dc108310 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.391+0000 7f91da7fc700 1 -- 172.21.15.16:0/3756214474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc004580 con 0x7f91dc108310 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.392+0000 7f91e1a7e700 1 -- 172.21.15.16:0/3756214474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 msgr2=0x7f91dc108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.392+0000 7f91e1a7e700 1 --2- 172.21.15.16:0/3756214474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc108730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009e80 tx=0x7f91cc00b330 comp rx=0 tx=0).stop 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.392+0000 7f91e1a7e700 1 -- 172.21.15.16:0/3756214474 shutdown_connections 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.392+0000 7f91e1a7e700 1 --2- 172.21.15.16:0/3756214474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc108730 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.392+0000 7f91e1a7e700 1 -- 172.21.15.16:0/3756214474 >> 172.21.15.16:0/3756214474 conn(0x7f91dc103990 msgr2=0x7f91dc105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.392+0000 7f91e1a7e700 1 -- 172.21.15.16:0/3756214474 shutdown_connections 2024-03-27T06:34:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.392+0000 7f91e1a7e700 1 -- 172.21.15.16:0/3756214474 wait complete. 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.392+0000 7f91e1a7e700 1 Processor -- start 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.392+0000 7f91e1a7e700 1 -- start start 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91e1a7e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91e1a7e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc10c780 con 0x7f91dc108310 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39566/0 (socket says 172.21.15.16:39566) 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91db7fe700 1 -- 172.21.15.16:0/653616801 learned_addr learned my addr 172.21.15.16:0/653616801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91db7fe700 1 -- 172.21.15.16:0/653616801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc07e0b0 con 0x7f91dc108310 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91db7fe700 1 --2- 172.21.15.16:0/653616801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc07db70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0072a0 tx=0x7f91cc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91d8ff9700 1 -- 172.21.15.16:0/653616801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc021020 con 0x7f91dc108310 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91e1a7e700 1 -- 172.21.15.16:0/653616801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc07c2a0 con 0x7f91dc108310 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91d8ff9700 1 -- 172.21.15.16:0/653616801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f91cc00bde0 con 0x7f91dc108310 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91d8ff9700 1 -- 172.21.15.16:0/653616801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91cc017040 con 0x7f91dc108310 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.393+0000 7f91e1a7e700 1 -- 172.21.15.16:0/653616801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc07c7e0 con 0x7f91dc108310 2024-03-27T06:34:26.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.394+0000 7f91d8ff9700 1 -- 172.21.15.16:0/653616801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46479+0+0 (secure 0 0 0) 0x7f91cc024020 con 0x7f91dc108310 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.394+0000 7f91e1a7e700 1 -- 172.21.15.16:0/653616801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91dc063c70 con 0x7f91dc108310 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.394+0000 7f91d8ff9700 1 --2- 172.21.15.16:0/653616801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f91c80393d0 0x7f91c803b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.394+0000 7f91d8ff9700 1 -- 172.21.15.16:0/653616801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f91cc00f050 con 0x7f91dc108310 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.395+0000 7f91daffd700 1 --2- 172.21.15.16:0/653616801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f91c80393d0 0x7f91c803b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.395+0000 7f91daffd700 1 --2- 172.21.15.16:0/653616801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f91c80393d0 0x7f91c803b890 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91c4006fd0 tx=0x7f91c4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.400+0000 7f91d8ff9700 1 -- 172.21.15.16:0/653616801 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f91cc02b020 con 0x7f91dc108310 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.609+0000 7f91e1a7e700 1 -- 172.21.15.16:0/653616801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f91dc07d100 con 0x7f91dc108310 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.609+0000 7f91d8ff9700 1 -- 172.21.15.16:0/653616801 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f91cc01db20 con 0x7f91dc108310 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.611+0000 7f91e1a7e700 1 -- 172.21.15.16:0/653616801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f91c80393d0 msgr2=0x7f91c803b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.611+0000 7f91e1a7e700 1 --2- 172.21.15.16:0/653616801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f91c80393d0 0x7f91c803b890 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91c4006fd0 tx=0x7f91c4006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.611+0000 7f91e1a7e700 1 -- 172.21.15.16:0/653616801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 msgr2=0x7f91dc07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.611+0000 7f91e1a7e700 1 --2- 172.21.15.16:0/653616801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc07db70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0072a0 tx=0x7f91cc003d00 comp rx=0 tx=0).stop 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.612+0000 7f91e1a7e700 1 -- 172.21.15.16:0/653616801 shutdown_connections 2024-03-27T06:34:26.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.612+0000 7f91e1a7e700 1 --2- 172.21.15.16:0/653616801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f91c80393d0 0x7f91c803b890 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:26.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.612+0000 7f91e1a7e700 1 --2- 172.21.15.16:0/653616801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc108310 0x7f91dc07db70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:26.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.612+0000 7f91e1a7e700 1 -- 172.21.15.16:0/653616801 >> 172.21.15.16:0/653616801 conn(0x7f91dc103990 msgr2=0x7f91dc1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:26.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.612+0000 7f91e1a7e700 1 -- 172.21.15.16:0/653616801 shutdown_connections 2024-03-27T06:34:26.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:25.612+0000 7f91e1a7e700 1 -- 172.21.15.16:0/653616801 wait complete. 2024-03-27T06:34:26.191 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-27T06:34:26.191 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-03-27T06:34:26.198 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-27T06:34:26.198 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-03-27T06:34:26.198 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-03-27T06:34:26.198 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-27T06:34:26.200 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-03-27T06:34:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016.front.sepia.ceph.com:8443/ 2024-03-27T06:34:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-03-27T06:34:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: Password: oezwaditd0 2024-03-27T06:34:26.200 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:26.201 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config directory 2024-03-27T06:34:26.201 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling autotune for osd_memory_target 2024-03-27T06:34:27.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:26 smithi016 ceph-mon[31908]: mgrmap e13: x(active, since 6s) 2024-03-27T06:34:27.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.605+0000 7f4b5585f700 1 Processor -- start 2024-03-27T06:34:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.605+0000 7f4b5585f700 1 -- start start 2024-03-27T06:34:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.606+0000 7f4b5585f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b5010b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.606+0000 7f4b5585f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5007be50 con 0x7f4b50108d80 2024-03-27T06:34:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.606+0000 7f4b4effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b5010b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.606+0000 7f4b4effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b5010b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42902/0 (socket says 172.21.15.16:42902) 2024-03-27T06:34:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.606+0000 7f4b4effd700 1 -- 172.21.15.16:0/1756841815 learned_addr learned my addr 172.21.15.16:0/1756841815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.606+0000 7f4b4effd700 1 -- 172.21.15.16:0/1756841815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5010b6b0 con 0x7f4b50108d80 2024-03-27T06:34:27.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.606+0000 7f4b4effd700 1 --2- 172.21.15.16:0/1756841815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b5010b170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009e80 tx=0x7f4b4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e939793253a55fd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.606+0000 7f4b4dffb700 1 -- 172.21.15.16:0/1756841815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b48017070 con 0x7f4b50108d80 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.606+0000 7f4b4dffb700 1 -- 172.21.15.16:0/1756841815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4b4800b8c0 con 0x7f4b50108d80 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.606+0000 7f4b4dffb700 1 -- 172.21.15.16:0/1756841815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b48004580 con 0x7f4b50108d80 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.607+0000 7f4b5585f700 1 -- 172.21.15.16:0/1756841815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 msgr2=0x7f4b5010b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.607+0000 7f4b5585f700 1 --2- 172.21.15.16:0/1756841815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b5010b170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009e80 tx=0x7f4b4800b330 comp rx=0 tx=0).stop 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.607+0000 7f4b5585f700 1 -- 172.21.15.16:0/1756841815 shutdown_connections 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.607+0000 7f4b5585f700 1 --2- 172.21.15.16:0/1756841815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b5010b170 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.607+0000 7f4b5585f700 1 -- 172.21.15.16:0/1756841815 >> 172.21.15.16:0/1756841815 conn(0x7f4b50102790 msgr2=0x7f4b50104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.607+0000 7f4b5585f700 1 -- 172.21.15.16:0/1756841815 shutdown_connections 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.607+0000 7f4b5585f700 1 -- 172.21.15.16:0/1756841815 wait complete. 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.608+0000 7f4b5585f700 1 Processor -- start 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.608+0000 7f4b5585f700 1 -- start start 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.608+0000 7f4b5585f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b50108600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:27.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.608+0000 7f4b5585f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5010c720 con 0x7f4b50108d80 2024-03-27T06:34:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.608+0000 7f4b4effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b50108600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.608+0000 7f4b4effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b50108600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42906/0 (socket says 172.21.15.16:42906) 2024-03-27T06:34:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.608+0000 7f4b4effd700 1 -- 172.21.15.16:0/3193934532 learned_addr learned my addr 172.21.15.16:0/3193934532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.608+0000 7f4b4effd700 1 -- 172.21.15.16:0/3193934532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b50108b40 con 0x7f4b50108d80 2024-03-27T06:34:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.608+0000 7f4b4effd700 1 --2- 172.21.15.16:0/3193934532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b50108600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b480072a0 tx=0x7f4b48003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.609+0000 7f4b5485d700 1 -- 172.21.15.16:0/3193934532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b48021020 con 0x7f4b50108d80 2024-03-27T06:34:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.609+0000 7f4b5485d700 1 -- 172.21.15.16:0/3193934532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4b4800bde0 con 0x7f4b50108d80 2024-03-27T06:34:27.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.609+0000 7f4b5585f700 1 -- 172.21.15.16:0/3193934532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b50106d30 con 0x7f4b50108d80 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.609+0000 7f4b5485d700 1 -- 172.21.15.16:0/3193934532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b48017040 con 0x7f4b50108d80 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.609+0000 7f4b5585f700 1 -- 172.21.15.16:0/3193934532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b50107270 con 0x7f4b50108d80 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.610+0000 7f4b5485d700 1 -- 172.21.15.16:0/3193934532 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f4b48024020 con 0x7f4b50108d80 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.610+0000 7f4b5585f700 1 -- 172.21.15.16:0/3193934532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b50063c70 con 0x7f4b50108d80 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.610+0000 7f4b5485d700 1 --2- 172.21.15.16:0/3193934532 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4b3c0394d0 0x7f4b3c03b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.610+0000 7f4b5485d700 1 -- 172.21.15.16:0/3193934532 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4b4800f050 con 0x7f4b50108d80 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.610+0000 7f4b4e7fc700 1 --2- 172.21.15.16:0/3193934532 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4b3c0394d0 0x7f4b3c03b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.610+0000 7f4b4e7fc700 1 --2- 172.21.15.16:0/3193934532 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4b3c0394d0 0x7f4b3c03b990 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4b38006fd0 tx=0x7f4b38006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.615+0000 7f4b5485d700 1 -- 172.21.15.16:0/3193934532 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4b4802b020 con 0x7f4b50108d80 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.821+0000 7f4b5585f700 1 -- 172.21.15.16:0/3193934532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f4b501075b0 con 0x7f4b50108d80 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.822+0000 7f4b5485d700 1 -- 172.21.15.16:0/3193934532 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f4b501075b0 con 0x7f4b50108d80 2024-03-27T06:34:27.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.824+0000 7f4b5585f700 1 -- 172.21.15.16:0/3193934532 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4b3c0394d0 msgr2=0x7f4b3c03b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.824+0000 7f4b5585f700 1 --2- 172.21.15.16:0/3193934532 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4b3c0394d0 0x7f4b3c03b990 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4b38006fd0 tx=0x7f4b38006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.824+0000 7f4b5585f700 1 -- 172.21.15.16:0/3193934532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 msgr2=0x7f4b50108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.824+0000 7f4b5585f700 1 --2- 172.21.15.16:0/3193934532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b50108600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b480072a0 tx=0x7f4b48003d00 comp rx=0 tx=0).stop 2024-03-27T06:34:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.824+0000 7f4b5585f700 1 -- 172.21.15.16:0/3193934532 shutdown_connections 2024-03-27T06:34:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.824+0000 7f4b5585f700 1 --2- 172.21.15.16:0/3193934532 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4b3c0394d0 0x7f4b3c03b990 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.824+0000 7f4b5585f700 1 --2- 172.21.15.16:0/3193934532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b50108d80 0x7f4b50108600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.824+0000 7f4b5585f700 1 -- 172.21.15.16:0/3193934532 >> 172.21.15.16:0/3193934532 conn(0x7f4b50102790 msgr2=0x7f4b50103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.824+0000 7f4b5585f700 1 -- 172.21.15.16:0/3193934532 shutdown_connections 2024-03-27T06:34:27.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:26.824+0000 7f4b5585f700 1 -- 172.21.15.16:0/3193934532 wait complete. 2024-03-27T06:34:28.823 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.879+0000 7ff8da2e6700 1 Processor -- start 2024-03-27T06:34:28.823 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.879+0000 7ff8da2e6700 1 -- start start 2024-03-27T06:34:28.823 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.879+0000 7ff8da2e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d4108730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:28.823 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.879+0000 7ff8da2e6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d4108c70 con 0x7ff8d4108310 2024-03-27T06:34:28.823 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.879+0000 7ff8d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d4108730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.879+0000 7ff8d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d4108730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42908/0 (socket says 172.21.15.16:42908) 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.879+0000 7ff8d3fff700 1 -- 172.21.15.16:0/2722458448 learned_addr learned my addr 172.21.15.16:0/2722458448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.880+0000 7ff8d3fff700 1 -- 172.21.15.16:0/2722458448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d4108df0 con 0x7ff8d4108310 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.880+0000 7ff8d3fff700 1 --2- 172.21.15.16:0/2722458448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d4108730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40098a0 tx=0x7ff8c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccab2ebd42699c0f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.880+0000 7ff8d2ffd700 1 -- 172.21.15.16:0/2722458448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c4004160 con 0x7ff8d4108310 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.880+0000 7ff8d2ffd700 1 -- 172.21.15.16:0/2722458448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8c4004300 con 0x7ff8d4108310 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.880+0000 7ff8d2ffd700 1 -- 172.21.15.16:0/2722458448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c4011bb0 con 0x7ff8d4108310 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.880+0000 7ff8da2e6700 1 -- 172.21.15.16:0/2722458448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 msgr2=0x7ff8d4108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.880+0000 7ff8da2e6700 1 --2- 172.21.15.16:0/2722458448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d4108730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40098a0 tx=0x7ff8c4009bb0 comp rx=0 tx=0).stop 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.880+0000 7ff8da2e6700 1 -- 172.21.15.16:0/2722458448 shutdown_connections 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.880+0000 7ff8da2e6700 1 --2- 172.21.15.16:0/2722458448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d4108730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.880+0000 7ff8da2e6700 1 -- 172.21.15.16:0/2722458448 >> 172.21.15.16:0/2722458448 conn(0x7ff8d4103990 msgr2=0x7ff8d4105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.881+0000 7ff8da2e6700 1 -- 172.21.15.16:0/2722458448 shutdown_connections 2024-03-27T06:34:28.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.881+0000 7ff8da2e6700 1 -- 172.21.15.16:0/2722458448 wait complete. 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.881+0000 7ff8da2e6700 1 Processor -- start 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.881+0000 7ff8da2e6700 1 -- start start 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.881+0000 7ff8da2e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.881+0000 7ff8da2e6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410c780 con 0x7ff8d4108310 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.882+0000 7ff8d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.882+0000 7ff8d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d419c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42920/0 (socket says 172.21.15.16:42920) 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.882+0000 7ff8d3fff700 1 -- 172.21.15.16:0/970601555 learned_addr learned my addr 172.21.15.16:0/970601555 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.882+0000 7ff8d3fff700 1 -- 172.21.15.16:0/970601555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d419c740 con 0x7ff8d4108310 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.882+0000 7ff8d3fff700 1 --2- 172.21.15.16:0/970601555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d419c200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4011870 tx=0x7ff8c4011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.882+0000 7ff8d17fa700 1 -- 172.21.15.16:0/970601555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c4003af0 con 0x7ff8d4108310 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.882+0000 7ff8d17fa700 1 -- 172.21.15.16:0/970601555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8c4003c90 con 0x7ff8d4108310 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.882+0000 7ff8da2e6700 1 -- 172.21.15.16:0/970601555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d419c9d0 con 0x7ff8d4108310 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.882+0000 7ff8d17fa700 1 -- 172.21.15.16:0/970601555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c4019960 con 0x7ff8d4108310 2024-03-27T06:34:28.825 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.882+0000 7ff8da2e6700 1 -- 172.21.15.16:0/970601555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d419ce30 con 0x7ff8d4108310 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.883+0000 7ff8d17fa700 1 -- 172.21.15.16:0/970601555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7ff8c4019c10 con 0x7ff8d4108310 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.883+0000 7ff8da2e6700 1 -- 172.21.15.16:0/970601555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d4063c70 con 0x7ff8d4108310 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.883+0000 7ff8d17fa700 1 --2- 172.21.15.16:0/970601555 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8bc0394d0 0x7ff8bc03b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.883+0000 7ff8d17fa700 1 -- 172.21.15.16:0/970601555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff8c404dc90 con 0x7ff8d4108310 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.883+0000 7ff8d37fe700 1 --2- 172.21.15.16:0/970601555 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8bc0394d0 0x7ff8bc03b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.884+0000 7ff8d37fe700 1 --2- 172.21.15.16:0/970601555 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8bc0394d0 0x7ff8bc03b990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8006fd0 tx=0x7ff8c8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:27.889+0000 7ff8d17fa700 1 -- 172.21.15.16:0/970601555 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff8c401ad10 con 0x7ff8d4108310 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.208+0000 7ff8da2e6700 1 -- 172.21.15.16:0/970601555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7ff8d419f800 con 0x7ff8d4108310 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.210+0000 7ff8d17fa700 1 -- 172.21.15.16:0/970601555 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7ff8c402dce0 con 0x7ff8d4108310 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.212+0000 7ff8da2e6700 1 -- 172.21.15.16:0/970601555 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8bc0394d0 msgr2=0x7ff8bc03b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.212+0000 7ff8da2e6700 1 --2- 172.21.15.16:0/970601555 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8bc0394d0 0x7ff8bc03b990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8006fd0 tx=0x7ff8c8006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.212+0000 7ff8da2e6700 1 -- 172.21.15.16:0/970601555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 msgr2=0x7ff8d419c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.212+0000 7ff8da2e6700 1 --2- 172.21.15.16:0/970601555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d419c200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4011870 tx=0x7ff8c4011d70 comp rx=0 tx=0).stop 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.212+0000 7ff8da2e6700 1 -- 172.21.15.16:0/970601555 shutdown_connections 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.212+0000 7ff8da2e6700 1 --2- 172.21.15.16:0/970601555 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8bc0394d0 0x7ff8bc03b990 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.213+0000 7ff8da2e6700 1 --2- 172.21.15.16:0/970601555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8d4108310 0x7ff8d419c200 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.213+0000 7ff8da2e6700 1 -- 172.21.15.16:0/970601555 >> 172.21.15.16:0/970601555 conn(0x7ff8d4103990 msgr2=0x7ff8d41043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.213+0000 7ff8da2e6700 1 -- 172.21.15.16:0/970601555 shutdown_connections 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-03-27T06:34:28.213+0000 7ff8da2e6700 1 -- 172.21.15.16:0/970601555 wait complete. 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-27T06:34:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:28.828 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-27T06:34:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-03-27T06:34:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:28.828 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-03-27T06:34:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-27T06:34:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:28.828 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-03-27T06:34:28.863 INFO:tasks.cephadm:Fetching config... 2024-03-27T06:34:28.864 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:34:28.864 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-27T06:34:28.879 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-27T06:34:28.879 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:34:28.879 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-27T06:34:28.935 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-27T06:34:28.935 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:34:28.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-27T06:34:29.003 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-27T06:34:29.003 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:34:29.003 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-27T06:34:29.058 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-27T06:34:29.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI4CrxITdb3xZjmn8nh7O0RL5aLUEGF6QHCFBJi304Ok6t9YNvf3ifmo1ibUgyaw2WYwABPn2BSRLzeKvEtT5psis1a8pctXgE8ZpVW4BMSvigNUjU2WHWbaaZZ4LTxna8dQPhUPyYtB0Xq2gZX2CudBKdjxUrBnuDOdbSI+IV08+SjvjumEutuz1swhMfuSpqnxMUpLHFuMaxcAR7xGRjmsuhYNoYIGfLTvZTg3pgQJMn3iv0SlEIDrV6rfLIP1I6Id7VnGq+sOYMITZGKsETDwm7FdOl3S7Uh/GyCZHAN49P+kNfC7dA+mqG94Wv2BuJ8ab+1ufxrB/KK4IwbjOeIaiLydlvK8VznzTSm3G2ZOh3E22HFDVffeCDO4LxhyOo2BxcOiZoDCf1HkrW+2go+jB3QzRJ8qnEr3REv27wAVfTaPds54WOTNJsCoNAmf71bJbxLYZfQH1vkxp/njaPQ8zpOtUQIedcSXsLX+pgd5jsqkYqTdvZv3FyZxryGm0= ceph-c9095bc4-ec03-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-27T06:34:29.138 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI4CrxITdb3xZjmn8nh7O0RL5aLUEGF6QHCFBJi304Ok6t9YNvf3ifmo1ibUgyaw2WYwABPn2BSRLzeKvEtT5psis1a8pctXgE8ZpVW4BMSvigNUjU2WHWbaaZZ4LTxna8dQPhUPyYtB0Xq2gZX2CudBKdjxUrBnuDOdbSI+IV08+SjvjumEutuz1swhMfuSpqnxMUpLHFuMaxcAR7xGRjmsuhYNoYIGfLTvZTg3pgQJMn3iv0SlEIDrV6rfLIP1I6Id7VnGq+sOYMITZGKsETDwm7FdOl3S7Uh/GyCZHAN49P+kNfC7dA+mqG94Wv2BuJ8ab+1ufxrB/KK4IwbjOeIaiLydlvK8VznzTSm3G2ZOh3E22HFDVffeCDO4LxhyOo2BxcOiZoDCf1HkrW+2go+jB3QzRJ8qnEr3REv27wAVfTaPds54WOTNJsCoNAmf71bJbxLYZfQH1vkxp/njaPQ8zpOtUQIedcSXsLX+pgd5jsqkYqTdvZv3FyZxryGm0= ceph-c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:34:29.152 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI4CrxITdb3xZjmn8nh7O0RL5aLUEGF6QHCFBJi304Ok6t9YNvf3ifmo1ibUgyaw2WYwABPn2BSRLzeKvEtT5psis1a8pctXgE8ZpVW4BMSvigNUjU2WHWbaaZZ4LTxna8dQPhUPyYtB0Xq2gZX2CudBKdjxUrBnuDOdbSI+IV08+SjvjumEutuz1swhMfuSpqnxMUpLHFuMaxcAR7xGRjmsuhYNoYIGfLTvZTg3pgQJMn3iv0SlEIDrV6rfLIP1I6Id7VnGq+sOYMITZGKsETDwm7FdOl3S7Uh/GyCZHAN49P+kNfC7dA+mqG94Wv2BuJ8ab+1ufxrB/KK4IwbjOeIaiLydlvK8VznzTSm3G2ZOh3E22HFDVffeCDO4LxhyOo2BxcOiZoDCf1HkrW+2go+jB3QzRJ8qnEr3REv27wAVfTaPds54WOTNJsCoNAmf71bJbxLYZfQH1vkxp/njaPQ8zpOtUQIedcSXsLX+pgd5jsqkYqTdvZv3FyZxryGm0= ceph-c9095bc4-ec03-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-27T06:34:29.200 INFO:teuthology.orchestra.run.smithi156.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI4CrxITdb3xZjmn8nh7O0RL5aLUEGF6QHCFBJi304Ok6t9YNvf3ifmo1ibUgyaw2WYwABPn2BSRLzeKvEtT5psis1a8pctXgE8ZpVW4BMSvigNUjU2WHWbaaZZ4LTxna8dQPhUPyYtB0Xq2gZX2CudBKdjxUrBnuDOdbSI+IV08+SjvjumEutuz1swhMfuSpqnxMUpLHFuMaxcAR7xGRjmsuhYNoYIGfLTvZTg3pgQJMn3iv0SlEIDrV6rfLIP1I6Id7VnGq+sOYMITZGKsETDwm7FdOl3S7Uh/GyCZHAN49P+kNfC7dA+mqG94Wv2BuJ8ab+1ufxrB/KK4IwbjOeIaiLydlvK8VznzTSm3G2ZOh3E22HFDVffeCDO4LxhyOo2BxcOiZoDCf1HkrW+2go+jB3QzRJ8qnEr3REv27wAVfTaPds54WOTNJsCoNAmf71bJbxLYZfQH1vkxp/njaPQ8zpOtUQIedcSXsLX+pgd5jsqkYqTdvZv3FyZxryGm0= ceph-c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:34:29.215 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI4CrxITdb3xZjmn8nh7O0RL5aLUEGF6QHCFBJi304Ok6t9YNvf3ifmo1ibUgyaw2WYwABPn2BSRLzeKvEtT5psis1a8pctXgE8ZpVW4BMSvigNUjU2WHWbaaZZ4LTxna8dQPhUPyYtB0Xq2gZX2CudBKdjxUrBnuDOdbSI+IV08+SjvjumEutuz1swhMfuSpqnxMUpLHFuMaxcAR7xGRjmsuhYNoYIGfLTvZTg3pgQJMn3iv0SlEIDrV6rfLIP1I6Id7VnGq+sOYMITZGKsETDwm7FdOl3S7Uh/GyCZHAN49P+kNfC7dA+mqG94Wv2BuJ8ab+1ufxrB/KK4IwbjOeIaiLydlvK8VznzTSm3G2ZOh3E22HFDVffeCDO4LxhyOo2BxcOiZoDCf1HkrW+2go+jB3QzRJ8qnEr3REv27wAVfTaPds54WOTNJsCoNAmf71bJbxLYZfQH1vkxp/njaPQ8zpOtUQIedcSXsLX+pgd5jsqkYqTdvZv3FyZxryGm0= ceph-c9095bc4-ec03-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-27T06:34:29.261 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI4CrxITdb3xZjmn8nh7O0RL5aLUEGF6QHCFBJi304Ok6t9YNvf3ifmo1ibUgyaw2WYwABPn2BSRLzeKvEtT5psis1a8pctXgE8ZpVW4BMSvigNUjU2WHWbaaZZ4LTxna8dQPhUPyYtB0Xq2gZX2CudBKdjxUrBnuDOdbSI+IV08+SjvjumEutuz1swhMfuSpqnxMUpLHFuMaxcAR7xGRjmsuhYNoYIGfLTvZTg3pgQJMn3iv0SlEIDrV6rfLIP1I6Id7VnGq+sOYMITZGKsETDwm7FdOl3S7Uh/GyCZHAN49P+kNfC7dA+mqG94Wv2BuJ8ab+1ufxrB/KK4IwbjOeIaiLydlvK8VznzTSm3G2ZOh3E22HFDVffeCDO4LxhyOo2BxcOiZoDCf1HkrW+2go+jB3QzRJ8qnEr3REv27wAVfTaPds54WOTNJsCoNAmf71bJbxLYZfQH1vkxp/njaPQ8zpOtUQIedcSXsLX+pgd5jsqkYqTdvZv3FyZxryGm0= ceph-c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:34:29.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-27T06:34:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:29 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/970601555' entity='client.admin' 2024-03-27T06:34:29.517 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:34:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.383+0000 7f8d2d29e700 1 -- 172.21.15.16:0/906857751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28102b00 msgr2=0x7f8d28104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.383+0000 7f8d2d29e700 1 --2- 172.21.15.16:0/906857751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28102b00 0x7f8d28104ef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009910 tx=0x7f8d18009c20 comp rx=0 tx=0).stop 2024-03-27T06:34:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.383+0000 7f8d2d29e700 1 -- 172.21.15.16:0/906857751 shutdown_connections 2024-03-27T06:34:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.383+0000 7f8d2d29e700 1 --2- 172.21.15.16:0/906857751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28102b00 0x7f8d28104ef0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.383+0000 7f8d2d29e700 1 -- 172.21.15.16:0/906857751 >> 172.21.15.16:0/906857751 conn(0x7f8d280fc510 msgr2=0x7f8d280fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.384+0000 7f8d2d29e700 1 -- 172.21.15.16:0/906857751 shutdown_connections 2024-03-27T06:34:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.384+0000 7f8d2d29e700 1 -- 172.21.15.16:0/906857751 wait complete. 2024-03-27T06:34:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.384+0000 7f8d2d29e700 1 Processor -- start 2024-03-27T06:34:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.384+0000 7f8d2d29e700 1 -- start start 2024-03-27T06:34:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d2d29e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28102b00 0x7f8d28199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d2d29e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d281081e0 con 0x7f8d28102b00 2024-03-27T06:34:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d26ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28102b00 0x7f8d28199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d26ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28102b00 0x7f8d28199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42934/0 (socket says 172.21.15.16:42934) 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d26ffd700 1 -- 172.21.15.16:0/3855459822 learned_addr learned my addr 172.21.15.16:0/3855459822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d26ffd700 1 -- 172.21.15.16:0/3855459822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d28199e40 con 0x7f8d28102b00 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d26ffd700 1 --2- 172.21.15.16:0/3855459822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28102b00 0x7f8d28199900 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005950 tx=0x7f8d18004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d0ffff700 1 -- 172.21.15.16:0/3855459822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d18027070 con 0x7f8d28102b00 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d0ffff700 1 -- 172.21.15.16:0/3855459822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8d18022470 con 0x7f8d28102b00 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d2d29e700 1 -- 172.21.15.16:0/3855459822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2819a130 con 0x7f8d28102b00 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d0ffff700 1 -- 172.21.15.16:0/3855459822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d1800f590 con 0x7f8d28102b00 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.385+0000 7f8d2d29e700 1 -- 172.21.15.16:0/3855459822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2819cdc0 con 0x7f8d28102b00 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.386+0000 7f8d0ffff700 1 -- 172.21.15.16:0/3855459822 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f8d1802a070 con 0x7f8d28102b00 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.386+0000 7f8d2d29e700 1 -- 172.21.15.16:0/3855459822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d28108960 con 0x7f8d28102b00 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.386+0000 7f8d0ffff700 1 --2- 172.21.15.16:0/3855459822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100394d0 0x7f8d1003b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.387+0000 7f8d0ffff700 1 -- 172.21.15.16:0/3855459822 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8d1801e030 con 0x7f8d28102b00 2024-03-27T06:34:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.387+0000 7f8d267fc700 1 --2- 172.21.15.16:0/3855459822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100394d0 0x7f8d1003b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.387+0000 7f8d267fc700 1 --2- 172.21.15.16:0/3855459822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100394d0 0x7f8d1003b990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c006fd0 tx=0x7f8d1c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.392+0000 7f8d0ffff700 1 -- 172.21.15.16:0/3855459822 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8d18013070 con 0x7f8d28102b00 2024-03-27T06:34:32.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:32 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:32.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:32 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:32.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:32 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:34:32.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:32 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:32.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:32 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:34:32.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:32 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:32.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.619+0000 7f8d2d29e700 1 -- 172.21.15.16:0/3855459822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f8d2802df20 con 0x7f8d28102b00 2024-03-27T06:34:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.622+0000 7f8d0ffff700 1 -- 172.21.15.16:0/3855459822 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8d180292c0 con 0x7f8d28102b00 2024-03-27T06:34:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.625+0000 7f8d2d29e700 1 -- 172.21.15.16:0/3855459822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100394d0 msgr2=0x7f8d1003b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.625+0000 7f8d2d29e700 1 --2- 172.21.15.16:0/3855459822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100394d0 0x7f8d1003b990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c006fd0 tx=0x7f8d1c006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:32.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.625+0000 7f8d2d29e700 1 -- 172.21.15.16:0/3855459822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28102b00 msgr2=0x7f8d28199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.625+0000 7f8d2d29e700 1 --2- 172.21.15.16:0/3855459822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28102b00 0x7f8d28199900 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005950 tx=0x7f8d18004740 comp rx=0 tx=0).stop 2024-03-27T06:34:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.625+0000 7f8d2d29e700 1 -- 172.21.15.16:0/3855459822 shutdown_connections 2024-03-27T06:34:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.625+0000 7f8d2d29e700 1 --2- 172.21.15.16:0/3855459822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100394d0 0x7f8d1003b990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.625+0000 7f8d2d29e700 1 --2- 172.21.15.16:0/3855459822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d28102b00 0x7f8d28199900 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.625+0000 7f8d2d29e700 1 -- 172.21.15.16:0/3855459822 >> 172.21.15.16:0/3855459822 conn(0x7f8d280fc510 msgr2=0x7f8d280fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.625+0000 7f8d2d29e700 1 -- 172.21.15.16:0/3855459822 shutdown_connections 2024-03-27T06:34:32.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:32.625+0000 7f8d2d29e700 1 -- 172.21.15.16:0/3855459822 wait complete. 2024-03-27T06:34:33.509 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-27T06:34:33.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-27T06:34:33.755 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:34:33.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:33 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3855459822' entity='client.admin' 2024-03-27T06:34:33.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:33 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:34:33.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:33 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.413+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/331788507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100f70 msgr2=0x7f7c58101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.413+0000 7f7c5ed3c700 1 --2- 172.21.15.16:0/331788507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100f70 0x7f7c58101390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009910 tx=0x7f7c4c009c20 comp rx=0 tx=0).stop 2024-03-27T06:34:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.413+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/331788507 shutdown_connections 2024-03-27T06:34:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.413+0000 7f7c5ed3c700 1 --2- 172.21.15.16:0/331788507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100f70 0x7f7c58101390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.413+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/331788507 >> 172.21.15.16:0/331788507 conn(0x7f7c580fc510 msgr2=0x7f7c580fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.413+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/331788507 shutdown_connections 2024-03-27T06:34:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.413+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/331788507 wait complete. 2024-03-27T06:34:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.414+0000 7f7c5ed3c700 1 Processor -- start 2024-03-27T06:34:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.414+0000 7f7c5ed3c700 1 -- start start 2024-03-27T06:34:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.414+0000 7f7c5ed3c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100f70 0x7f7c581966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.414+0000 7f7c5ed3c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58107140 con 0x7f7c58100f70 2024-03-27T06:34:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.414+0000 7f7c5cad8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100f70 0x7f7c581966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.414+0000 7f7c5cad8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100f70 0x7f7c581966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42946/0 (socket says 172.21.15.16:42946) 2024-03-27T06:34:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.414+0000 7f7c5cad8700 1 -- 172.21.15.16:0/4245014073 learned_addr learned my addr 172.21.15.16:0/4245014073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.414+0000 7f7c5cad8700 1 -- 172.21.15.16:0/4245014073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c58196c00 con 0x7f7c58100f70 2024-03-27T06:34:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.415+0000 7f7c5cad8700 1 --2- 172.21.15.16:0/4245014073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100f70 0x7f7c581966c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c005840 tx=0x7f7c4c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.415+0000 7f7c55ffb700 1 -- 172.21.15.16:0/4245014073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c4c029070 con 0x7f7c58100f70 2024-03-27T06:34:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.415+0000 7f7c55ffb700 1 -- 172.21.15.16:0/4245014073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7c4c019650 con 0x7f7c58100f70 2024-03-27T06:34:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.415+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/4245014073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c58196e90 con 0x7f7c58100f70 2024-03-27T06:34:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.415+0000 7f7c55ffb700 1 -- 172.21.15.16:0/4245014073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c4c024720 con 0x7f7c58100f70 2024-03-27T06:34:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.415+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/4245014073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5806a2c0 con 0x7f7c58100f70 2024-03-27T06:34:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.416+0000 7f7c55ffb700 1 -- 172.21.15.16:0/4245014073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f7c4c02c070 con 0x7f7c58100f70 2024-03-27T06:34:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.416+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/4245014073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c581078c0 con 0x7f7c58100f70 2024-03-27T06:34:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.416+0000 7f7c55ffb700 1 --2- 172.21.15.16:0/4245014073 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c48039470 0x7f7c4803b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.416+0000 7f7c55ffb700 1 -- 172.21.15.16:0/4245014073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7c4c020030 con 0x7f7c58100f70 2024-03-27T06:34:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.417+0000 7f7c57fff700 1 --2- 172.21.15.16:0/4245014073 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c48039470 0x7f7c4803b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.417+0000 7f7c57fff700 1 --2- 172.21.15.16:0/4245014073 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c48039470 0x7f7c4803b930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7c44006fd0 tx=0x7f7c44006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.421+0000 7f7c55ffb700 1 -- 172.21.15.16:0/4245014073 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7c4c013070 con 0x7f7c58100f70 2024-03-27T06:34:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.632+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/4245014073 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f7c5806aee0 con 0x7f7c48039470 2024-03-27T06:34:35.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.636+0000 7f7c55ffb700 1 -- 172.21.15.16:0/4245014073 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7c5806aee0 con 0x7f7c48039470 2024-03-27T06:34:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.638+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/4245014073 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c48039470 msgr2=0x7f7c4803b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.638+0000 7f7c5ed3c700 1 --2- 172.21.15.16:0/4245014073 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c48039470 0x7f7c4803b930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7c44006fd0 tx=0x7f7c44006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.638+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/4245014073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100f70 msgr2=0x7f7c581966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.638+0000 7f7c5ed3c700 1 --2- 172.21.15.16:0/4245014073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100f70 0x7f7c581966c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c005840 tx=0x7f7c4c0050b0 comp rx=0 tx=0).stop 2024-03-27T06:34:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.638+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/4245014073 shutdown_connections 2024-03-27T06:34:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.638+0000 7f7c5ed3c700 1 --2- 172.21.15.16:0/4245014073 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c48039470 0x7f7c4803b930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.638+0000 7f7c5ed3c700 1 --2- 172.21.15.16:0/4245014073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58100f70 0x7f7c581966c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.638+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/4245014073 >> 172.21.15.16:0/4245014073 conn(0x7f7c580fc510 msgr2=0x7f7c580fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.638+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/4245014073 shutdown_connections 2024-03-27T06:34:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:35.638+0000 7f7c5ed3c700 1 -- 172.21.15.16:0/4245014073 wait complete. 2024-03-27T06:34:36.381 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi156 2024-03-27T06:34:36.381 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-27T06:34:36.381 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.conf 2024-03-27T06:34:36.400 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-27T06:34:36.400 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:34:36.457 INFO:tasks.cephadm:Adding host smithi156 to orchestrator... 2024-03-27T06:34:36.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch host add smithi156 2024-03-27T06:34:36.705 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: Updating smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: Updating smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.152+0000 7fc8843e8700 1 -- 172.21.15.16:0/2870900591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c101160 msgr2=0x7fc87c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.152+0000 7fc8843e8700 1 --2- 172.21.15.16:0/2870900591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c101160 0x7fc87c101540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc874009910 tx=0x7fc874009c20 comp rx=0 tx=0).stop 2024-03-27T06:34:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.152+0000 7fc8843e8700 1 -- 172.21.15.16:0/2870900591 shutdown_connections 2024-03-27T06:34:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.152+0000 7fc8843e8700 1 --2- 172.21.15.16:0/2870900591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c101160 0x7fc87c101540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.152+0000 7fc8843e8700 1 -- 172.21.15.16:0/2870900591 >> 172.21.15.16:0/2870900591 conn(0x7fc87c0fc9f0 msgr2=0x7fc87c0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:38.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.152+0000 7fc8843e8700 1 -- 172.21.15.16:0/2870900591 shutdown_connections 2024-03-27T06:34:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.152+0000 7fc8843e8700 1 -- 172.21.15.16:0/2870900591 wait complete. 2024-03-27T06:34:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc8843e8700 1 Processor -- start 2024-03-27T06:34:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc8843e8700 1 -- start start 2024-03-27T06:34:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc8843e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c101160 0x7fc87c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc8843e8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c10b670 con 0x7fc87c101160 2024-03-27T06:34:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc882184700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c101160 0x7fc87c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc882184700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c101160 0x7fc87c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37204/0 (socket says 172.21.15.16:37204) 2024-03-27T06:34:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc882184700 1 -- 172.21.15.16:0/2115706142 learned_addr learned my addr 172.21.15.16:0/2115706142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc882184700 1 -- 172.21.15.16:0/2115706142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c198f40 con 0x7fc87c101160 2024-03-27T06:34:38.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc882184700 1 --2- 172.21.15.16:0/2115706142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c101160 0x7fc87c198a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc8740052a0 tx=0x7fc874004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc87b7fe700 1 -- 172.21.15.16:0/2115706142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc874026070 con 0x7fc87c101160 2024-03-27T06:34:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc87b7fe700 1 -- 172.21.15.16:0/2115706142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc87400bbf0 con 0x7fc87c101160 2024-03-27T06:34:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.153+0000 7fc8843e8700 1 -- 172.21.15.16:0/2115706142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c199230 con 0x7fc87c101160 2024-03-27T06:34:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.154+0000 7fc87b7fe700 1 -- 172.21.15.16:0/2115706142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc874003a30 con 0x7fc87c101160 2024-03-27T06:34:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.154+0000 7fc8843e8700 1 -- 172.21.15.16:0/2115706142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c1996f0 con 0x7fc87c101160 2024-03-27T06:34:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.154+0000 7fc87b7fe700 1 -- 172.21.15.16:0/2115706142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7fc874029070 con 0x7fc87c101160 2024-03-27T06:34:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.154+0000 7fc8843e8700 1 -- 172.21.15.16:0/2115706142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc87c102640 con 0x7fc87c101160 2024-03-27T06:34:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.155+0000 7fc87b7fe700 1 --2- 172.21.15.16:0/2115706142 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc864039470 0x7fc86403b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.155+0000 7fc87b7fe700 1 -- 172.21.15.16:0/2115706142 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc87401e070 con 0x7fc87c101160 2024-03-27T06:34:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.155+0000 7fc881983700 1 --2- 172.21.15.16:0/2115706142 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc864039470 0x7fc86403b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:38.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.155+0000 7fc881983700 1 --2- 172.21.15.16:0/2115706142 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc864039470 0x7fc86403b930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc870006fd0 tx=0x7fc870006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:38.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.160+0000 7fc87b7fe700 1 -- 172.21.15.16:0/2115706142 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc874013070 con 0x7fc87c101160 2024-03-27T06:34:38.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:38.371+0000 7fc8843e8700 1 -- 172.21.15.16:0/2115706142 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}) -- 0x7fc87c06a8b0 con 0x7fc864039470 2024-03-27T06:34:38.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:38 smithi016 ceph-mon[31908]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:34:39.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:39 smithi016 ceph-mon[31908]: Deploying cephadm binary to smithi156 2024-03-27T06:34:40.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:40 smithi016 ceph-mon[31908]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:34:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:42.654+0000 7fc87b7fe700 1 -- 172.21.15.16:0/2115706142 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc87c06a8b0 con 0x7fc864039470 2024-03-27T06:34:42.648 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi156' with addr '172.21.15.156' 2024-03-27T06:34:42.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:42.657+0000 7fc8843e8700 1 -- 172.21.15.16:0/2115706142 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc864039470 msgr2=0x7fc86403b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:42.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:42.657+0000 7fc8843e8700 1 --2- 172.21.15.16:0/2115706142 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc864039470 0x7fc86403b930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc870006fd0 tx=0x7fc870006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:42.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:42.657+0000 7fc8843e8700 1 -- 172.21.15.16:0/2115706142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c101160 msgr2=0x7fc87c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:42.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:42.657+0000 7fc8843e8700 1 --2- 172.21.15.16:0/2115706142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c101160 0x7fc87c198a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc8740052a0 tx=0x7fc874004cc0 comp rx=0 tx=0).stop 2024-03-27T06:34:42.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:42.657+0000 7fc8843e8700 1 -- 172.21.15.16:0/2115706142 shutdown_connections 2024-03-27T06:34:42.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:42.657+0000 7fc8843e8700 1 --2- 172.21.15.16:0/2115706142 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc864039470 0x7fc86403b930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:42.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:42.657+0000 7fc8843e8700 1 --2- 172.21.15.16:0/2115706142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c101160 0x7fc87c198a00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:42.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:42.657+0000 7fc8843e8700 1 -- 172.21.15.16:0/2115706142 >> 172.21.15.16:0/2115706142 conn(0x7fc87c0fc9f0 msgr2=0x7fc87c103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:42.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:42.657+0000 7fc8843e8700 1 -- 172.21.15.16:0/2115706142 shutdown_connections 2024-03-27T06:34:42.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:42.657+0000 7fc8843e8700 1 -- 172.21.15.16:0/2115706142 wait complete. 2024-03-27T06:34:42.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:42 smithi016 ceph-mon[31908]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:34:43.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-27T06:34:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:34:43.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:43.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:43 smithi016 ceph-mon[31908]: Added host smithi156 2024-03-27T06:34:43.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:34:43.976 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:44.833 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:44 smithi016 ceph-mon[31908]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:34:44.833 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:44 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.050+0000 7fa3ff68a700 1 -- 172.21.15.16:0/1281551340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f8103220 msgr2=0x7fa3f8103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.050+0000 7fa3ff68a700 1 --2- 172.21.15.16:0/1281551340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f8103220 0x7fa3f8103600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800b0a0 tx=0x7fa3e800b3b0 comp rx=0 tx=0).stop 2024-03-27T06:34:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.050+0000 7fa3ff68a700 1 -- 172.21.15.16:0/1281551340 shutdown_connections 2024-03-27T06:34:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.050+0000 7fa3ff68a700 1 --2- 172.21.15.16:0/1281551340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f8103220 0x7fa3f8103600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.050+0000 7fa3ff68a700 1 -- 172.21.15.16:0/1281551340 >> 172.21.15.16:0/1281551340 conn(0x7fa3f80fc9f0 msgr2=0x7fa3f80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.050+0000 7fa3ff68a700 1 -- 172.21.15.16:0/1281551340 shutdown_connections 2024-03-27T06:34:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.050+0000 7fa3ff68a700 1 -- 172.21.15.16:0/1281551340 wait complete. 2024-03-27T06:34:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3ff68a700 1 Processor -- start 2024-03-27T06:34:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3ff68a700 1 -- start start 2024-03-27T06:34:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3ff68a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f8103220 0x7fa3f819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3ff68a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f810c7e0 con 0x7fa3f8103220 2024-03-27T06:34:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3fd426700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f8103220 0x7fa3f819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3fd426700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f8103220 0x7fa3f819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37228/0 (socket says 172.21.15.16:37228) 2024-03-27T06:34:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3fd426700 1 -- 172.21.15.16:0/2195061239 learned_addr learned my addr 172.21.15.16:0/2195061239 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3fd426700 1 -- 172.21.15.16:0/2195061239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f819de20 con 0x7fa3f8103220 2024-03-27T06:34:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3fd426700 1 --2- 172.21.15.16:0/2195061239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f8103220 0x7fa3f819d8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80062a0 tx=0x7fa3e8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3f67fc700 1 -- 172.21.15.16:0/2195061239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3e8017040 con 0x7fa3f8103220 2024-03-27T06:34:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3f67fc700 1 -- 172.21.15.16:0/2195061239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa3e8022070 con 0x7fa3f8103220 2024-03-27T06:34:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.051+0000 7fa3ff68a700 1 -- 172.21.15.16:0/2195061239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f819e0b0 con 0x7fa3f8103220 2024-03-27T06:34:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.052+0000 7fa3ff68a700 1 -- 172.21.15.16:0/2195061239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f8197aa0 con 0x7fa3f8103220 2024-03-27T06:34:45.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.052+0000 7fa3f67fc700 1 -- 172.21.15.16:0/2195061239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3e801d450 con 0x7fa3f8103220 2024-03-27T06:34:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.052+0000 7fa3f67fc700 1 -- 172.21.15.16:0/2195061239 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7fa3e801d750 con 0x7fa3f8103220 2024-03-27T06:34:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.052+0000 7fa3ff68a700 1 -- 172.21.15.16:0/2195061239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f806ab40 con 0x7fa3f8103220 2024-03-27T06:34:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.053+0000 7fa3f67fc700 1 --2- 172.21.15.16:0/2195061239 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3e0039470 0x7fa3e003b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.053+0000 7fa3f67fc700 1 -- 172.21.15.16:0/2195061239 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa3e800f050 con 0x7fa3f8103220 2024-03-27T06:34:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.053+0000 7fa3fcc25700 1 --2- 172.21.15.16:0/2195061239 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3e0039470 0x7fa3e003b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.053+0000 7fa3fcc25700 1 --2- 172.21.15.16:0/2195061239 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3e0039470 0x7fa3e003b930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec006fd0 tx=0x7fa3ec006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.058+0000 7fa3f67fc700 1 -- 172.21.15.16:0/2195061239 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fa3e80093e0 con 0x7fa3f8103220 2024-03-27T06:34:45.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.272+0000 7fa3ff68a700 1 -- 172.21.15.16:0/2195061239 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3f8198fd0 con 0x7fa3e0039470 2024-03-27T06:34:45.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.274+0000 7fa3f67fc700 1 -- 172.21.15.16:0/2195061239 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fa3f8198fd0 con 0x7fa3e0039470 2024-03-27T06:34:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:45.267 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}] 2024-03-27T06:34:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.276+0000 7fa3ff68a700 1 -- 172.21.15.16:0/2195061239 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3e0039470 msgr2=0x7fa3e003b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.276+0000 7fa3ff68a700 1 --2- 172.21.15.16:0/2195061239 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3e0039470 0x7fa3e003b930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec006fd0 tx=0x7fa3ec006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.276+0000 7fa3ff68a700 1 -- 172.21.15.16:0/2195061239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f8103220 msgr2=0x7fa3f819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.276+0000 7fa3ff68a700 1 --2- 172.21.15.16:0/2195061239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f8103220 0x7fa3f819d8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80062a0 tx=0x7fa3e8003ce0 comp rx=0 tx=0).stop 2024-03-27T06:34:45.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.276+0000 7fa3ff68a700 1 -- 172.21.15.16:0/2195061239 shutdown_connections 2024-03-27T06:34:45.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.276+0000 7fa3ff68a700 1 --2- 172.21.15.16:0/2195061239 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3e0039470 0x7fa3e003b930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:45.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.276+0000 7fa3ff68a700 1 --2- 172.21.15.16:0/2195061239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f8103220 0x7fa3f819d8e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:45.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.276+0000 7fa3ff68a700 1 -- 172.21.15.16:0/2195061239 >> 172.21.15.16:0/2195061239 conn(0x7fa3f80fc9f0 msgr2=0x7fa3f8103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:45.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.276+0000 7fa3ff68a700 1 -- 172.21.15.16:0/2195061239 shutdown_connections 2024-03-27T06:34:45.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:45.276+0000 7fa3ff68a700 1 -- 172.21.15.16:0/2195061239 wait complete. 2024-03-27T06:34:45.872 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2024-03-27T06:34:45.872 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-27T06:34:45.872 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2024-03-27T06:34:45.890 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-27T06:34:45.890 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:34:45.948 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2024-03-27T06:34:45.948 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch host add smithi170 2024-03-27T06:34:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:34:46.539 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:46 smithi016 ceph-mon[31908]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:34:46.539 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:46 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:47.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:47 smithi016 ceph-mon[31908]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:34:47.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.639+0000 7fb5aad02700 1 -- 172.21.15.16:0/3093091077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a4101180 msgr2=0x7fb5a4101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.639+0000 7fb5aad02700 1 --2- 172.21.15.16:0/3093091077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a4101180 0x7fb5a4101560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb598009910 tx=0x7fb598009c20 comp rx=0 tx=0).stop 2024-03-27T06:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.639+0000 7fb5aad02700 1 -- 172.21.15.16:0/3093091077 shutdown_connections 2024-03-27T06:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.639+0000 7fb5aad02700 1 --2- 172.21.15.16:0/3093091077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a4101180 0x7fb5a4101560 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.639+0000 7fb5aad02700 1 -- 172.21.15.16:0/3093091077 >> 172.21.15.16:0/3093091077 conn(0x7fb5a40fca10 msgr2=0x7fb5a40fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.639+0000 7fb5aad02700 1 -- 172.21.15.16:0/3093091077 shutdown_connections 2024-03-27T06:34:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.639+0000 7fb5aad02700 1 -- 172.21.15.16:0/3093091077 wait complete. 2024-03-27T06:34:47.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.640+0000 7fb5aad02700 1 Processor -- start 2024-03-27T06:34:47.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.640+0000 7fb5aad02700 1 -- start start 2024-03-27T06:34:47.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.640+0000 7fb5aad02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a4101180 0x7fb5a419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:47.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.640+0000 7fb5aad02700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a410c7b0 con 0x7fb5a4101180 2024-03-27T06:34:47.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.640+0000 7fb5a8a9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a4101180 0x7fb5a419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:47.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.640+0000 7fb5a8a9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a4101180 0x7fb5a419d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54064/0 (socket says 172.21.15.16:54064) 2024-03-27T06:34:47.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.640+0000 7fb5a8a9e700 1 -- 172.21.15.16:0/1702409167 learned_addr learned my addr 172.21.15.16:0/1702409167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:47.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.640+0000 7fb5a8a9e700 1 -- 172.21.15.16:0/1702409167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a419dd70 con 0x7fb5a4101180 2024-03-27T06:34:47.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.641+0000 7fb5a8a9e700 1 --2- 172.21.15.16:0/1702409167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a4101180 0x7fb5a419d830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb5980052a0 tx=0x7fb598004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:47.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.641+0000 7fb5a1ffb700 1 -- 172.21.15.16:0/1702409167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb598026070 con 0x7fb5a4101180 2024-03-27T06:34:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.641+0000 7fb5a1ffb700 1 -- 172.21.15.16:0/1702409167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb59800bbf0 con 0x7fb5a4101180 2024-03-27T06:34:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.641+0000 7fb5aad02700 1 -- 172.21.15.16:0/1702409167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a419e060 con 0x7fb5a4101180 2024-03-27T06:34:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.641+0000 7fb5a1ffb700 1 -- 172.21.15.16:0/1702409167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb598003a30 con 0x7fb5a4101180 2024-03-27T06:34:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.641+0000 7fb5aad02700 1 -- 172.21.15.16:0/1702409167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a41979f0 con 0x7fb5a4101180 2024-03-27T06:34:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.642+0000 7fb5a1ffb700 1 -- 172.21.15.16:0/1702409167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7fb598029070 con 0x7fb5a4101180 2024-03-27T06:34:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.642+0000 7fb5aad02700 1 -- 172.21.15.16:0/1702409167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a4102660 con 0x7fb5a4101180 2024-03-27T06:34:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.642+0000 7fb5a1ffb700 1 --2- 172.21.15.16:0/1702409167 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb5940394d0 0x7fb59403b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.642+0000 7fb5a1ffb700 1 -- 172.21.15.16:0/1702409167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb59801e070 con 0x7fb5a4101180 2024-03-27T06:34:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.642+0000 7fb5a3fff700 1 --2- 172.21.15.16:0/1702409167 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb5940394d0 0x7fb59403b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.643+0000 7fb5a3fff700 1 --2- 172.21.15.16:0/1702409167 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb5940394d0 0x7fb59403b990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb590006fd0 tx=0x7fb590006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.647+0000 7fb5a1ffb700 1 -- 172.21.15.16:0/1702409167 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb598013070 con 0x7fb5a4101180 2024-03-27T06:34:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:47.856+0000 7fb5aad02700 1 -- 172.21.15.16:0/1702409167 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}) -- 0x7fb5a4198f90 con 0x7fb5940394d0 2024-03-27T06:34:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:49 smithi016 ceph-mon[31908]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:34:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:49 smithi016 ceph-mon[31908]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:34:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:34:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:34:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:34:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:49 smithi016 ceph-mon[31908]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-27T06:34:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:50 smithi016 ceph-mon[31908]: Deploying cephadm binary to smithi170 2024-03-27T06:34:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:50 smithi016 ceph-mon[31908]: Updating smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:34:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:50 smithi016 ceph-mon[31908]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:34:51.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:51 smithi016 ceph-mon[31908]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:34:52.204 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi170' with addr '172.21.15.170' 2024-03-27T06:34:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:52.210+0000 7fb5a1ffb700 1 -- 172.21.15.16:0/1702409167 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb5a4198f90 con 0x7fb5940394d0 2024-03-27T06:34:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:52.213+0000 7fb5aad02700 1 -- 172.21.15.16:0/1702409167 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb5940394d0 msgr2=0x7fb59403b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:52.213+0000 7fb5aad02700 1 --2- 172.21.15.16:0/1702409167 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb5940394d0 0x7fb59403b990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb590006fd0 tx=0x7fb590006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:52.213+0000 7fb5aad02700 1 -- 172.21.15.16:0/1702409167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a4101180 msgr2=0x7fb5a419d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:52.213+0000 7fb5aad02700 1 --2- 172.21.15.16:0/1702409167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a4101180 0x7fb5a419d830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb5980052a0 tx=0x7fb598004cc0 comp rx=0 tx=0).stop 2024-03-27T06:34:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:52.213+0000 7fb5aad02700 1 -- 172.21.15.16:0/1702409167 shutdown_connections 2024-03-27T06:34:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:52.213+0000 7fb5aad02700 1 --2- 172.21.15.16:0/1702409167 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb5940394d0 0x7fb59403b990 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:52.213+0000 7fb5aad02700 1 --2- 172.21.15.16:0/1702409167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a4101180 0x7fb5a419d830 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:52.213+0000 7fb5aad02700 1 -- 172.21.15.16:0/1702409167 >> 172.21.15.16:0/1702409167 conn(0x7fb5a40fca10 msgr2=0x7fb5a4102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:52.213+0000 7fb5aad02700 1 -- 172.21.15.16:0/1702409167 shutdown_connections 2024-03-27T06:34:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:52.213+0000 7fb5aad02700 1 -- 172.21.15.16:0/1702409167 wait complete. 2024-03-27T06:34:52.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:52 smithi016 ceph-mon[31908]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:34:52.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:52 smithi016 ceph-mon[31908]: Updating smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:34:52.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:52 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:52 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:52 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:52 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:52 smithi016 ceph-mon[31908]: Added host smithi170 2024-03-27T06:34:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:52 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:34:52.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-27T06:34:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:34:53.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:53 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:53.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:53 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.753+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/282290116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102290 msgr2=0x7ff4f4102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.753+0000 7ff4fbaa6700 1 --2- 172.21.15.16:0/282290116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102290 0x7ff4f4102670 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009910 tx=0x7ff4ec009c20 comp rx=0 tx=0).stop 2024-03-27T06:34:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.753+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/282290116 shutdown_connections 2024-03-27T06:34:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.753+0000 7ff4fbaa6700 1 --2- 172.21.15.16:0/282290116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102290 0x7ff4f4102670 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.753+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/282290116 >> 172.21.15.16:0/282290116 conn(0x7ff4f40fdb40 msgr2=0x7ff4f40fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.753+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/282290116 shutdown_connections 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.753+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/282290116 wait complete. 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.754+0000 7ff4fbaa6700 1 Processor -- start 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.754+0000 7ff4fbaa6700 1 -- start start 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.754+0000 7ff4fbaa6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102290 0x7ff4f4074940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.754+0000 7ff4fbaa6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410c7a0 con 0x7ff4f4102290 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.754+0000 7ff4f9842700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102290 0x7ff4f4074940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.754+0000 7ff4f9842700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102290 0x7ff4f4074940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54082/0 (socket says 172.21.15.16:54082) 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.754+0000 7ff4f9842700 1 -- 172.21.15.16:0/541606992 learned_addr learned my addr 172.21.15.16:0/541606992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.754+0000 7ff4f9842700 1 -- 172.21.15.16:0/541606992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f4076830 con 0x7ff4f4102290 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.754+0000 7ff4f9842700 1 --2- 172.21.15.16:0/541606992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102290 0x7ff4f4074940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0052a0 tx=0x7ff4ec004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.755+0000 7ff4f2ffd700 1 -- 172.21.15.16:0/541606992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4ec026070 con 0x7ff4f4102290 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.755+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/541606992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f4074ee0 con 0x7ff4f4102290 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.755+0000 7ff4f2ffd700 1 -- 172.21.15.16:0/541606992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff4ec00bc50 con 0x7ff4f4102290 2024-03-27T06:34:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.755+0000 7ff4f2ffd700 1 -- 172.21.15.16:0/541606992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4ec003a20 con 0x7ff4f4102290 2024-03-27T06:34:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.755+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/541606992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f40753a0 con 0x7ff4f4102290 2024-03-27T06:34:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.755+0000 7ff4f2ffd700 1 -- 172.21.15.16:0/541606992 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7ff4ec029070 con 0x7ff4f4102290 2024-03-27T06:34:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.756+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/541606992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f4103770 con 0x7ff4f4102290 2024-03-27T06:34:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.756+0000 7ff4f2ffd700 1 --2- 172.21.15.16:0/541606992 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4dc039470 0x7ff4dc03b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.756+0000 7ff4f2ffd700 1 -- 172.21.15.16:0/541606992 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff4ec01e070 con 0x7ff4f4102290 2024-03-27T06:34:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.756+0000 7ff4f9041700 1 --2- 172.21.15.16:0/541606992 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4dc039470 0x7ff4dc03b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:54.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.756+0000 7ff4f9041700 1 --2- 172.21.15.16:0/541606992 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4dc039470 0x7ff4dc03b930 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8006fd0 tx=0x7ff4e8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.761+0000 7ff4f2ffd700 1 -- 172.21.15.16:0/541606992 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff4ec013070 con 0x7ff4f4102290 2024-03-27T06:34:54.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.974+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/541606992 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4f4075e70 con 0x7ff4dc039470 2024-03-27T06:34:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.975+0000 7ff4f2ffd700 1 -- 172.21.15.16:0/541606992 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7ff4f4075e70 con 0x7ff4dc039470 2024-03-27T06:34:54.969 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:34:54.969 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2024-03-27T06:34:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.977+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/541606992 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4dc039470 msgr2=0x7ff4dc03b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.977+0000 7ff4fbaa6700 1 --2- 172.21.15.16:0/541606992 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4dc039470 0x7ff4dc03b930 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8006fd0 tx=0x7ff4e8006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.977+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/541606992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102290 msgr2=0x7ff4f4074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.977+0000 7ff4fbaa6700 1 --2- 172.21.15.16:0/541606992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102290 0x7ff4f4074940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0052a0 tx=0x7ff4ec004cc0 comp rx=0 tx=0).stop 2024-03-27T06:34:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.977+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/541606992 shutdown_connections 2024-03-27T06:34:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.977+0000 7ff4fbaa6700 1 --2- 172.21.15.16:0/541606992 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4dc039470 0x7ff4dc03b930 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.977+0000 7ff4fbaa6700 1 --2- 172.21.15.16:0/541606992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f4102290 0x7ff4f4074940 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.978+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/541606992 >> 172.21.15.16:0/541606992 conn(0x7ff4f40fdb40 msgr2=0x7ff4f4104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.978+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/541606992 shutdown_connections 2024-03-27T06:34:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:54.978+0000 7ff4fbaa6700 1 -- 172.21.15.16:0/541606992 wait complete. 2024-03-27T06:34:55.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:55 smithi016 ceph-mon[31908]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:34:55.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:55 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:55.532 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-27T06:34:55.532 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-27T06:34:55.771 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:34:56.600 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:56 smithi016 ceph-mon[31908]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:34:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.390+0000 7f8403d66700 1 -- 172.21.15.16:0/3955400472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83fc1032a0 msgr2=0x7f83fc103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.390+0000 7f8403d66700 1 --2- 172.21.15.16:0/3955400472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83fc1032a0 0x7f83fc103680 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009910 tx=0x7f83f4009c20 comp rx=0 tx=0).stop 2024-03-27T06:34:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.390+0000 7f8403d66700 1 -- 172.21.15.16:0/3955400472 shutdown_connections 2024-03-27T06:34:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.390+0000 7f8403d66700 1 --2- 172.21.15.16:0/3955400472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83fc1032a0 0x7f83fc103680 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.390+0000 7f8403d66700 1 -- 172.21.15.16:0/3955400472 >> 172.21.15.16:0/3955400472 conn(0x7f83fc0fca10 msgr2=0x7f83fc0fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.390+0000 7f8403d66700 1 -- 172.21.15.16:0/3955400472 shutdown_connections 2024-03-27T06:34:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.390+0000 7f8403d66700 1 -- 172.21.15.16:0/3955400472 wait complete. 2024-03-27T06:34:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.390+0000 7f8403d66700 1 Processor -- start 2024-03-27T06:34:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.390+0000 7f8403d66700 1 -- start start 2024-03-27T06:34:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.391+0000 7f8403d66700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83fc1032a0 0x7f83fc100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.391+0000 7f8403d66700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc10c800 con 0x7f83fc1032a0 2024-03-27T06:34:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.391+0000 7f8401b02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83fc1032a0 0x7f83fc100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.391+0000 7f8401b02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83fc1032a0 0x7f83fc100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32844/0 (socket says 172.21.15.16:32844) 2024-03-27T06:34:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.391+0000 7f8401b02700 1 -- 172.21.15.16:0/2410512778 learned_addr learned my addr 172.21.15.16:0/2410512778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:34:57.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.391+0000 7f8401b02700 1 -- 172.21.15.16:0/2410512778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc102df0 con 0x7f83fc1032a0 2024-03-27T06:34:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.391+0000 7f8401b02700 1 --2- 172.21.15.16:0/2410512778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83fc1032a0 0x7f83fc100f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f83f4004c10 tx=0x7f83f4004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.391+0000 7f83faffd700 1 -- 172.21.15.16:0/2410512778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83f4026070 con 0x7f83fc1032a0 2024-03-27T06:34:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.391+0000 7f8403d66700 1 -- 172.21.15.16:0/2410512778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc101440 con 0x7f83fc1032a0 2024-03-27T06:34:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.391+0000 7f83faffd700 1 -- 172.21.15.16:0/2410512778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f83f4004370 con 0x7f83fc1032a0 2024-03-27T06:34:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.391+0000 7f83faffd700 1 -- 172.21.15.16:0/2410512778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83f4003bf0 con 0x7f83fc1032a0 2024-03-27T06:34:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.392+0000 7f8403d66700 1 -- 172.21.15.16:0/2410512778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc101900 con 0x7f83fc1032a0 2024-03-27T06:34:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.392+0000 7f83faffd700 1 -- 172.21.15.16:0/2410512778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f83f4029070 con 0x7f83fc1032a0 2024-03-27T06:34:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.392+0000 7f8403d66700 1 -- 172.21.15.16:0/2410512778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83fc06aca0 con 0x7f83fc1032a0 2024-03-27T06:34:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.393+0000 7f83faffd700 1 --2- 172.21.15.16:0/2410512778 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83e40394d0 0x7f83e403b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:34:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.393+0000 7f83faffd700 1 -- 172.21.15.16:0/2410512778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f83f401e070 con 0x7f83fc1032a0 2024-03-27T06:34:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.393+0000 7f8401301700 1 --2- 172.21.15.16:0/2410512778 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83e40394d0 0x7f83e403b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:34:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.393+0000 7f8401301700 1 --2- 172.21.15.16:0/2410512778 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83e40394d0 0x7f83e403b990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f83f0006fd0 tx=0x7f83f0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:34:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.398+0000 7f83faffd700 1 -- 172.21.15.16:0/2410512778 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f83f4013070 con 0x7f83fc1032a0 2024-03-27T06:34:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:57.611+0000 7f8403d66700 1 -- 172.21.15.16:0/2410512778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f83fc102430 con 0x7f83fc1032a0 2024-03-27T06:34:57.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:57 smithi016 ceph-mon[31908]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:34:58.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:58.284+0000 7f83faffd700 1 -- 172.21.15.16:0/2410512778 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f83f400bce0 con 0x7f83fc1032a0 2024-03-27T06:34:58.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:58.287+0000 7f8403d66700 1 -- 172.21.15.16:0/2410512778 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83e40394d0 msgr2=0x7f83e403b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:58.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:58.287+0000 7f8403d66700 1 --2- 172.21.15.16:0/2410512778 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83e40394d0 0x7f83e403b990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f83f0006fd0 tx=0x7f83f0006e40 comp rx=0 tx=0).stop 2024-03-27T06:34:58.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:58.287+0000 7f8403d66700 1 -- 172.21.15.16:0/2410512778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83fc1032a0 msgr2=0x7f83fc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:34:58.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:58.287+0000 7f8403d66700 1 --2- 172.21.15.16:0/2410512778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83fc1032a0 0x7f83fc100f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f83f4004c10 tx=0x7f83f4004cf0 comp rx=0 tx=0).stop 2024-03-27T06:34:58.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:58.287+0000 7f8403d66700 1 -- 172.21.15.16:0/2410512778 shutdown_connections 2024-03-27T06:34:58.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:58.287+0000 7f8403d66700 1 --2- 172.21.15.16:0/2410512778 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83e40394d0 0x7f83e403b990 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:58.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:58.287+0000 7f8403d66700 1 --2- 172.21.15.16:0/2410512778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83fc1032a0 0x7f83fc100f00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:34:58.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:58.287+0000 7f8403d66700 1 -- 172.21.15.16:0/2410512778 >> 172.21.15.16:0/2410512778 conn(0x7f83fc0fca10 msgr2=0x7f83fc104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:34:58.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:58.287+0000 7f8403d66700 1 -- 172.21.15.16:0/2410512778 shutdown_connections 2024-03-27T06:34:58.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:34:58.287+0000 7f8403d66700 1 -- 172.21.15.16:0/2410512778 wait complete. 2024-03-27T06:34:58.282 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-03-27T06:34:58.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:58 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2410512778' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-27T06:34:58.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:58.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:58.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:58.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:58.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:34:58.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:34:58.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:34:58.969 INFO:tasks.cephadm:Adding mon.a on smithi016 2024-03-27T06:34:58.969 INFO:tasks.cephadm:Adding mon.b on smithi156 2024-03-27T06:34:58.970 INFO:tasks.cephadm:Adding mon.c on smithi170 2024-03-27T06:34:58.970 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi016:172.21.15.16=a;smithi156:172.21.15.156=b;smithi170:172.21.15.170=c' 2024-03-27T06:34:59.196 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:34:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:59 smithi016 ceph-mon[31908]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:34:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:59 smithi016 ceph-mon[31908]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-27T06:34:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:59 smithi016 ceph-mon[31908]: Updating smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:34:59.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:59 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2410512778' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-27T06:34:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:59 smithi016 ceph-mon[31908]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T06:34:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:59 smithi016 ceph-mon[31908]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:34:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:34:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:34:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:00.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:00 smithi016 ceph-mon[31908]: Updating smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:35:00.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.837+0000 7fc7c7986700 1 -- 172.21.15.170:0/2513152931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01066d0 msgr2=0x7fc7c0106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:00.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.837+0000 7fc7c7986700 1 --2- 172.21.15.170:0/2513152931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01066d0 0x7fc7c0106ab0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00aa20 tx=0x7fc7bc00ad30 comp rx=0 tx=0).stop 2024-03-27T06:35:00.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.837+0000 7fc7c7986700 1 -- 172.21.15.170:0/2513152931 shutdown_connections 2024-03-27T06:35:00.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.837+0000 7fc7c7986700 1 --2- 172.21.15.170:0/2513152931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01066d0 0x7fc7c0106ab0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:00.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.837+0000 7fc7c7986700 1 -- 172.21.15.170:0/2513152931 >> 172.21.15.170:0/2513152931 conn(0x7fc7c00fc1c0 msgr2=0x7fc7c00fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:00.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.837+0000 7fc7c7986700 1 -- 172.21.15.170:0/2513152931 shutdown_connections 2024-03-27T06:35:00.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.837+0000 7fc7c7986700 1 -- 172.21.15.170:0/2513152931 wait complete. 2024-03-27T06:35:00.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.838+0000 7fc7c7986700 1 Processor -- start 2024-03-27T06:35:00.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.838+0000 7fc7c7986700 1 -- start start 2024-03-27T06:35:00.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.838+0000 7fc7c7986700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01066d0 0x7fc7c019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:00.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.838+0000 7fc7c7986700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c010a5d0 con 0x7fc7c01066d0 2024-03-27T06:35:00.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.838+0000 7fc7c5722700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01066d0 0x7fc7c019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:00.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.838+0000 7fc7c5722700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01066d0 0x7fc7c019d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.170:49694/0 (socket says 172.21.15.170:49694) 2024-03-27T06:35:00.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.838+0000 7fc7c5722700 1 -- 172.21.15.170:0/4113491098 learned_addr learned my addr 172.21.15.170:0/4113491098 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-27T06:35:00.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.838+0000 7fc7c5722700 1 -- 172.21.15.170:0/4113491098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c019ddc0 con 0x7fc7c01066d0 2024-03-27T06:35:00.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.839+0000 7fc7c5722700 1 --2- 172.21.15.170:0/4113491098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01066d0 0x7fc7c019d880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc005b40 tx=0x7fc7bc004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:00.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.839+0000 7fc7b67fc700 1 -- 172.21.15.170:0/4113491098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7bc0052f0 con 0x7fc7c01066d0 2024-03-27T06:35:00.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.839+0000 7fc7b67fc700 1 -- 172.21.15.170:0/4113491098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc7bc005490 con 0x7fc7c01066d0 2024-03-27T06:35:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.839+0000 7fc7c7986700 1 -- 172.21.15.170:0/4113491098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c019e050 con 0x7fc7c01066d0 2024-03-27T06:35:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.839+0000 7fc7b67fc700 1 -- 172.21.15.170:0/4113491098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7bc00e7d0 con 0x7fc7c01066d0 2024-03-27T06:35:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.839+0000 7fc7c7986700 1 -- 172.21.15.170:0/4113491098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c0197a40 con 0x7fc7c01066d0 2024-03-27T06:35:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.840+0000 7fc7c7986700 1 -- 172.21.15.170:0/4113491098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c0101e10 con 0x7fc7c01066d0 2024-03-27T06:35:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.840+0000 7fc7b67fc700 1 -- 172.21.15.170:0/4113491098 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7fc7bc016b00 con 0x7fc7c01066d0 2024-03-27T06:35:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.840+0000 7fc7b67fc700 1 --2- 172.21.15.170:0/4113491098 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7ac039470 0x7fc7ac03b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.840+0000 7fc7b67fc700 1 -- 172.21.15.170:0/4113491098 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc7bc01e070 con 0x7fc7c01066d0 2024-03-27T06:35:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.840+0000 7fc7c4f21700 1 --2- 172.21.15.170:0/4113491098 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7ac039470 0x7fc7ac03b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:00.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.841+0000 7fc7c4f21700 1 --2- 172.21.15.170:0/4113491098 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7ac039470 0x7fc7ac03b930 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0006fd0 tx=0x7fc7b0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:00.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:00.846+0000 7fc7b67fc700 1 -- 172.21.15.170:0/4113491098 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc7bc014b80 con 0x7fc7c01066d0 2024-03-27T06:35:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.053+0000 7fc7c7986700 1 -- 172.21.15.170:0/4113491098 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi156:172.21.15.156=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}) -- 0x7fc7c0198f70 con 0x7fc7ac039470 2024-03-27T06:35:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.061+0000 7fc7b67fc700 1 -- 172.21.15.170:0/4113491098 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc7c0198f70 con 0x7fc7ac039470 2024-03-27T06:35:01.054 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mon update... 2024-03-27T06:35:01.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.062+0000 7fc7c7986700 1 -- 172.21.15.170:0/4113491098 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7ac039470 msgr2=0x7fc7ac03b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:01.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.062+0000 7fc7c7986700 1 --2- 172.21.15.170:0/4113491098 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7ac039470 0x7fc7ac03b930 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0006fd0 tx=0x7fc7b0006e40 comp rx=0 tx=0).stop 2024-03-27T06:35:01.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.062+0000 7fc7c7986700 1 -- 172.21.15.170:0/4113491098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01066d0 msgr2=0x7fc7c019d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:01.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.062+0000 7fc7c7986700 1 --2- 172.21.15.170:0/4113491098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01066d0 0x7fc7c019d880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc005b40 tx=0x7fc7bc004da0 comp rx=0 tx=0).stop 2024-03-27T06:35:01.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.062+0000 7fc7c7986700 1 -- 172.21.15.170:0/4113491098 shutdown_connections 2024-03-27T06:35:01.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.062+0000 7fc7c7986700 1 --2- 172.21.15.170:0/4113491098 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7ac039470 0x7fc7ac03b930 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:01.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.062+0000 7fc7c7986700 1 --2- 172.21.15.170:0/4113491098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01066d0 0x7fc7c019d880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:01.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.062+0000 7fc7c7986700 1 -- 172.21.15.170:0/4113491098 >> 172.21.15.170:0/4113491098 conn(0x7fc7c00fc1c0 msgr2=0x7fc7c018feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:01.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.062+0000 7fc7c7986700 1 -- 172.21.15.170:0/4113491098 shutdown_connections 2024-03-27T06:35:01.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:01.062+0000 7fc7c7986700 1 -- 172.21.15.170:0/4113491098 wait complete. 2024-03-27T06:35:01.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:01 smithi016 ceph-mon[31908]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:01.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:01.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:01.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T06:35:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:01.635 DEBUG:teuthology.orchestra.run.smithi156:mon.b> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.b.service 2024-03-27T06:35:01.638 DEBUG:teuthology.orchestra.run.smithi170:mon.c> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.c.service 2024-03-27T06:35:01.640 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-27T06:35:01.641 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-27T06:35:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:02 smithi016 ceph-mon[31908]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi156:172.21.15.156=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:35:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:02 smithi016 ceph-mon[31908]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi156:172.21.15.156=b;smithi170:172.21.15.170=c;count:3 2024-03-27T06:35:02.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:02 smithi016 ceph-mon[31908]: Deploying daemon mon.c on smithi170 2024-03-27T06:35:02.777 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:35:03.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:03 smithi016 ceph-mon[31908]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:05.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.577+0000 7f179153e700 1 -- 172.21.15.170:0/2711212510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c101170 msgr2=0x7f178c101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:05.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.577+0000 7f179153e700 1 --2- 172.21.15.170:0/2711212510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c101170 0x7f178c101550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f177c009910 tx=0x7f177c009c20 comp rx=0 tx=0).stop 2024-03-27T06:35:05.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.577+0000 7f179153e700 1 -- 172.21.15.170:0/2711212510 shutdown_connections 2024-03-27T06:35:05.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.577+0000 7f179153e700 1 --2- 172.21.15.170:0/2711212510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c101170 0x7f178c101550 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:05.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.577+0000 7f179153e700 1 -- 172.21.15.170:0/2711212510 >> 172.21.15.170:0/2711212510 conn(0x7f178c0fca00 msgr2=0x7f178c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:05.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.577+0000 7f179153e700 1 -- 172.21.15.170:0/2711212510 shutdown_connections 2024-03-27T06:35:05.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.577+0000 7f179153e700 1 -- 172.21.15.170:0/2711212510 wait complete. 2024-03-27T06:35:05.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.577+0000 7f179153e700 1 Processor -- start 2024-03-27T06:35:05.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.578+0000 7f179153e700 1 -- start start 2024-03-27T06:35:05.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.578+0000 7f179153e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c101170 0x7f178c196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:05.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.578+0000 7f179153e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f178c10b680 con 0x7f178c101170 2024-03-27T06:35:05.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.578+0000 7f178affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c101170 0x7f178c196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:05.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.578+0000 7f178affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c101170 0x7f178c196800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.170:49718/0 (socket says 172.21.15.170:49718) 2024-03-27T06:35:05.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.578+0000 7f178affd700 1 -- 172.21.15.170:0/4060760121 learned_addr learned my addr 172.21.15.170:0/4060760121 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-27T06:35:05.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.578+0000 7f178affd700 1 -- 172.21.15.170:0/4060760121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c06b220 con 0x7f178c101170 2024-03-27T06:35:05.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.578+0000 7f178affd700 1 --2- 172.21.15.170:0/4060760121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c101170 0x7f178c196800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f177c0052a0 tx=0x7f177c004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.579+0000 7f1773fff700 1 -- 172.21.15.170:0/4060760121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c026070 con 0x7f178c101170 2024-03-27T06:35:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.579+0000 7f1773fff700 1 -- 172.21.15.170:0/4060760121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f177c00bbf0 con 0x7f178c101170 2024-03-27T06:35:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.579+0000 7f179153e700 1 -- 172.21.15.170:0/4060760121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178c196d40 con 0x7f178c101170 2024-03-27T06:35:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.579+0000 7f1773fff700 1 -- 172.21.15.170:0/4060760121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c003a30 con 0x7f178c101170 2024-03-27T06:35:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.579+0000 7f179153e700 1 -- 172.21.15.170:0/4060760121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f178c197200 con 0x7f178c101170 2024-03-27T06:35:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.580+0000 7f1773fff700 1 -- 172.21.15.170:0/4060760121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f177c029070 con 0x7f178c101170 2024-03-27T06:35:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.580+0000 7f179153e700 1 -- 172.21.15.170:0/4060760121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178c102650 con 0x7f178c101170 2024-03-27T06:35:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.580+0000 7f1773fff700 1 --2- 172.21.15.170:0/4060760121 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1774039470 0x7f177403b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.580+0000 7f1773fff700 1 -- 172.21.15.170:0/4060760121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f177c01e070 con 0x7f178c101170 2024-03-27T06:35:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.580+0000 7f178a7fc700 1 --2- 172.21.15.170:0/4060760121 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1774039470 0x7f177403b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.580+0000 7f178a7fc700 1 --2- 172.21.15.170:0/4060760121 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1774039470 0x7f177403b930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1780006fd0 tx=0x7f1780006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.586+0000 7f1773fff700 1 -- 172.21.15.170:0/4060760121 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f177c013070 con 0x7f178c101170 2024-03-27T06:35:05.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:05 smithi016 ceph-mon[31908]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:05.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.871+0000 7f179153e700 1 -- 172.21.15.170:0/4060760121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f178c06aa10 con 0x7f178c101170 2024-03-27T06:35:05.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.872+0000 7f1773fff700 1 -- 172.21.15.170:0/4060760121 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f177c030090 con 0x7f178c101170 2024-03-27T06:35:05.866 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:35:05.867 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":1,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","modified":"2024-03-27T06:33:22.632787Z","created":"2024-03-27T06:33:22.632787Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-27T06:35:05.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.874+0000 7f179153e700 1 -- 172.21.15.170:0/4060760121 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1774039470 msgr2=0x7f177403b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:05.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.874+0000 7f179153e700 1 --2- 172.21.15.170:0/4060760121 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1774039470 0x7f177403b930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1780006fd0 tx=0x7f1780006e40 comp rx=0 tx=0).stop 2024-03-27T06:35:05.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.874+0000 7f179153e700 1 -- 172.21.15.170:0/4060760121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c101170 msgr2=0x7f178c196800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:05.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.874+0000 7f179153e700 1 --2- 172.21.15.170:0/4060760121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c101170 0x7f178c196800 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f177c0052a0 tx=0x7f177c004cc0 comp rx=0 tx=0).stop 2024-03-27T06:35:05.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.874+0000 7f179153e700 1 -- 172.21.15.170:0/4060760121 shutdown_connections 2024-03-27T06:35:05.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.874+0000 7f179153e700 1 --2- 172.21.15.170:0/4060760121 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1774039470 0x7f177403b930 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:05.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.874+0000 7f179153e700 1 --2- 172.21.15.170:0/4060760121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c101170 0x7f178c196800 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:05.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.874+0000 7f179153e700 1 -- 172.21.15.170:0/4060760121 >> 172.21.15.170:0/4060760121 conn(0x7f178c0fca00 msgr2=0x7f178c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:05.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.874+0000 7f179153e700 1 -- 172.21.15.170:0/4060760121 shutdown_connections 2024-03-27T06:35:05.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:05.874+0000 7f179153e700 1 -- 172.21.15.170:0/4060760121 wait complete. 2024-03-27T06:35:05.869 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 1 2024-03-27T06:35:07.597 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-27T06:35:07.597 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-27T06:35:07.844 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:35:11.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:11.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:35:11.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:35:11.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: mon.a calling monitor election 2024-03-27T06:35:11.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:35:11.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:11.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: mon.c calling monitor election 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: monmap epoch 2 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: fsid c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: last_changed 2024-03-27T06:35:05.980188+0000 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: created 2024-03-27T06:33:22.632787+0000 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: min_mon_release 19 (squid) 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: election_strategy: 1 2024-03-27T06:35:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-03-27T06:35:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-03-27T06:35:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: fsmap 2024-03-27T06:35:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T06:35:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: mgrmap e13: x(active, since 51s) 2024-03-27T06:35:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: overall HEALTH_OK 2024-03-27T06:35:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T06:35:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:12.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.294+0000 7f00a45be700 1 -- 172.21.15.170:0/1349997425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f009c102380 msgr2=0x7f0088005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:12.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.294+0000 7f00a45be700 1 --2- 172.21.15.170:0/1349997425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f009c102380 0x7f0088005610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0094009bf0 tx=0x7f0094009f00 comp rx=0 tx=0).stop 2024-03-27T06:35:12.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.294+0000 7f00a45be700 1 -- 172.21.15.170:0/1349997425 shutdown_connections 2024-03-27T06:35:12.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.294+0000 7f00a45be700 1 --2- 172.21.15.170:0/1349997425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f009c102380 0x7f0088005610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:12.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.294+0000 7f00a45be700 1 -- 172.21.15.170:0/1349997425 >> 172.21.15.170:0/1349997425 conn(0x7f009c0fdc10 msgr2=0x7f009c100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:12.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.294+0000 7f00a45be700 1 -- 172.21.15.170:0/1349997425 shutdown_connections 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.294+0000 7f00a45be700 1 -- 172.21.15.170:0/1349997425 wait complete. 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.295+0000 7f00a45be700 1 Processor -- start 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.295+0000 7f00a45be700 1 -- start start 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.295+0000 7f00a45be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f009c102380 0x7f009c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.295+0000 7f00a45be700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f009c19df90 0x7f009c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a45be700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f009c10a2f0 con 0x7f009c102380 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a45be700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f009c10a170 con 0x7f009c19df90 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a1b59700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f009c19df90 0x7f009c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a1b59700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f009c19df90 0x7f009c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48458/0 (socket says 172.21.15.170:48458) 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a1b59700 1 -- 172.21.15.170:0/4226101074 learned_addr learned my addr 172.21.15.170:0/4226101074 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a235a700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f009c102380 0x7f009c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:12.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a1b59700 1 -- 172.21.15.170:0/4226101074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f009c19df90 msgr2=0x7f009c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-27T06:35:12.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a1b59700 1 -- 172.21.15.170:0/4226101074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f009c19df90 msgr2=0x7f009c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-27T06:35:12.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a1b59700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f009c19df90 0x7f009c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-27T06:35:12.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a1b59700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f009c19df90 0x7f009c197b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:35:12.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a235a700 1 -- 172.21.15.170:0/4226101074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f009c19df90 msgr2=0x7f009c197b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:35:12.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a235a700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f009c19df90 0x7f009c197b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:12.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a235a700 1 -- 172.21.15.170:0/4226101074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009c19e510 con 0x7f009c102380 2024-03-27T06:35:12.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.296+0000 7f00a235a700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f009c102380 0x7f009c19da50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0094000f80 tx=0x7f00940044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:12.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.297+0000 7f00937fe700 1 -- 172.21.15.170:0/4226101074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f00940174b0 con 0x7f009c102380 2024-03-27T06:35:12.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.297+0000 7f00937fe700 1 -- 172.21.15.170:0/4226101074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0094024030 con 0x7f009c102380 2024-03-27T06:35:12.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.297+0000 7f00a45be700 1 -- 172.21.15.170:0/4226101074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009c1980c0 con 0x7f009c102380 2024-03-27T06:35:12.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.297+0000 7f00937fe700 1 -- 172.21.15.170:0/4226101074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0094020ac0 con 0x7f009c102380 2024-03-27T06:35:12.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.297+0000 7f00a45be700 1 -- 172.21.15.170:0/4226101074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f009c198650 con 0x7f009c102380 2024-03-27T06:35:12.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.298+0000 7f00937fe700 1 -- 172.21.15.170:0/4226101074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f009401e070 con 0x7f009c102380 2024-03-27T06:35:12.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.298+0000 7f00937fe700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0080039260 0x7f008003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:12.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.298+0000 7f00937fe700 1 -- 172.21.15.170:0/4226101074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f009401c070 con 0x7f009c102380 2024-03-27T06:35:12.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.298+0000 7f00a45be700 1 -- 172.21.15.170:0/4226101074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009c103860 con 0x7f009c102380 2024-03-27T06:35:12.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.298+0000 7f00a1b59700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0080039260 0x7f008003b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:12.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.299+0000 7f00a1b59700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0080039260 0x7f008003b720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f008c009180 tx=0x7f008c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:12.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.304+0000 7f00937fe700 1 -- 172.21.15.170:0/4226101074 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f0094025280 con 0x7f009c102380 2024-03-27T06:35:12.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:12 smithi016 ceph-mon[31908]: Deploying daemon mon.b on smithi156 2024-03-27T06:35:12.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:12 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:35:12.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.586+0000 7f00a45be700 1 -- 172.21.15.170:0/4226101074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f009c198fe0 con 0x7f009c102380 2024-03-27T06:35:12.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.587+0000 7f00937fe700 1 -- 172.21.15.170:0/4226101074 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f009401fcb0 con 0x7f009c102380 2024-03-27T06:35:12.582 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:35:12.583 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","modified":"2024-03-27T06:35:05.980188Z","created":"2024-03-27T06:33:22.632787Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-27T06:35:12.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.589+0000 7f00a45be700 1 -- 172.21.15.170:0/4226101074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0080039260 msgr2=0x7f008003b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:12.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.589+0000 7f00a45be700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0080039260 0x7f008003b720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f008c009180 tx=0x7f008c008040 comp rx=0 tx=0).stop 2024-03-27T06:35:12.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.589+0000 7f00a45be700 1 -- 172.21.15.170:0/4226101074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f009c102380 msgr2=0x7f009c19da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:12.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.589+0000 7f00a45be700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f009c102380 0x7f009c19da50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0094000f80 tx=0x7f00940044b0 comp rx=0 tx=0).stop 2024-03-27T06:35:12.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.589+0000 7f00a45be700 1 -- 172.21.15.170:0/4226101074 shutdown_connections 2024-03-27T06:35:12.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.589+0000 7f00a45be700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f009c102380 0x7f009c19da50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:12.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.589+0000 7f00a45be700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0080039260 0x7f008003b720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:12.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.589+0000 7f00a45be700 1 --2- 172.21.15.170:0/4226101074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f009c19df90 0x7f009c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:12.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.589+0000 7f00a45be700 1 -- 172.21.15.170:0/4226101074 >> 172.21.15.170:0/4226101074 conn(0x7f009c0fdc10 msgr2=0x7f009c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:12.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.589+0000 7f00a45be700 1 -- 172.21.15.170:0/4226101074 shutdown_connections 2024-03-27T06:35:12.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:12.589+0000 7f00a45be700 1 -- 172.21.15.170:0/4226101074 wait complete. 2024-03-27T06:35:12.584 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 2 2024-03-27T06:35:13.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:13 smithi170 ceph-mon[36438]: from='client.? 172.21.15.170:0/4226101074' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T06:35:13.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:13 smithi016 ceph-mon[31908]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:13.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:13 smithi016 ceph-mon[31908]: from='client.? 172.21.15.170:0/4226101074' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T06:35:14.209 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-27T06:35:14.210 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-27T06:35:14.452 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:35:15.295 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:15 smithi170 ceph-mon[36438]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:15.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:15 smithi016 ceph-mon[31908]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:15.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:15 smithi156 systemd[1]: Started Ceph mon.b for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:35:20.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.174+0000 7f29be685700 1 -- 172.21.15.170:0/2237327858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a00035c0 msgr2=0x7f29a0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:20.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.174+0000 7f29be685700 1 --2- 172.21.15.170:0/2237327858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a00035c0 0x7f29a0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f29b806a900 tx=0x7f29a800fe60 comp rx=0 tx=0).stop 2024-03-27T06:35:20.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.174+0000 7f29be685700 1 -- 172.21.15.170:0/2237327858 shutdown_connections 2024-03-27T06:35:20.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.174+0000 7f29be685700 1 --2- 172.21.15.170:0/2237327858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a00035c0 0x7f29a0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:20.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.174+0000 7f29be685700 1 -- 172.21.15.170:0/2237327858 >> 172.21.15.170:0/2237327858 conn(0x7f29b80fcae0 msgr2=0x7f29b80fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:20.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.174+0000 7f29be685700 1 -- 172.21.15.170:0/2237327858 shutdown_connections 2024-03-27T06:35:20.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.174+0000 7f29be685700 1 -- 172.21.15.170:0/2237327858 wait complete. 2024-03-27T06:35:20.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.175+0000 7f29be685700 1 Processor -- start 2024-03-27T06:35:20.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.175+0000 7f29be685700 1 -- start start 2024-03-27T06:35:20.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29be685700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29a00035c0 0x7f29b819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:20.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29be685700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29b819a5d0 0x7f29b819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:20.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29be685700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b819eea0 0x7f29b819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:20.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29be685700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810a290 con 0x7f29b819eea0 2024-03-27T06:35:20.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29be685700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810a110 con 0x7f29b819a5d0 2024-03-27T06:35:20.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29be685700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810a410 con 0x7f29a00035c0 2024-03-27T06:35:20.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29a00035c0 0x7f29b819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:20.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29a00035c0 0x7f29b819a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55756/0 (socket says 172.21.15.170:55756) 2024-03-27T06:35:20.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b7fff700 1 -- 172.21.15.170:0/3625714765 learned_addr learned my addr 172.21.15.170:0/3625714765 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-27T06:35:20.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29bcc22700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b819eea0 0x7f29b819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:20.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b7fff700 1 -- 172.21.15.170:0/3625714765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29a00035c0 msgr2=0x7f29b819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-27T06:35:20.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b7fff700 1 -- 172.21.15.170:0/3625714765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29a00035c0 msgr2=0x7f29b819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-27T06:35:20.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b7fff700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29a00035c0 0x7f29b819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-27T06:35:20.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b7fff700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29a00035c0 0x7f29b819a090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:35:20.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b77fe700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29b819a5d0 0x7f29b819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:20.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b77fe700 1 -- 172.21.15.170:0/3625714765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29b819a5d0 msgr2=0x7f29b819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-27T06:35:20.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b77fe700 1 -- 172.21.15.170:0/3625714765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29b819a5d0 msgr2=0x7f29b819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-27T06:35:20.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b77fe700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29b819a5d0 0x7f29b819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-27T06:35:20.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29b77fe700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29b819a5d0 0x7f29b819e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:35:20.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29bcc22700 1 -- 172.21.15.170:0/3625714765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29a00035c0 msgr2=0x7f29b819a090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:35:20.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29bcc22700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29a00035c0 0x7f29b819a090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:20.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29bcc22700 1 -- 172.21.15.170:0/3625714765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29b819a5d0 msgr2=0x7f29b819e960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:35:20.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29bcc22700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29b819a5d0 0x7f29b819e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:20.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.176+0000 7f29bcc22700 1 -- 172.21.15.170:0/3625714765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b81a6c10 con 0x7f29b819eea0 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.177+0000 7f29bcc22700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b819eea0 0x7f29b819f350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29b000bd50 tx=0x7f29b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.177+0000 7f29b57fa700 1 -- 172.21.15.170:0/3625714765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0019070 con 0x7f29b819eea0 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.177+0000 7f29b57fa700 1 -- 172.21.15.170:0/3625714765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f29b000cd10 con 0x7f29b819eea0 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.177+0000 7f29be685700 1 -- 172.21.15.170:0/3625714765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b81a6e40 con 0x7f29b819eea0 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.177+0000 7f29b57fa700 1 -- 172.21.15.170:0/3625714765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0012820 con 0x7f29b819eea0 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.177+0000 7f29be685700 1 -- 172.21.15.170:0/3625714765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b81a72d0 con 0x7f29b819eea0 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.178+0000 7f29b57fa700 1 -- 172.21.15.170:0/3625714765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f29b001c070 con 0x7f29b819eea0 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.178+0000 7f29be685700 1 -- 172.21.15.170:0/3625714765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b8063c70 con 0x7f29b819eea0 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.178+0000 7f29b57fa700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f29a4039900 0x7f29a403bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.178+0000 7f29b57fa700 1 -- 172.21.15.170:0/3625714765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f29b004bff0 con 0x7f29b819eea0 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.178+0000 7f29b7fff700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f29a4039900 0x7f29a403bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:20.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.179+0000 7f29b7fff700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f29a4039900 0x7f29a403bdc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29a800fa10 tx=0x7f29a8005a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:20.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.184+0000 7f29b57fa700 1 -- 172.21.15.170:0/3625714765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f29b0013c30 con 0x7f29b819eea0 2024-03-27T06:35:20.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.475+0000 7f29be685700 1 -- 172.21.15.170:0/3625714765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f29b802def0 con 0x7f29b819eea0 2024-03-27T06:35:20.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: mon.c calling monitor election 2024-03-27T06:35:20.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:35:20.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: mon.a calling monitor election 2024-03-27T06:35:20.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:35:20.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:20.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: monmap epoch 3 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: fsid c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: last_changed 2024-03-27T06:35:15.095895+0000 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: created 2024-03-27T06:33:22.632787+0000 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: min_mon_release 19 (squid) 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: election_strategy: 1 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: 2: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.b 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: fsmap 2024-03-27T06:35:20.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T06:35:20.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: mgrmap e13: x(active, since 60s) 2024-03-27T06:35:20.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: overall HEALTH_OK 2024-03-27T06:35:20.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:20.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:20.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:20.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:20.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.476+0000 7f29b57fa700 1 -- 172.21.15.170:0/3625714765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f29b001d730 con 0x7f29b819eea0 2024-03-27T06:35:20.473 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-27T06:35:20.473 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","modified":"2024-03-27T06:35:15.095895Z","created":"2024-03-27T06:33:22.632787Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-27T06:35:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 -- 172.21.15.170:0/3625714765 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f29a4039900 msgr2=0x7f29a403bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f29a4039900 0x7f29a403bdc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29a800fa10 tx=0x7f29a8005a90 comp rx=0 tx=0).stop 2024-03-27T06:35:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 -- 172.21.15.170:0/3625714765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b819eea0 msgr2=0x7f29b819f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b819eea0 0x7f29b819f350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29b000bd50 tx=0x7f29b000be30 comp rx=0 tx=0).stop 2024-03-27T06:35:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 -- 172.21.15.170:0/3625714765 shutdown_connections 2024-03-27T06:35:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b819eea0 0x7f29b819f350 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f29a4039900 0x7f29a403bdc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29a00035c0 0x7f29b819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 --2- 172.21.15.170:0/3625714765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29b819a5d0 0x7f29b819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 -- 172.21.15.170:0/3625714765 >> 172.21.15.170:0/3625714765 conn(0x7f29b80fcae0 msgr2=0x7f29b80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 -- 172.21.15.170:0/3625714765 shutdown_connections 2024-03-27T06:35:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:20.478+0000 7f29be685700 1 -- 172.21.15.170:0/3625714765 wait complete. 2024-03-27T06:35:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 3 2024-03-27T06:35:20.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: mon.c calling monitor election 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: mon.a calling monitor election 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-27T06:35:20.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: monmap epoch 3 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: fsid c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: last_changed 2024-03-27T06:35:15.095895+0000 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: created 2024-03-27T06:33:22.632787+0000 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: min_mon_release 19 (squid) 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: election_strategy: 1 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: 2: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.b 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: fsmap 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: mgrmap e13: x(active, since 60s) 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: overall HEALTH_OK 2024-03-27T06:35:20.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:20.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:20.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:20.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:21.077 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-27T06:35:21.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-27T06:35:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:35:21.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:21 smithi170 ceph-mon[36438]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:21.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:21 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:21.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:21 smithi170 ceph-mon[36438]: from='client.? 172.21.15.170:0/3625714765' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T06:35:21.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:21 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:21.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:21 smithi016 ceph-mon[31908]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:21.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:21 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:21.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:21 smithi016 ceph-mon[31908]: from='client.? 172.21.15.170:0/3625714765' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T06:35:21.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:21 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:22.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:22.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: mon.c calling monitor election 2024-03-27T06:35:22.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: mon.a calling monitor election 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: monmap epoch 3 2024-03-27T06:35:22.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: fsid c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: last_changed 2024-03-27T06:35:15.095895+0000 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: created 2024-03-27T06:33:22.632787+0000 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: min_mon_release 19 (squid) 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: election_strategy: 1 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: 2: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.b 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: fsmap 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: mgrmap e13: x(active, since 60s) 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: overall HEALTH_OK 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:22.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:22.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:22.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:22.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:22.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='client.? 172.21.15.170:0/3625714765' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-27T06:35:22.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:23.063 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: mon.b calling monitor election 2024-03-27T06:35:23.063 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:23.063 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-27T06:35:23.063 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-27T06:35:23.063 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-27T06:35:23.063 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: mon.b calling monitor election 2024-03-27T06:35:23.063 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: mon.a calling monitor election 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: mon.c calling monitor election 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: monmap epoch 3 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: fsid c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: last_changed 2024-03-27T06:35:15.095895+0000 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: created 2024-03-27T06:33:22.632787+0000 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: min_mon_release 19 (squid) 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: election_strategy: 1 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: 2: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.b 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: fsmap 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T06:35:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: mgrmap e13: x(active, since 62s) 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: overall HEALTH_OK 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: Updating smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: Updating smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: Updating smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T06:35:23.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T06:35:23.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.073+0000 7f80e2747700 1 -- 172.21.15.16:0/1302237216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80dc101160 msgr2=0x7f80dc101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.073+0000 7f80e2747700 1 --2- 172.21.15.16:0/1302237216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80dc101160 0x7f80dc101540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009910 tx=0x7f80cc009c20 comp rx=0 tx=0).stop 2024-03-27T06:35:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.073+0000 7f80e2747700 1 -- 172.21.15.16:0/1302237216 shutdown_connections 2024-03-27T06:35:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.073+0000 7f80e2747700 1 --2- 172.21.15.16:0/1302237216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80dc101160 0x7f80dc101540 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.073+0000 7f80e2747700 1 -- 172.21.15.16:0/1302237216 >> 172.21.15.16:0/1302237216 conn(0x7f80dc0fc9f0 msgr2=0x7f80dc0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.073+0000 7f80e2747700 1 -- 172.21.15.16:0/1302237216 shutdown_connections 2024-03-27T06:35:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.073+0000 7f80e2747700 1 -- 172.21.15.16:0/1302237216 wait complete. 2024-03-27T06:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80e2747700 1 Processor -- start 2024-03-27T06:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80e2747700 1 -- start start 2024-03-27T06:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80e2747700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80dc101160 0x7f80dc197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80e2747700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80dc1983f0 0x7f80dc19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80e2747700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80dc19ccc0 0x7f80dc19d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80e2747700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc10b630 con 0x7f80dc19ccc0 2024-03-27T06:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80e2747700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc10b4b0 con 0x7f80dc101160 2024-03-27T06:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80e2747700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc10b7b0 con 0x7f80dc1983f0 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80e0ce4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80dc19ccc0 0x7f80dc19d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80e0ce4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80dc19ccc0 0x7f80dc19d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50244/0 (socket says 172.21.15.16:50244) 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80e0ce4700 1 -- 172.21.15.16:0/1213098423 learned_addr learned my addr 172.21.15.16:0/1213098423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.074+0000 7f80db7fe700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80dc1983f0 0x7f80dc19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80dbfff700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80dc101160 0x7f80dc197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80e0ce4700 1 -- 172.21.15.16:0/1213098423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80dc1983f0 msgr2=0x7f80dc19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80e0ce4700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80dc1983f0 0x7f80dc19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80e0ce4700 1 -- 172.21.15.16:0/1213098423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80dc101160 msgr2=0x7f80dc197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80e0ce4700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80dc101160 0x7f80dc197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80e0ce4700 1 -- 172.21.15.16:0/1213098423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80dc1a4a30 con 0x7f80dc19ccc0 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80dbfff700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80dc101160 0x7f80dc197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80e0ce4700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80dc19ccc0 0x7f80dc19d170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f80d400ba50 tx=0x7f80d400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80d97fa700 1 -- 172.21.15.16:0/1213098423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4019070 con 0x7f80dc19ccc0 2024-03-27T06:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80e2747700 1 -- 172.21.15.16:0/1213098423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80dc1a4c60 con 0x7f80dc19ccc0 2024-03-27T06:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80d97fa700 1 -- 172.21.15.16:0/1213098423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f80d4014440 con 0x7f80dc19ccc0 2024-03-27T06:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80d97fa700 1 -- 172.21.15.16:0/1213098423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d40075b0 con 0x7f80dc19ccc0 2024-03-27T06:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.075+0000 7f80e2747700 1 -- 172.21.15.16:0/1213098423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80dc1a51a0 con 0x7f80dc19ccc0 2024-03-27T06:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.076+0000 7f80d97fa700 1 -- 172.21.15.16:0/1213098423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f80d401c070 con 0x7f80dc19ccc0 2024-03-27T06:35:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.076+0000 7f80e2747700 1 -- 172.21.15.16:0/1213098423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80dc063c70 con 0x7f80dc19ccc0 2024-03-27T06:35:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.076+0000 7f80d97fa700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f80c4039900 0x7f80c403bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.076+0000 7f80d97fa700 1 -- 172.21.15.16:0/1213098423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f80d404bc50 con 0x7f80dc19ccc0 2024-03-27T06:35:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.076+0000 7f80dbfff700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f80c4039900 0x7f80c403bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.077+0000 7f80dbfff700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f80c4039900 0x7f80c403bdc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009750 tx=0x7f80cc005ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.081+0000 7f80d97fa700 1 -- 172.21.15.16:0/1213098423 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f80d4014a10 con 0x7f80dc19ccc0 2024-03-27T06:35:23.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: mon.b calling monitor election 2024-03-27T06:35:23.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: mon.b calling monitor election 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: mon.a calling monitor election 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: mon.c calling monitor election 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: monmap epoch 3 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: fsid c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: last_changed 2024-03-27T06:35:15.095895+0000 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: created 2024-03-27T06:33:22.632787+0000 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: min_mon_release 19 (squid) 2024-03-27T06:35:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: election_strategy: 1 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: 2: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.b 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: fsmap 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: mgrmap e13: x(active, since 62s) 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: overall HEALTH_OK 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: Updating smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: Updating smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: Updating smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T06:35:23.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T06:35:23.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.292+0000 7f80e2747700 1 -- 172.21.15.16:0/1213098423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f80dc1a5430 con 0x7f80dc19ccc0 2024-03-27T06:35:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.293+0000 7f80d97fa700 1 -- 172.21.15.16:0/1213098423 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f80d400e090 con 0x7f80dc19ccc0 2024-03-27T06:35:23.288 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:35:23.288 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-03-27T06:35:23.288 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:35:23.288 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2024-03-27T06:35:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 -- 172.21.15.16:0/1213098423 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f80c4039900 msgr2=0x7f80c403bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f80c4039900 0x7f80c403bdc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009750 tx=0x7f80cc005ab0 comp rx=0 tx=0).stop 2024-03-27T06:35:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 -- 172.21.15.16:0/1213098423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80dc19ccc0 msgr2=0x7f80dc19d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80dc19ccc0 0x7f80dc19d170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f80d400ba50 tx=0x7f80d400bd60 comp rx=0 tx=0).stop 2024-03-27T06:35:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 -- 172.21.15.16:0/1213098423 shutdown_connections 2024-03-27T06:35:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80dc19ccc0 0x7f80dc19d170 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80dc101160 0x7f80dc197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f80c4039900 0x7f80c403bdc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 --2- 172.21.15.16:0/1213098423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80dc1983f0 0x7f80dc19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 -- 172.21.15.16:0/1213098423 >> 172.21.15.16:0/1213098423 conn(0x7f80dc0fc9f0 msgr2=0x7f80dc06a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 -- 172.21.15.16:0/1213098423 shutdown_connections 2024-03-27T06:35:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:23.295+0000 7f80e2747700 1 -- 172.21.15.16:0/1213098423 wait complete. 2024-03-27T06:35:23.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: mon.b calling monitor election 2024-03-27T06:35:23.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:23.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-27T06:35:23.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-27T06:35:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-27T06:35:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: mon.b calling monitor election 2024-03-27T06:35:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: mon.a calling monitor election 2024-03-27T06:35:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: mon.c calling monitor election 2024-03-27T06:35:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-27T06:35:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: monmap epoch 3 2024-03-27T06:35:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: fsid c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:35:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: last_changed 2024-03-27T06:35:15.095895+0000 2024-03-27T06:35:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: created 2024-03-27T06:33:22.632787+0000 2024-03-27T06:35:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: min_mon_release 19 (squid) 2024-03-27T06:35:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: election_strategy: 1 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: 2: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.b 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: fsmap 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: osdmap e4: 0 total, 0 up, 0 in 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: mgrmap e13: x(active, since 62s) 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: overall HEALTH_OK 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: Updating smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: Updating smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: Updating smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T06:35:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T06:35:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:24.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:23 smithi156 ceph-mon[36261]: Reconfiguring mon.a (unknown last config time)... 2024-03-27T06:35:24.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:23 smithi156 ceph-mon[36261]: Reconfiguring daemon mon.a on smithi016 2024-03-27T06:35:24.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:24.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:23 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1213098423' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:24.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:23 smithi170 ceph-mon[36438]: Reconfiguring mon.a (unknown last config time)... 2024-03-27T06:35:24.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:23 smithi170 ceph-mon[36438]: Reconfiguring daemon mon.a on smithi016 2024-03-27T06:35:24.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:24.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:23 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1213098423' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:24.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:23 smithi016 ceph-mon[31908]: Reconfiguring mon.a (unknown last config time)... 2024-03-27T06:35:24.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:23 smithi016 ceph-mon[31908]: Reconfiguring daemon mon.a on smithi016 2024-03-27T06:35:24.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:35:24.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:23 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1213098423' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:24.787 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-27T06:35:24.787 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:35:24.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-03-27T06:35:24.818 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:35:24.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:35:24.887 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-27T06:35:24.887 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.conf 2024-03-27T06:35:24.918 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-27T06:35:24.918 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:35:24.987 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-27T06:35:24.987 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2024-03-27T06:35:25.021 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-27T06:35:25.021 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:35:25.091 INFO:tasks.cephadm:Adding mgr.x on smithi016 2024-03-27T06:35:25.092 INFO:tasks.cephadm:Adding mgr.y on smithi156 2024-03-27T06:35:25.092 INFO:tasks.cephadm:Adding mgr.z on smithi170 2024-03-27T06:35:25.092 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch apply mgr '3;smithi016=x;smithi156=y;smithi170=z' 2024-03-27T06:35:25.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:24 smithi016 ceph-mon[31908]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:25.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:24 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:25.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:24 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:25.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:24 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T06:35:25.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:24 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T06:35:25.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:24 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:25.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:24 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:25.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:24 smithi170 ceph-mon[36438]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:25.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:24 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:25.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:24 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:25.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:24 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T06:35:25.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:24 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T06:35:25.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:24 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:25.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:24 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:25.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:24 smithi156 ceph-mon[36261]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:25.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:24 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:25.201 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:24 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:25.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:24 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T06:35:25.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:24 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T06:35:25.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:24 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:25.202 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:24 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:25.364 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:35:26.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:25 smithi170 ceph-mon[36438]: Reconfiguring mon.b (monmap changed)... 2024-03-27T06:35:26.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:25 smithi170 ceph-mon[36438]: Reconfiguring daemon mon.b on smithi156 2024-03-27T06:35:26.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:25 smithi156 ceph-mon[36261]: Reconfiguring mon.b (monmap changed)... 2024-03-27T06:35:26.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:25 smithi156 ceph-mon[36261]: Reconfiguring daemon mon.b on smithi156 2024-03-27T06:35:26.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:25 smithi016 ceph-mon[31908]: Reconfiguring mon.b (monmap changed)... 2024-03-27T06:35:26.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:25 smithi016 ceph-mon[31908]: Reconfiguring daemon mon.b on smithi156 2024-03-27T06:35:27.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:26 smithi156 ceph-mon[36261]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:27.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:27.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:27.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:26 smithi156 ceph-mon[36261]: Reconfiguring mon.c (monmap changed)... 2024-03-27T06:35:27.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T06:35:27.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T06:35:27.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:27.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:26 smithi156 ceph-mon[36261]: Reconfiguring daemon mon.c on smithi170 2024-03-27T06:35:27.324 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:26 smithi170 ceph-mon[36438]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:27.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:27.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:27.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:26 smithi170 ceph-mon[36438]: Reconfiguring mon.c (monmap changed)... 2024-03-27T06:35:27.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T06:35:27.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T06:35:27.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:27.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:26 smithi170 ceph-mon[36438]: Reconfiguring daemon mon.c on smithi170 2024-03-27T06:35:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:26 smithi016 ceph-mon[31908]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:27.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:26 smithi016 ceph-mon[31908]: Reconfiguring mon.c (monmap changed)... 2024-03-27T06:35:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-27T06:35:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-27T06:35:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:26 smithi016 ceph-mon[31908]: Reconfiguring daemon mon.c on smithi170 2024-03-27T06:35:27.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.768+0000 7fd20a10a700 1 -- 172.21.15.170:0/3906848040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2041027e0 msgr2=0x7fd204102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:27.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.768+0000 7fd20a10a700 1 --2- 172.21.15.170:0/3906848040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2041027e0 0x7fd204102c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80099c0 tx=0x7fd1f8009cd0 comp rx=0 tx=0).stop 2024-03-27T06:35:27.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.768+0000 7fd20a10a700 1 -- 172.21.15.170:0/3906848040 shutdown_connections 2024-03-27T06:35:27.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.768+0000 7fd20a10a700 1 --2- 172.21.15.170:0/3906848040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204103180 0x7fd204109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:27.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.768+0000 7fd20a10a700 1 --2- 172.21.15.170:0/3906848040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2041015e0 0x7fd204101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:27.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.768+0000 7fd20a10a700 1 --2- 172.21.15.170:0/3906848040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2041027e0 0x7fd204102c40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:27.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.768+0000 7fd20a10a700 1 -- 172.21.15.170:0/3906848040 >> 172.21.15.170:0/3906848040 conn(0x7fd2040fcba0 msgr2=0x7fd2040fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:27.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.768+0000 7fd20a10a700 1 -- 172.21.15.170:0/3906848040 shutdown_connections 2024-03-27T06:35:27.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.768+0000 7fd20a10a700 1 -- 172.21.15.170:0/3906848040 wait complete. 2024-03-27T06:35:27.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.768+0000 7fd20a10a700 1 Processor -- start 2024-03-27T06:35:27.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd20a10a700 1 -- start start 2024-03-27T06:35:27.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd20a10a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2041015e0 0x7fd20406b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:27.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd20a10a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2041027e0 0x7fd20419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:27.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd20a10a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204103180 0x7fd20419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:27.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd20a10a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd20410d230 con 0x7fd2041015e0 2024-03-27T06:35:27.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd20a10a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd20410d0b0 con 0x7fd204103180 2024-03-27T06:35:27.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd20a10a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd20410d3b0 con 0x7fd2041027e0 2024-03-27T06:35:27.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd202ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2041027e0 0x7fd20419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:27.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd202ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2041027e0 0x7fd20419d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55764/0 (socket says 172.21.15.170:55764) 2024-03-27T06:35:27.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd202ffd700 1 -- 172.21.15.170:0/1545276666 learned_addr learned my addr 172.21.15.170:0/1545276666 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-27T06:35:27.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd2037fe700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2041015e0 0x7fd20406b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd203fff700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204103180 0x7fd20419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd202ffd700 1 -- 172.21.15.170:0/1545276666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204103180 msgr2=0x7fd20419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd202ffd700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204103180 0x7fd20419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd202ffd700 1 -- 172.21.15.170:0/1545276666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2041015e0 msgr2=0x7fd20406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd202ffd700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2041015e0 0x7fd20406b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd202ffd700 1 -- 172.21.15.170:0/1545276666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2041a0570 con 0x7fd2041027e0 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.769+0000 7fd202ffd700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2041027e0 0x7fd20419d650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8003930 tx=0x7fd1f800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.770+0000 7fd200ff9700 1 -- 172.21.15.170:0/1545276666 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8009e60 con 0x7fd2041027e0 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.770+0000 7fd200ff9700 1 -- 172.21.15.170:0/1545276666 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd1f800be40 con 0x7fd2041027e0 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.770+0000 7fd20a10a700 1 -- 172.21.15.170:0/1545276666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2041a0800 con 0x7fd2041027e0 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.770+0000 7fd200ff9700 1 -- 172.21.15.170:0/1545276666 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80173f0 con 0x7fd2041027e0 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.770+0000 7fd20a10a700 1 -- 172.21.15.170:0/1545276666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2041a0cb0 con 0x7fd2041027e0 2024-03-27T06:35:27.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.770+0000 7fd200ff9700 1 -- 172.21.15.170:0/1545276666 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7fd1f8017590 con 0x7fd2041027e0 2024-03-27T06:35:27.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.771+0000 7fd200ff9700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1ec0396b0 0x7fd1ec03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:27.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.771+0000 7fd20a10a700 1 -- 172.21.15.170:0/1545276666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd204069300 con 0x7fd2041027e0 2024-03-27T06:35:27.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.771+0000 7fd200ff9700 1 -- 172.21.15.170:0/1545276666 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd1f804d660 con 0x7fd2041027e0 2024-03-27T06:35:27.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.771+0000 7fd2037fe700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1ec0396b0 0x7fd1ec03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:27.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.771+0000 7fd2037fe700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1ec0396b0 0x7fd1ec03bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009680 tx=0x7fd1f4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:27.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.776+0000 7fd200ff9700 1 -- 172.21.15.170:0/1545276666 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd1f8013070 con 0x7fd2041027e0 2024-03-27T06:35:27.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.982+0000 7fd20a10a700 1 -- 172.21.15.170:0/1545276666 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi156=y;smithi170=z", "target": ["mon-mgr", ""]}) -- 0x7fd204062bd0 con 0x7fd1ec0396b0 2024-03-27T06:35:27.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.987+0000 7fd200ff9700 1 -- 172.21.15.170:0/1545276666 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd204062bd0 con 0x7fd1ec0396b0 2024-03-27T06:35:27.981 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mgr update... 2024-03-27T06:35:27.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 -- 172.21.15.170:0/1545276666 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1ec0396b0 msgr2=0x7fd1ec03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:27.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1ec0396b0 0x7fd1ec03bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009680 tx=0x7fd1f4009260 comp rx=0 tx=0).stop 2024-03-27T06:35:27.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 -- 172.21.15.170:0/1545276666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2041027e0 msgr2=0x7fd20419d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:27.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2041027e0 0x7fd20419d650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8003930 tx=0x7fd1f800f690 comp rx=0 tx=0).stop 2024-03-27T06:35:27.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 -- 172.21.15.170:0/1545276666 shutdown_connections 2024-03-27T06:35:27.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd204103180 0x7fd20419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:27.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2041015e0 0x7fd20406b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:27.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1ec0396b0 0x7fd1ec03bb70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:27.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 --2- 172.21.15.170:0/1545276666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2041027e0 0x7fd20419d650 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:27.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 -- 172.21.15.170:0/1545276666 >> 172.21.15.170:0/1545276666 conn(0x7fd2040fcba0 msgr2=0x7fd204105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:27.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 -- 172.21.15.170:0/1545276666 shutdown_connections 2024-03-27T06:35:27.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:35:27.989+0000 7fd20a10a700 1 -- 172.21.15.170:0/1545276666 wait complete. 2024-03-27T06:35:28.491 DEBUG:teuthology.orchestra.run.smithi156:mgr.y> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.y.service 2024-03-27T06:35:28.493 DEBUG:teuthology.orchestra.run.smithi170:mgr.z> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.z.service 2024-03-27T06:35:28.533 INFO:tasks.cephadm:Deploying OSDs... 2024-03-27T06:35:28.534 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:35:28.534 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-03-27T06:35:28.553 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-27T06:35:28.553 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-03-27T06:35:28.611 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-27T06:35:28.612 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:28.612 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-27T06:35:28.612 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:28.612 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:28.612 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-03-27 06:34:32.148697741 +0000 2024-03-27T06:35:28.612 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-03-27 06:29:48.223465505 +0000 2024-03-27T06:35:28.613 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-03-27 06:29:48.223465505 +0000 2024-03-27T06:35:28.613 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-03-27 06:29:48.223465505 +0000 2024-03-27T06:35:28.613 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-27T06:35:28.674 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.674 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.674 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:28.674 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.674 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:28.674 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:28.674 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.674 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T06:35:28.674 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T06:35:28.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T06:35:28.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:28.677 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.677 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:28.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:28.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:28.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T06:35:28.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T06:35:28.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T06:35:28.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:28.680 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-03-27T06:35:28.680 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-03-27T06:35:28.680 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000113528 s, 4.5 MB/s 2024-03-27T06:35:28.682 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-27T06:35:28.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:28.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T06:35:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T06:35:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T06:35:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:28.740 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-03-27T06:35:28.796 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-27T06:35:28.796 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:28.797 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-27T06:35:28.797 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:28.797 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:28.797 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-03-27 06:34:32.184697183 +0000 2024-03-27T06:35:28.797 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-03-27 06:29:48.561460279 +0000 2024-03-27T06:35:28.797 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-03-27 06:29:48.561460279 +0000 2024-03-27T06:35:28.797 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-03-27 06:29:48.561460279 +0000 2024-03-27T06:35:28.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-27T06:35:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-03-27T06:35:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-03-27T06:35:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000104501 s, 4.9 MB/s 2024-03-27T06:35:28.865 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-27T06:35:28.922 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-03-27T06:35:28.978 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-27T06:35:28.978 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:28.979 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-27T06:35:28.979 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:28.979 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:28.979 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-03-27 06:34:32.219696641 +0000 2024-03-27T06:35:28.979 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-03-27 06:29:48.908454913 +0000 2024-03-27T06:35:28.979 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-03-27 06:29:48.908454913 +0000 2024-03-27T06:35:28.979 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-03-27 06:29:48.908454913 +0000 2024-03-27T06:35:28.979 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-27T06:35:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-03-27T06:35:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-03-27T06:35:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000108449 s, 4.7 MB/s 2024-03-27T06:35:29.046 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-27T06:35:29.102 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-03-27T06:35:29.157 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-27T06:35:29.158 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:29.158 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-27T06:35:29.158 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:29.158 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:29.158 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-03-27 06:34:32.253696114 +0000 2024-03-27T06:35:29.158 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-03-27 06:29:49.237449826 +0000 2024-03-27T06:35:29.158 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-03-27 06:29:49.237449826 +0000 2024-03-27T06:35:29.158 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-03-27 06:29:49.237449826 +0000 2024-03-27T06:35:29.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-27T06:35:29.225 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-03-27T06:35:29.226 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-03-27T06:35:29.226 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000123713 s, 4.1 MB/s 2024-03-27T06:35:29.226 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-27T06:35:29.282 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2024-03-27T06:35:29.282 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2024-03-27T06:35:29.301 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-27T06:35:29.301 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2024-03-27T06:35:29.359 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-27T06:35:29.359 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:29.359 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-27T06:35:29.359 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:29.359 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:29.359 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-27 06:35:14.459500221 +0000 2024-03-27T06:35:29.359 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-27 06:30:02.110786690 +0000 2024-03-27T06:35:29.360 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-27 06:30:02.110786690 +0000 2024-03-27T06:35:29.360 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-27 06:30:02.110786690 +0000 2024-03-27T06:35:29.360 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-27T06:35:29.429 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-27T06:35:29.430 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-27T06:35:29.430 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000108798 s, 4.7 MB/s 2024-03-27T06:35:29.431 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-27T06:35:29.489 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2024-03-27T06:35:29.545 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-27T06:35:29.546 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:29.546 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-27T06:35:29.546 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:29.546 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:29.546 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-27 06:35:14.459500221 +0000 2024-03-27T06:35:29.546 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-27 06:30:02.449781759 +0000 2024-03-27T06:35:29.546 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-27 06:30:02.449781759 +0000 2024-03-27T06:35:29.546 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-27 06:30:02.449781759 +0000 2024-03-27T06:35:29.547 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-27T06:35:29.614 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-27T06:35:29.614 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-27T06:35:29.614 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000117199 s, 4.4 MB/s 2024-03-27T06:35:29.615 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-27T06:35:29.672 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2024-03-27T06:35:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:29 smithi170 ceph-mon[36438]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:29 smithi170 ceph-mon[36438]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi156=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:35:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:29 smithi170 ceph-mon[36438]: Saving service mgr spec with placement smithi016=x;smithi156=y;smithi170=z;count:3 2024-03-27T06:35:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:29 smithi170 ceph-mon[36438]: Deploying daemon mgr.z on smithi170 2024-03-27T06:35:29.720 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:29 smithi156 ceph-mon[36261]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:29.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:29 smithi156 ceph-mon[36261]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi156=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:35:29.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:29 smithi156 ceph-mon[36261]: Saving service mgr spec with placement smithi016=x;smithi156=y;smithi170=z;count:3 2024-03-27T06:35:29.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:29 smithi156 ceph-mon[36261]: Deploying daemon mgr.z on smithi170 2024-03-27T06:35:29.728 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-27T06:35:29.728 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:29.728 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-27T06:35:29.728 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:29.728 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:29.728 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-27 06:35:14.460500207 +0000 2024-03-27T06:35:29.728 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-27 06:30:02.770777090 +0000 2024-03-27T06:35:29.728 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-27 06:30:02.770777090 +0000 2024-03-27T06:35:29.728 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-27 06:30:02.770777090 +0000 2024-03-27T06:35:29.728 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-27T06:35:29.795 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-27T06:35:29.795 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-27T06:35:29.795 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000116631 s, 4.4 MB/s 2024-03-27T06:35:29.796 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-27T06:35:29.852 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2024-03-27T06:35:29.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:29 smithi016 ceph-mon[31908]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:29.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:29 smithi016 ceph-mon[31908]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi156=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:35:29.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:29 smithi016 ceph-mon[31908]: Saving service mgr spec with placement smithi016=x;smithi156=y;smithi170=z;count:3 2024-03-27T06:35:29.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:29 smithi016 ceph-mon[31908]: Deploying daemon mgr.z on smithi170 2024-03-27T06:35:29.907 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-27T06:35:29.907 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:29.908 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-27T06:35:29.908 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:29.908 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:29.908 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2024-03-27 06:35:14.460500207 +0000 2024-03-27T06:35:29.908 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2024-03-27 06:30:03.097772334 +0000 2024-03-27T06:35:29.908 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2024-03-27 06:30:03.097772334 +0000 2024-03-27T06:35:29.908 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2024-03-27 06:30:03.097772334 +0000 2024-03-27T06:35:29.908 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-27T06:35:29.973 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2024-03-27T06:35:29.973 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2024-03-27T06:35:29.973 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000115723 s, 4.4 MB/s 2024-03-27T06:35:29.975 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-27T06:35:30.031 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-27T06:35:30.031 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-03-27T06:35:30.047 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-27T06:35:30.048 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-03-27T06:35:30.103 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-27T06:35:30.104 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:30.104 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-03-27T06:35:30.104 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:30.104 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:30.104 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-27 06:35:29.980469273 +0000 2024-03-27T06:35:30.104 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-27 06:30:03.928935177 +0000 2024-03-27T06:35:30.104 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-27 06:30:03.928935177 +0000 2024-03-27T06:35:30.104 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-03-27 06:30:03.928935177 +0000 2024-03-27T06:35:30.104 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-27T06:35:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-27T06:35:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-27T06:35:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.2525e-05 s, 5.5 MB/s 2024-03-27T06:35:30.174 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-27T06:35:30.230 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-03-27T06:35:30.286 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-27T06:35:30.286 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:30.286 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-03-27T06:35:30.286 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:30.286 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:30.286 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-27 06:35:29.980469273 +0000 2024-03-27T06:35:30.286 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-27 06:30:04.242927608 +0000 2024-03-27T06:35:30.287 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-27 06:30:04.242927608 +0000 2024-03-27T06:35:30.287 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-03-27 06:30:04.242927608 +0000 2024-03-27T06:35:30.287 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-27T06:35:30.315 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:30 smithi170 podman[38094]: 2024-03-27 06:35:30.18948452 +0000 UTC m=+0.025901894 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:35:30.316 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:30 smithi170 podman[38094]: 2024-03-27 06:35:30.322133219 +0000 UTC m=+0.158550596 container create 712531754fddc89c071e10a2d79afc25a9ab8e4c466446190da2bbeec26c7621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-27T06:35:30.341 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-27T06:35:30.341 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-27T06:35:30.341 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000107396 s, 4.8 MB/s 2024-03-27T06:35:30.343 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-27T06:35:30.398 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-03-27T06:35:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-27T06:35:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:30.452 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-27T06:35:30.452 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:30.452 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:30.452 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-27 06:35:29.980469273 +0000 2024-03-27T06:35:30.452 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-27 06:30:04.561919917 +0000 2024-03-27T06:35:30.453 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-27 06:30:04.561919917 +0000 2024-03-27T06:35:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-03-27 06:30:04.561919917 +0000 2024-03-27T06:35:30.453 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-27T06:35:30.519 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-27T06:35:30.519 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-27T06:35:30.519 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000114239 s, 4.5 MB/s 2024-03-27T06:35:30.520 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-27T06:35:30.577 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-03-27T06:35:30.593 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:30 smithi170 podman[38094]: 2024-03-27 06:35:30.541511448 +0000 UTC m=+0.377928838 container init 712531754fddc89c071e10a2d79afc25a9ab8e4c466446190da2bbeec26c7621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:35:30.593 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:30 smithi170 podman[38094]: 2024-03-27 06:35:30.544755514 +0000 UTC m=+0.381172898 container start 712531754fddc89c071e10a2d79afc25a9ab8e4c466446190da2bbeec26c7621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-03-27T06:35:30.593 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:30 smithi170 ceph-mgr[38238]: -- 172.21.15.170:0/3768964108 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b7b9a2c5a0 con 0x55b7b9a00800 2024-03-27T06:35:30.607 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-27T06:35:30.608 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-27T06:35:30.608 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-27T06:35:30.608 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:35:30.608 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-03-27T06:35:30.608 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-27 06:35:29.981469249 +0000 2024-03-27T06:35:30.608 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-27 06:30:04.880912227 +0000 2024-03-27T06:35:30.608 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-27 06:30:04.880912227 +0000 2024-03-27T06:35:30.608 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-03-27 06:30:04.880912227 +0000 2024-03-27T06:35:30.608 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-27T06:35:30.679 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-27T06:35:30.679 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-27T06:35:30.679 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000118918 s, 4.3 MB/s 2024-03-27T06:35:30.680 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-27T06:35:30.737 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-03-27T06:35:30.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-27T06:35:30.984 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:35:31.093 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:30 smithi170 bash[38094]: 712531754fddc89c071e10a2d79afc25a9ab8e4c466446190da2bbeec26c7621 2024-03-27T06:35:31.093 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:30 smithi170 systemd[1]: Started Ceph mgr.z for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:35:31.380 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:31.105+0000 7f19402d7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T06:35:31.380 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:31.200+0000 7f19402d7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T06:35:31.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:31 smithi156 ceph-mon[36261]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:31.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:31.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:31.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:31.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T06:35:31.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T06:35:31.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T06:35:31.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:31.839 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-mon[36438]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:31.839 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:31.839 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:31.839 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:31.840 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T06:35:31.840 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T06:35:31.840 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T06:35:31.840 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:31.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:31 smithi016 ceph-mon[31908]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:31.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T06:35:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-27T06:35:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T06:35:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:32.093 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:31.845+0000 7f19402d7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T06:35:32.093 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:31.936+0000 7f19402d7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T06:35:32.093 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:32.076+0000 7f19402d7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T06:35:32.507 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:32 smithi170 ceph-mon[36438]: Deploying daemon mgr.y on smithi156 2024-03-27T06:35:32.508 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:32.170+0000 7f19402d7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T06:35:32.508 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:32.250+0000 7f19402d7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T06:35:32.508 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:32.513+0000 7f19402d7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T06:35:32.510 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:32 smithi156 ceph-mon[36261]: Deploying daemon mgr.y on smithi156 2024-03-27T06:35:32.666 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:32 smithi016 ceph-mon[31908]: Deploying daemon mgr.y on smithi156 2024-03-27T06:35:32.757 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:32.623+0000 7f19402d7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T06:35:32.758 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:32.707+0000 7f19402d7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T06:35:32.891 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:32 smithi156 systemd[1]: Starting Ceph mgr.y for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:35:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-27T06:35:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-27T06:35:33.021 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-03-27T06:35:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-03-27T06:35:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00848761 s, 1.2 GB/s 2024-03-27T06:35:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-03-27T06:35:33.093 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:32.802+0000 7f19402d7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T06:35:33.093 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:32.978+0000 7f19402d7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T06:35:33.184 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:32 smithi156 podman[37630]: 2024-03-27 06:35:32.897331738 +0000 UTC m=+0.113783908 container create 1df63da56546fd9e9c9602442eb08cee2d8a55004ee851151ce498a56bf93236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:35:33.184 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:32 smithi156 podman[37630]: 2024-03-27 06:35:32.801365202 +0000 UTC m=+0.017817362 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:35:33.184 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:33 smithi156 podman[37630]: 2024-03-27 06:35:33.116129394 +0000 UTC m=+0.332581553 container init 1df63da56546fd9e9c9602442eb08cee2d8a55004ee851151ce498a56bf93236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=) 2024-03-27T06:35:33.184 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:33 smithi156 podman[37630]: 2024-03-27 06:35:33.1189518 +0000 UTC m=+0.335403960 container start 1df63da56546fd9e9c9602442eb08cee2d8a55004ee851151ce498a56bf93236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:35:33.184 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:33 smithi156 ceph-mgr[37650]: -- 172.21.15.156:0/627241410 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a58a505a0 con 0x559a58a24c00 2024-03-27T06:35:33.184 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:33 smithi156 bash[37630]: 1df63da56546fd9e9c9602442eb08cee2d8a55004ee851151ce498a56bf93236 2024-03-27T06:35:33.459 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:33 smithi156 systemd[1]: Started Ceph mgr.y for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:35:33.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:33 smithi156 ceph-mon[36261]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:33.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:33 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:33 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:33 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.459 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:33 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:33 smithi016 ceph-mon[31908]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:33.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:33 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:33 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:33 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:33 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-03-27T06:35:33.721 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:33 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:33.705+0000 7ff8abe0e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T06:35:33.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:33 smithi170 ceph-mon[36438]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:33.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:33 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:33 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:33 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:33 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:35:34.221 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:33 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:33.803+0000 7ff8abe0e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T06:35:34.290 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:33 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:33.974+0000 7f19402d7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T06:35:34.290 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:34.119+0000 7f19402d7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T06:35:34.546 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:34 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:34.546 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:34.463+0000 7f19402d7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T06:35:34.586 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:34 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:34.586 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:34.497+0000 7ff8abe0e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T06:35:34.586 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:34.591+0000 7ff8abe0e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T06:35:34.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:34 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:34.837 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:34.741+0000 7ff8abe0e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T06:35:34.837 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:34.843+0000 7ff8abe0e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T06:35:34.842 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:34.552+0000 7f19402d7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T06:35:34.843 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:34.776+0000 7f19402d7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:35:35.192 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:34.929+0000 7ff8abe0e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T06:35:35.192 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:35.198+0000 7ff8abe0e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T06:35:35.240 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:34.961+0000 7f19402d7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T06:35:35.471 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:35.308+0000 7ff8abe0e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T06:35:35.471 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:35.394+0000 7ff8abe0e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T06:35:35.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:35.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T06:35:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T06:35:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:35.501 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:35.245+0000 7f19402d7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:35:35.502 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:35.507+0000 7f19402d7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T06:35:35.502 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:35.502 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.502 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.502 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.502 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.502 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.502 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:35.502 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:35.502 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.503 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T06:35:35.503 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T06:35:35.503 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-27T06:35:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-27T06:35:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.620+0000 7f1166562700 1 -- 172.21.15.16:0/2639356085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1160103910 msgr2=0x7f1160103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.620+0000 7f1166562700 1 --2- 172.21.15.16:0/2639356085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1160103910 0x7f1160103d70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1154009990 tx=0x7f1154009ca0 comp rx=0 tx=0).stop 2024-03-27T06:35:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.620+0000 7f1166562700 1 -- 172.21.15.16:0/2639356085 shutdown_connections 2024-03-27T06:35:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.620+0000 7f1166562700 1 --2- 172.21.15.16:0/2639356085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11601042b0 0x7f116010ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.620+0000 7f1166562700 1 --2- 172.21.15.16:0/2639356085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1160100930 0x7f1160100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.620+0000 7f1166562700 1 --2- 172.21.15.16:0/2639356085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1160103910 0x7f1160103d70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.620+0000 7f1166562700 1 -- 172.21.15.16:0/2639356085 >> 172.21.15.16:0/2639356085 conn(0x7f11600fc2b0 msgr2=0x7f11600fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.620+0000 7f1166562700 1 -- 172.21.15.16:0/2639356085 shutdown_connections 2024-03-27T06:35:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.620+0000 7f1166562700 1 -- 172.21.15.16:0/2639356085 wait complete. 2024-03-27T06:35:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f1166562700 1 Processor -- start 2024-03-27T06:35:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f1166562700 1 -- start start 2024-03-27T06:35:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f1166562700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1160100930 0x7f1160198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f1166562700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160103910 0x7f1160198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f1166562700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11601042b0 0x7f116019ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f1166562700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f116010e390 con 0x7f1160100930 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f1166562700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f116010e210 con 0x7f11601042b0 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f1166562700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f116010e510 con 0x7f1160103910 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f115ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1160100930 0x7f1160198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f115ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1160100930 0x7f1160198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34394/0 (socket says 172.21.15.16:34394) 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f115f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160103910 0x7f1160198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f115ffff700 1 -- 172.21.15.16:0/1513136846 learned_addr learned my addr 172.21.15.16:0/1513136846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.621+0000 7f1164aff700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11601042b0 0x7f116019ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f115f7fe700 1 -- 172.21.15.16:0/1513136846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11601042b0 msgr2=0x7f116019ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f115f7fe700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11601042b0 0x7f116019ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f115f7fe700 1 -- 172.21.15.16:0/1513136846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1160100930 msgr2=0x7f1160198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f115f7fe700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1160100930 0x7f1160198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f115f7fe700 1 -- 172.21.15.16:0/1513136846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11601a05b0 con 0x7f1160103910 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f1164aff700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11601042b0 0x7f116019ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f115ffff700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1160100930 0x7f1160198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:35:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f115f7fe700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160103910 0x7f1160198e90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1154000c00 tx=0x7f115400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f115d7fa700 1 -- 172.21.15.16:0/1513136846 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154026070 con 0x7f1160103910 2024-03-27T06:35:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f115d7fa700 1 -- 172.21.15.16:0/1513136846 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f115400fd70 con 0x7f1160103910 2024-03-27T06:35:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f115d7fa700 1 -- 172.21.15.16:0/1513136846 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154021e60 con 0x7f1160103910 2024-03-27T06:35:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f1166562700 1 -- 172.21.15.16:0/1513136846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11601a0840 con 0x7f1160103910 2024-03-27T06:35:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.622+0000 7f1166562700 1 -- 172.21.15.16:0/1513136846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11601a0cf0 con 0x7f1160103910 2024-03-27T06:35:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.623+0000 7f115d7fa700 1 -- 172.21.15.16:0/1513136846 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) ==== 46522+0+0 (secure 0 0 0) 0x7f1154029070 con 0x7f1160103910 2024-03-27T06:35:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.623+0000 7f1166562700 1 -- 172.21.15.16:0/1513136846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1160069300 con 0x7f1160103910 2024-03-27T06:35:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.623+0000 7f115d7fa700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f11480396b0 0x7f114803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:35:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.623+0000 7f115d7fa700 1 -- 172.21.15.16:0/1513136846 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f115404d430 con 0x7f1160103910 2024-03-27T06:35:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.624+0000 7f115ffff700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f11480396b0 0x7f114803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:35:35.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.624+0000 7f115ffff700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f11480396b0 0x7f114803bb70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1150005950 tx=0x7f1150009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:35:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.628+0000 7f115d7fa700 1 -- 172.21.15.16:0/1513136846 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f1154013070 con 0x7f1160103910 2024-03-27T06:35:35.843 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:35 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:35.742+0000 7f19402d7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T06:35:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:35.851+0000 7f1166562700 1 -- 172.21.15.16:0/1513136846 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1160062b40 con 0x7f11480396b0 2024-03-27T06:35:35.971 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:35.489+0000 7ff8abe0e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T06:35:35.971 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:35.666+0000 7ff8abe0e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T06:35:36.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:36 smithi016 ceph-mon[31908]: Reconfiguring mgr.x (unknown last config time)... 2024-03-27T06:35:36.653 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:36 smithi016 ceph-mon[31908]: Reconfiguring daemon mgr.x on smithi016 2024-03-27T06:35:36.654 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:35:36.654 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:35:36.654 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:36.721 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:36 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:36.638+0000 7ff8abe0e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T06:35:36.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:36 smithi156 ceph-mon[36261]: Reconfiguring mgr.x (unknown last config time)... 2024-03-27T06:35:36.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:36 smithi156 ceph-mon[36261]: Reconfiguring daemon mgr.x on smithi016 2024-03-27T06:35:36.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:36 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:35:36.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:36 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:35:36.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:36 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:36.843 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:36 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:36.540+0000 7f19402d7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T06:35:36.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:36 smithi170 ceph-mon[36438]: Reconfiguring mgr.x (unknown last config time)... 2024-03-27T06:35:36.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:36 smithi170 ceph-mon[36438]: Reconfiguring daemon mgr.x on smithi016 2024-03-27T06:35:36.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:36 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:35:36.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:36 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:35:36.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:36 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:37.102 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:36 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:36.771+0000 7ff8abe0e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T06:35:37.102 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:37.108+0000 7ff8abe0e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T06:35:37.387 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:37.198+0000 7ff8abe0e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T06:35:37.387 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:37.393+0000 7ff8abe0e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:35:37.397 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:37 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:37.118+0000 7f19402d7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T06:35:37.397 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:37 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:37.214+0000 7f19402d7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T06:35:37.543 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:37 smithi016 ceph-mon[31908]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:37.543 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:37 smithi016 ceph-mon[31908]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:35:37.543 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:37 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:37.543 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:37 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:37.543 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:37 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:37.544 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:37 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:37.544 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:37 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:37.544 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:37 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:37.721 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:37.578+0000 7ff8abe0e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T06:35:37.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-mon[36261]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:37.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-mon[36261]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:35:37.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:37.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:37.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:37.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:37.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:37.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:37.843 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:35:37 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:35:37.428+0000 7f19402d7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T06:35:37.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:37 smithi170 ceph-mon[36438]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:37.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:37 smithi170 ceph-mon[36438]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:35:37.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:37 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:37.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:37 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:37.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:37 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:37.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:37 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:37.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:37 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:37.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:37 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:38.142 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:37.862+0000 7ff8abe0e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:35:38.398 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:38 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:38.149+0000 7ff8abe0e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T06:35:38.398 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:38 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:38.386+0000 7ff8abe0e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T06:35:38.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:38.412+0000 7f115d7fa700 1 -- 172.21.15.16:0/1513136846 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 14) ==== 92418+0+0 (secure 0 0 0) 0x7f1154009d10 con 0x7f1160103910 2024-03-27T06:35:38.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:38 smithi156 ceph-mon[36261]: Standby manager daemon z started 2024-03-27T06:35:38.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:38 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-27T06:35:38.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:38 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:35:38.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:38 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-27T06:35:38.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:38 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:35:38.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:38 smithi170 ceph-mon[36438]: Standby manager daemon z started 2024-03-27T06:35:38.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:38 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-27T06:35:38.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:38 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:35:38.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:38 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-27T06:35:38.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:38 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:35:38.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:38 smithi016 ceph-mon[31908]: Standby manager daemon z started 2024-03-27T06:35:38.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:38 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-27T06:35:38.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:38 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:35:38.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:38 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-27T06:35:38.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:38 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.170:0/2409505506' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:35:39.471 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:39 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:39.191+0000 7ff8abe0e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T06:35:39.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:39 smithi156 ceph-mon[36261]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:39.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:39 smithi156 ceph-mon[36261]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-27T06:35:39.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:39 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-27T06:35:39.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:39 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3566490790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e99199df-d2f6-4677-8f3c-b96df2232304"}]: dispatch 2024-03-27T06:35:39.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:39 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3566490790' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e99199df-d2f6-4677-8f3c-b96df2232304"}]': finished 2024-03-27T06:35:39.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:39 smithi156 ceph-mon[36261]: osdmap e5: 1 total, 0 up, 1 in 2024-03-27T06:35:39.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:39 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:39 smithi016 ceph-mon[31908]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:39 smithi016 ceph-mon[31908]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-27T06:35:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:39 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-27T06:35:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:39 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3566490790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e99199df-d2f6-4677-8f3c-b96df2232304"}]: dispatch 2024-03-27T06:35:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:39 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3566490790' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e99199df-d2f6-4677-8f3c-b96df2232304"}]': finished 2024-03-27T06:35:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:39 smithi016 ceph-mon[31908]: osdmap e5: 1 total, 0 up, 1 in 2024-03-27T06:35:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:39 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:39.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:39 smithi170 ceph-mon[36438]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:39.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:39 smithi170 ceph-mon[36438]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-27T06:35:39.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:39 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-27T06:35:39.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:39 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3566490790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e99199df-d2f6-4677-8f3c-b96df2232304"}]: dispatch 2024-03-27T06:35:39.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:39 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3566490790' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e99199df-d2f6-4677-8f3c-b96df2232304"}]': finished 2024-03-27T06:35:39.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:39 smithi170 ceph-mon[36438]: osdmap e5: 1 total, 0 up, 1 in 2024-03-27T06:35:39.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:39 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:40.073 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:39 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:39.765+0000 7ff8abe0e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T06:35:40.073 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:39 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:39.861+0000 7ff8abe0e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T06:35:40.073 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:35:40 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:35:40.079+0000 7ff8abe0e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T06:35:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:40.421+0000 7f115d7fa700 1 -- 172.21.15.16:0/1513136846 <== mon.1 v2:172.21.15.170:3300/0 8 ==== mgrmap(e 15) ==== 138314+0+0 (secure 0 0 0) 0x7f115402a3b0 con 0x7f1160103910 2024-03-27T06:35:40.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:40 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1853031902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:35:40.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:40 smithi156 ceph-mon[36261]: Standby manager daemon y started 2024-03-27T06:35:40.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:40 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-27T06:35:40.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:40 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:35:40.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:40 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-27T06:35:40.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:40 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:35:40.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:40 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1853031902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:35:40.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:40 smithi170 ceph-mon[36438]: Standby manager daemon y started 2024-03-27T06:35:40.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:40 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-27T06:35:40.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:40 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:35:40.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:40 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-27T06:35:40.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:40 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:35:40.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:40 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1853031902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:35:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:40 smithi016 ceph-mon[31908]: Standby manager daemon y started 2024-03-27T06:35:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:40 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-27T06:35:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:40 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:35:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:40 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-27T06:35:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:40 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.156:0/2546714379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:35:41.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:41 smithi156 ceph-mon[36261]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:41.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:41 smithi156 ceph-mon[36261]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-27T06:35:41.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:41 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-27T06:35:41.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:41 smithi170 ceph-mon[36438]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:41.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:41 smithi170 ceph-mon[36438]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-27T06:35:41.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:41 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-27T06:35:41.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:41 smithi016 ceph-mon[31908]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:41.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:41 smithi016 ceph-mon[31908]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-27T06:35:41.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:41 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-27T06:35:43.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:43 smithi016 ceph-mon[31908]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:43.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:43 smithi156 ceph-mon[36261]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:43.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:43 smithi170 ceph-mon[36438]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:45.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:45 smithi156 ceph-mon[36261]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:45.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:45 smithi170 ceph-mon[36438]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:45.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:45 smithi016 ceph-mon[31908]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:46.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:46 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-27T06:35:46.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:46 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:46.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:46 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-27T06:35:46.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:46 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:46.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:46 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-27T06:35:46.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:46 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:47.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:47 smithi016 ceph-mon[31908]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:47.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:47 smithi016 ceph-mon[31908]: Deploying daemon osd.0 on smithi016 2024-03-27T06:35:47.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:47 smithi156 ceph-mon[36261]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:47.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:47 smithi156 ceph-mon[36261]: Deploying daemon osd.0 on smithi016 2024-03-27T06:35:47.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:47 smithi170 ceph-mon[36438]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:47.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:47 smithi170 ceph-mon[36438]: Deploying daemon osd.0 on smithi016 2024-03-27T06:35:49.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:49 smithi016 ceph-mon[31908]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:49.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:49 smithi156 ceph-mon[36261]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:49.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:49 smithi170 ceph-mon[36438]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:51.480 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:51 smithi016 ceph-mon[31908]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:51.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:51 smithi156 ceph-mon[36261]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:51.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:51 smithi170 ceph-mon[36438]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:52.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:52 smithi016 ceph-mon[31908]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:52 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:52 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:52 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:52.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:52 smithi156 ceph-mon[36261]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:52.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:52 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:52.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:52 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:52.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:52 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:53.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:52 smithi170 ceph-mon[36438]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:53.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:52 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:53.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:52 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:53.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:52 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:54.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:54 smithi156 ceph-mon[36261]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:54.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:54 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:54.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:54 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:54.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:54 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:54.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:54 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:54.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:54 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:55.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:54 smithi016 ceph-mon[31908]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:55.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:54 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:55.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:54 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:55.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:54 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:55.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:54 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:55.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:54 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:55.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:54 smithi170 ceph-mon[36438]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:55.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:54 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:55.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:54 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:55.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:54 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:55.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:54 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:55.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:54 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:55.355 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-03-27T06:35:55.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.361+0000 7f115d7fa700 1 -- 172.21.15.16:0/1513136846 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1160062b40 con 0x7f11480396b0 2024-03-27T06:35:55.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.363+0000 7f1166562700 1 -- 172.21.15.16:0/1513136846 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f11480396b0 msgr2=0x7f114803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:55.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.363+0000 7f1166562700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f11480396b0 0x7f114803bb70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1150005950 tx=0x7f1150009260 comp rx=0 tx=0).stop 2024-03-27T06:35:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.364+0000 7f1166562700 1 -- 172.21.15.16:0/1513136846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160103910 msgr2=0x7f1160198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:35:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.364+0000 7f1166562700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160103910 0x7f1160198e90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1154000c00 tx=0x7f115400f690 comp rx=0 tx=0).stop 2024-03-27T06:35:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.364+0000 7f1166562700 1 -- 172.21.15.16:0/1513136846 shutdown_connections 2024-03-27T06:35:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.364+0000 7f1166562700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11601042b0 0x7f116019ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.364+0000 7f1166562700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1160100930 0x7f1160198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.364+0000 7f1166562700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f11480396b0 0x7f114803bb70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.364+0000 7f1166562700 1 --2- 172.21.15.16:0/1513136846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1160103910 0x7f1160198e90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:35:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.364+0000 7f1166562700 1 -- 172.21.15.16:0/1513136846 >> 172.21.15.16:0/1513136846 conn(0x7f11600fc2b0 msgr2=0x7f1160106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:35:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.364+0000 7f1166562700 1 -- 172.21.15.16:0/1513136846 shutdown_connections 2024-03-27T06:35:55.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:35:55.364+0000 7f1166562700 1 -- 172.21.15.16:0/1513136846 wait complete. 2024-03-27T06:35:56.026 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.0.service 2024-03-27T06:35:56.028 INFO:tasks.cephadm:Deploying osd.1 on smithi156 with /dev/vg_nvme/lv_4... 2024-03-27T06:35:56.028 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-27T06:35:56.277 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:35:56.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:56.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:56.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:56.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:56 smithi170 ceph-mon[36438]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-27T06:35:56.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:56.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:56.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:56.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:56 smithi016 ceph-mon[31908]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-27T06:35:56.614 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:56.614 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:56.614 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:35:56.614 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:56 smithi156 ceph-mon[36261]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-27T06:35:57.540 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:57 smithi156 ceph-mon[36261]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:57.540 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:57 smithi156 ceph-mon[36261]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-27T06:35:57.540 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:57 smithi156 ceph-mon[36261]: osdmap e6: 1 total, 0 up, 1 in 2024-03-27T06:35:57.540 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:57 smithi156 ceph-mon[36261]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:35:57.540 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:57 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:57.571 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:57 smithi016 ceph-mon[31908]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:57.571 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:57 smithi016 ceph-mon[31908]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-27T06:35:57.571 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:57 smithi016 ceph-mon[31908]: osdmap e6: 1 total, 0 up, 1 in 2024-03-27T06:35:57.571 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:57 smithi016 ceph-mon[31908]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:35:57.571 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:57 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:57.572 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:35:57 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0[47018]: 2024-03-27T06:35:57.358+0000 7f05d96ed700 -1 osd.0 0 waiting for initial osdmap 2024-03-27T06:35:57.572 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:35:57 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0[47018]: 2024-03-27T06:35:57.361+0000 7f05d54e0700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:35:57.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:57 smithi170 ceph-mon[36438]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:57.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:57 smithi170 ceph-mon[36438]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-27T06:35:57.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:57 smithi170 ceph-mon[36438]: osdmap e6: 1 total, 0 up, 1 in 2024-03-27T06:35:57.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:57 smithi170 ceph-mon[36438]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:35:57.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:57 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:58.073 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-27T06:35:58.073 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-27T06:35:58.074 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-03-27T06:35:58.074 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-03-27T06:35:58.074 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00854379 s, 1.2 GB/s 2024-03-27T06:35:58.074 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-03-27T06:35:58.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:58 smithi016 ceph-mon[31908]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:35:58.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:58 smithi016 ceph-mon[31908]: osdmap e7: 1 total, 0 up, 1 in 2024-03-27T06:35:58.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:58.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:58.640 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:58 smithi156 ceph-mon[36261]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:35:58.640 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:58 smithi156 ceph-mon[36261]: osdmap e7: 1 total, 0 up, 1 in 2024-03-27T06:35:58.640 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:58 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:58.640 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:58 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:58.669 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_4 2024-03-27T06:35:58.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:58 smithi170 ceph-mon[36438]: from='osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:35:58.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:58 smithi170 ceph-mon[36438]: osdmap e7: 1 total, 0 up, 1 in 2024-03-27T06:35:58.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:58 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:58.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:58 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:58.912 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:35:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:35:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:35:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078] boot 2024-03-27T06:35:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: osdmap e8: 1 total, 1 up, 1 in 2024-03-27T06:35:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:59.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:35:59.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:59.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:59.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:59.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:35:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:59.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078] boot 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: osdmap e8: 1 total, 1 up, 1 in 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:59.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:35:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:59.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:35:59.744 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: osd.0 [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078] boot 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: osdmap e8: 1 total, 1 up, 1 in 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:35:59.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:35:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:00.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.507+0000 7f9929b96700 1 -- 172.21.15.156:0/249035365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924102770 msgr2=0x7f9924102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:00.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.507+0000 7f9929b96700 1 --2- 172.21.15.156:0/249035365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924102770 0x7f9924102bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f990c0099c0 tx=0x7f990c009cd0 comp rx=0 tx=0).stop 2024-03-27T06:36:00.504 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.507+0000 7f9929b96700 1 -- 172.21.15.156:0/249035365 shutdown_connections 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.507+0000 7f9929b96700 1 --2- 172.21.15.156:0/249035365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9924103110 0x7f99241099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.507+0000 7f9929b96700 1 --2- 172.21.15.156:0/249035365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9924101570 0x7f9924101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.507+0000 7f9929b96700 1 --2- 172.21.15.156:0/249035365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924102770 0x7f9924102bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.507+0000 7f9929b96700 1 -- 172.21.15.156:0/249035365 >> 172.21.15.156:0/249035365 conn(0x7f99240fcb30 msgr2=0x7f99240fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.507+0000 7f9929b96700 1 -- 172.21.15.156:0/249035365 shutdown_connections 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.507+0000 7f9929b96700 1 -- 172.21.15.156:0/249035365 wait complete. 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.508+0000 7f9929b96700 1 Processor -- start 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.508+0000 7f9929b96700 1 -- start start 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.508+0000 7f9929b96700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101570 0x7f992406b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.508+0000 7f9929b96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9924102770 0x7f992419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.508+0000 7f9929b96700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9924103110 0x7f992419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.508+0000 7f9929b96700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f992410d240 con 0x7f9924102770 2024-03-27T06:36:00.505 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.508+0000 7f9929b96700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f992410d0c0 con 0x7f9924101570 2024-03-27T06:36:00.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.508+0000 7f9929b96700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f992410d3c0 con 0x7f9924103110 2024-03-27T06:36:00.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.508+0000 7f99237fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101570 0x7f992406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:00.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.508+0000 7f99237fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101570 0x7f992406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:60266/0 (socket says 172.21.15.156:60266) 2024-03-27T06:36:00.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.508+0000 7f99237fe700 1 -- 172.21.15.156:0/3549209543 learned_addr learned my addr 172.21.15.156:0/3549209543 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-27T06:36:00.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f9923fff700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9924103110 0x7f992419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:00.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f991affd700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9924102770 0x7f992419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:00.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f99237fe700 1 -- 172.21.15.156:0/3549209543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9924103110 msgr2=0x7f992419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:00.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f99237fe700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9924103110 0x7f992419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:00.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f99237fe700 1 -- 172.21.15.156:0/3549209543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9924102770 msgr2=0x7f992419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:00.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f99237fe700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9924102770 0x7f992419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:00.506 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f99237fe700 1 -- 172.21.15.156:0/3549209543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99241a05b0 con 0x7f9924101570 2024-03-27T06:36:00.507 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f9923fff700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9924103110 0x7f992419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:36:00.507 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f99237fe700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101570 0x7f992406b900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f991400c970 tx=0x7f991400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:36:00.507 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f99217fa700 1 -- 172.21.15.156:0/3549209543 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914004220 con 0x7f9924101570 2024-03-27T06:36:00.507 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f9929b96700 1 -- 172.21.15.156:0/3549209543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99241a08a0 con 0x7f9924101570 2024-03-27T06:36:00.507 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f99217fa700 1 -- 172.21.15.156:0/3549209543 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f99140043c0 con 0x7f9924101570 2024-03-27T06:36:00.507 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f99217fa700 1 -- 172.21.15.156:0/3549209543 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914003d50 con 0x7f9924101570 2024-03-27T06:36:00.507 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.509+0000 7f9929b96700 1 -- 172.21.15.156:0/3549209543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99241a0e00 con 0x7f9924101570 2024-03-27T06:36:00.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.510+0000 7f9929b96700 1 -- 172.21.15.156:0/3549209543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9924069300 con 0x7f9924101570 2024-03-27T06:36:00.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.511+0000 7f99217fa700 1 -- 172.21.15.156:0/3549209543 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) ==== 138314+0+0 (secure 0 0 0) 0x7f99140076d0 con 0x7f9924101570 2024-03-27T06:36:00.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.512+0000 7f99217fa700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99100a3690 0x7f99100a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:00.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.512+0000 7f99217fa700 1 -- 172.21.15.156:0/3549209543 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f99140ce120 con 0x7f9924101570 2024-03-27T06:36:00.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.512+0000 7f991affd700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99100a3690 0x7f99100a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:00.508 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.513+0000 7f991affd700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99100a3690 0x7f99100a5b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f990c000c00 tx=0x7f990c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:36:00.511 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.516+0000 7f99217fa700 1 -- 172.21.15.156:0/3549209543 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f9914014030 con 0x7f9924101570 2024-03-27T06:36:00.717 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:00.722+0000 7f9929b96700 1 -- 172.21.15.156:0/3549209543 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9924062a80 con 0x7f99100a3690 2024-03-27T06:36:00.717 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:00 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi016 2024-03-27T06:36:00.717 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:00 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi016 to 17093M 2024-03-27T06:36:00.717 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:00 smithi156 ceph-mon[36261]: osdmap e9: 1 total, 1 up, 1 in 2024-03-27T06:36:00.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:00 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi016 2024-03-27T06:36:00.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:00 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi016 to 17093M 2024-03-27T06:36:00.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:00 smithi170 ceph-mon[36438]: osdmap e9: 1 total, 1 up, 1 in 2024-03-27T06:36:00.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:00 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi016 2024-03-27T06:36:00.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:00 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi016 to 17093M 2024-03-27T06:36:00.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:00 smithi016 ceph-mon[31908]: osdmap e9: 1 total, 1 up, 1 in 2024-03-27T06:36:01.660 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:01 smithi156 ceph-mon[36261]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:01.660 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:36:01.660 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:36:01.660 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:01.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:01 smithi170 ceph-mon[36438]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:01.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:36:01.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:36:01.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:01.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:01 smithi016 ceph-mon[31908]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:01.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:36:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:36:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:02.581 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:02 smithi156 ceph-mon[36261]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:36:02.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:02 smithi170 ceph-mon[36438]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:36:02.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:02 smithi016 ceph-mon[31908]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:36:03.706 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:03 smithi156 ceph-mon[36261]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:03.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:03 smithi170 ceph-mon[36438]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:03.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:03 smithi016 ceph-mon[31908]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:04.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:04 smithi156 ceph-mon[36261]: from='client.? 172.21.15.156:0/1663882049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "948ae15e-e954-4653-9d1d-da3bac45bd28"}]: dispatch 2024-03-27T06:36:04.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:04 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "948ae15e-e954-4653-9d1d-da3bac45bd28"}]: dispatch 2024-03-27T06:36:04.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:04 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "948ae15e-e954-4653-9d1d-da3bac45bd28"}]': finished 2024-03-27T06:36:04.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:04 smithi156 ceph-mon[36261]: osdmap e10: 2 total, 1 up, 2 in 2024-03-27T06:36:04.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:04 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:04.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:04 smithi156 ceph-mon[36261]: from='client.? 172.21.15.156:0/1288450502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:36:04.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:04 smithi170 ceph-mon[36438]: from='client.? 172.21.15.156:0/1663882049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "948ae15e-e954-4653-9d1d-da3bac45bd28"}]: dispatch 2024-03-27T06:36:04.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:04 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "948ae15e-e954-4653-9d1d-da3bac45bd28"}]: dispatch 2024-03-27T06:36:04.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:04 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "948ae15e-e954-4653-9d1d-da3bac45bd28"}]': finished 2024-03-27T06:36:04.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:04 smithi170 ceph-mon[36438]: osdmap e10: 2 total, 1 up, 2 in 2024-03-27T06:36:04.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:04 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:04.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:04 smithi170 ceph-mon[36438]: from='client.? 172.21.15.156:0/1288450502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:36:04.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:04 smithi016 ceph-mon[31908]: from='client.? 172.21.15.156:0/1663882049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "948ae15e-e954-4653-9d1d-da3bac45bd28"}]: dispatch 2024-03-27T06:36:04.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:04 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "948ae15e-e954-4653-9d1d-da3bac45bd28"}]: dispatch 2024-03-27T06:36:04.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:04 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "948ae15e-e954-4653-9d1d-da3bac45bd28"}]': finished 2024-03-27T06:36:04.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:04 smithi016 ceph-mon[31908]: osdmap e10: 2 total, 1 up, 2 in 2024-03-27T06:36:04.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:04 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:04.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:04 smithi016 ceph-mon[31908]: from='client.? 172.21.15.156:0/1288450502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:36:05.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:05 smithi156 ceph-mon[36261]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:05.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:05 smithi170 ceph-mon[36438]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:05.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:05 smithi016 ceph-mon[31908]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:07.621 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:07 smithi156 ceph-mon[36261]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:07.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:07 smithi170 ceph-mon[36438]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:07.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:07 smithi016 ceph-mon[31908]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:09.456 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:09 smithi156 ceph-mon[36261]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:09.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:09 smithi170 ceph-mon[36438]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:09.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:09 smithi016 ceph-mon[31908]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:11.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:11 smithi156 ceph-mon[36261]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:11.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-27T06:36:11.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:11.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:11 smithi170 ceph-mon[36438]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:11.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-27T06:36:11.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:11.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:11 smithi016 ceph-mon[31908]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:11.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-27T06:36:11.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:12.567 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:12 smithi156 ceph-mon[36261]: Deploying daemon osd.1 on smithi156 2024-03-27T06:36:12.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:12 smithi170 ceph-mon[36438]: Deploying daemon osd.1 on smithi156 2024-03-27T06:36:12.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:12 smithi016 ceph-mon[31908]: Deploying daemon osd.1 on smithi156 2024-03-27T06:36:13.630 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:13 smithi156 ceph-mon[36261]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:13.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:13 smithi170 ceph-mon[36438]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:13.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:13 smithi016 ceph-mon[31908]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:15.527 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:15 smithi156 ceph-mon[36261]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:15.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:15 smithi016 ceph-mon[31908]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:15.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:15 smithi170 ceph-mon[36438]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:16.944 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:16 smithi016 ceph-mon[31908]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:16.944 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:16 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:16.944 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:16 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:16.944 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:16 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:16.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:16 smithi156 ceph-mon[36261]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:16.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:16 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:16.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:16 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:16.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:16 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:17.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:16 smithi170 ceph-mon[36438]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:17.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:16 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:17.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:16 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:17.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:16 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:18.830 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:18 smithi156 ceph-mon[36261]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:18.882 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 1 on host 'smithi156' 2024-03-27T06:36:18.883 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.888+0000 7f99217fa700 1 -- 172.21.15.156:0/3549209543 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9924062a80 con 0x7f99100a3690 2024-03-27T06:36:18.885 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 -- 172.21.15.156:0/3549209543 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99100a3690 msgr2=0x7f99100a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:18.885 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99100a3690 0x7f99100a5b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f990c000c00 tx=0x7f990c005fb0 comp rx=0 tx=0).stop 2024-03-27T06:36:18.885 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 -- 172.21.15.156:0/3549209543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101570 msgr2=0x7f992406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:18.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101570 0x7f992406b900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f991400c970 tx=0x7f991400cc80 comp rx=0 tx=0).stop 2024-03-27T06:36:18.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 -- 172.21.15.156:0/3549209543 shutdown_connections 2024-03-27T06:36:18.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99100a3690 0x7f99100a5b50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:18.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9924103110 0x7f992419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:18.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9924101570 0x7f992406b900 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:18.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 --2- 172.21.15.156:0/3549209543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9924102770 0x7f992419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:18.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 -- 172.21.15.156:0/3549209543 >> 172.21.15.156:0/3549209543 conn(0x7f99240fcb30 msgr2=0x7f9924105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:36:18.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 -- 172.21.15.156:0/3549209543 shutdown_connections 2024-03-27T06:36:18.886 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:36:18.891+0000 7f9929b96700 1 -- 172.21.15.156:0/3549209543 wait complete. 2024-03-27T06:36:19.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:18 smithi170 ceph-mon[36438]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:19.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:18 smithi016 ceph-mon[31908]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:19.537 DEBUG:teuthology.orchestra.run.smithi156:osd.1> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.1.service 2024-03-27T06:36:19.539 INFO:tasks.cephadm:Deploying osd.2 on smithi170 with /dev/vg_nvme/lv_4... 2024-03-27T06:36:19.540 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-27T06:36:19.768 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:36:19 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1[40822]: 2024-03-27T06:36:19.635+0000 7f74a27187c0 -1 osd.1 0 log_to_monitors true 2024-03-27T06:36:19.787 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:36:20.061 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.062 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.062 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.062 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.062 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:19 smithi156 ceph-mon[36261]: from='osd.1 [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T06:36:20.062 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:19 smithi156 ceph-mon[36261]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T06:36:20.235 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.237 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.237 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.237 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.237 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:19 smithi170 ceph-mon[36438]: from='osd.1 [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T06:36:20.238 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:19 smithi170 ceph-mon[36438]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T06:36:20.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:20.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:19 smithi016 ceph-mon[31908]: from='osd.1 [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T06:36:20.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:19 smithi016 ceph-mon[31908]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-27T06:36:21.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:21 smithi170 ceph-mon[36438]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:21.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:21 smithi170 ceph-mon[36438]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-27T06:36:21.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:21 smithi170 ceph-mon[36438]: from='osd.1 [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:36:21.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:21 smithi170 ceph-mon[36438]: osdmap e11: 2 total, 1 up, 2 in 2024-03-27T06:36:21.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:21 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:21.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:21 smithi170 ceph-mon[36438]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:36:21.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:21 smithi156 ceph-mon[36261]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:21.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:21 smithi156 ceph-mon[36261]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-27T06:36:21.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:21 smithi156 ceph-mon[36261]: from='osd.1 [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:36:21.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:21 smithi156 ceph-mon[36261]: osdmap e11: 2 total, 1 up, 2 in 2024-03-27T06:36:21.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:21 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:21.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:21 smithi156 ceph-mon[36261]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:36:21.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:21 smithi016 ceph-mon[31908]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:21.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:21 smithi016 ceph-mon[31908]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-27T06:36:21.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:21 smithi016 ceph-mon[31908]: from='osd.1 [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:36:21.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:21 smithi016 ceph-mon[31908]: osdmap e11: 2 total, 1 up, 2 in 2024-03-27T06:36:21.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:21 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:21.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:21 smithi016 ceph-mon[31908]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:36:21.890 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:36:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1[40822]: 2024-03-27T06:36:21.643+0000 7f749857e700 -1 osd.1 0 waiting for initial osdmap 2024-03-27T06:36:21.891 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:36:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1[40822]: 2024-03-27T06:36:21.646+0000 7f748fb68700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:36:21.988 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-27T06:36:21.988 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-27T06:36:21.988 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-03-27T06:36:21.988 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-03-27T06:36:21.988 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00841206 s, 1.2 GB/s 2024-03-27T06:36:21.988 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-03-27T06:36:22.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:22 smithi170 ceph-mon[36438]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:36:22.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:22 smithi170 ceph-mon[36438]: osdmap e12: 2 total, 1 up, 2 in 2024-03-27T06:36:22.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:22.443 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:22.443 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:22 smithi170 ceph-mon[36438]: from='osd.1 ' entity='osd.1' 2024-03-27T06:36:22.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:22 smithi156 ceph-mon[36261]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:36:22.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:22 smithi156 ceph-mon[36261]: osdmap e12: 2 total, 1 up, 2 in 2024-03-27T06:36:22.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:22.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:22.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:22 smithi156 ceph-mon[36261]: from='osd.1 ' entity='osd.1' 2024-03-27T06:36:22.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:22 smithi016 ceph-mon[31908]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:36:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:22 smithi016 ceph-mon[31908]: osdmap e12: 2 total, 1 up, 2 in 2024-03-27T06:36:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:22 smithi016 ceph-mon[31908]: from='osd.1 ' entity='osd.1' 2024-03-27T06:36:22.617 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi170:vg_nvme/lv_4 2024-03-27T06:36:22.860 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:36:23.246 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:36:23.246 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:36:23.246 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:23.246 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:23.246 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.246 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.246 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:36:23.246 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.246 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:23.246 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:23.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: osd.1 [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995] boot 2024-03-27T06:36:23.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: osdmap e13: 2 total, 2 up, 2 in 2024-03-27T06:36:23.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:23.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:23.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:23.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:23.247 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:23 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:36:23.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: osd.1 [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995] boot 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: osdmap e13: 2 total, 2 up, 2 in 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:23.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:23.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:23 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: osd.1 [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995] boot 2024-03-27T06:36:23.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: osdmap e13: 2 total, 2 up, 2 in 2024-03-27T06:36:23.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:36:23.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:23.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:23.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:23.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:23 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:24.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.371+0000 7f56d50c2700 1 -- 172.21.15.170:0/942198574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56d0101570 msgr2=0x7f56d0101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:24.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.371+0000 7f56d50c2700 1 --2- 172.21.15.170:0/942198574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56d0101570 0x7f56d0101990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009990 tx=0x7f56c0009ca0 comp rx=0 tx=0).stop 2024-03-27T06:36:24.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.371+0000 7f56d50c2700 1 -- 172.21.15.170:0/942198574 shutdown_connections 2024-03-27T06:36:24.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.371+0000 7f56d50c2700 1 --2- 172.21.15.170:0/942198574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56d0103110 0x7f56d01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:24.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.371+0000 7f56d50c2700 1 --2- 172.21.15.170:0/942198574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56d0101570 0x7f56d0101990 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:24.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.371+0000 7f56d50c2700 1 --2- 172.21.15.170:0/942198574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56d0102770 0x7f56d0102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:24.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.371+0000 7f56d50c2700 1 -- 172.21.15.170:0/942198574 >> 172.21.15.170:0/942198574 conn(0x7f56d00fcb30 msgr2=0x7f56d00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:36:24.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.371+0000 7f56d50c2700 1 -- 172.21.15.170:0/942198574 shutdown_connections 2024-03-27T06:36:24.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.371+0000 7f56d50c2700 1 -- 172.21.15.170:0/942198574 wait complete. 2024-03-27T06:36:24.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.372+0000 7f56d50c2700 1 Processor -- start 2024-03-27T06:36:24.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.372+0000 7f56d50c2700 1 -- start start 2024-03-27T06:36:24.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.372+0000 7f56d50c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56d0101570 0x7f56d006b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:24.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.372+0000 7f56d50c2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56d0102770 0x7f56d019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:24.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.372+0000 7f56d50c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56d0103110 0x7f56d019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:24.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56d50c2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56d010d280 con 0x7f56d0101570 2024-03-27T06:36:24.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56d50c2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f56d010d100 con 0x7f56d0102770 2024-03-27T06:36:24.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56d50c2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56d010d400 con 0x7f56d0103110 2024-03-27T06:36:24.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56cf59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56d0103110 0x7f56d019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:24.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56cf59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56d0103110 0x7f56d019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36816/0 (socket says 172.21.15.170:36816) 2024-03-27T06:36:24.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56cf59e700 1 -- 172.21.15.170:0/3843423645 learned_addr learned my addr 172.21.15.170:0/3843423645 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-27T06:36:24.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56ce59c700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56d0102770 0x7f56d019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:24.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56ced9d700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56d0101570 0x7f56d006b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:24.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56cf59e700 1 -- 172.21.15.170:0/3843423645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56d0102770 msgr2=0x7f56d019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:24.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56cf59e700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56d0102770 0x7f56d019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:24.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56cf59e700 1 -- 172.21.15.170:0/3843423645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56d0101570 msgr2=0x7f56d006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:24.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56cf59e700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56d0101570 0x7f56d006b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:24.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56cf59e700 1 -- 172.21.15.170:0/3843423645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d01a0570 con 0x7f56d0103110 2024-03-27T06:36:24.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56cf59e700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56d0103110 0x7f56d019fca0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f56c8007d70 tx=0x7f56c800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:36:24.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56b7fff700 1 -- 172.21.15.170:0/3843423645 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8016020 con 0x7f56d0103110 2024-03-27T06:36:24.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56d50c2700 1 -- 172.21.15.170:0/3843423645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d01a0860 con 0x7f56d0103110 2024-03-27T06:36:24.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56b7fff700 1 -- 172.21.15.170:0/3843423645 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f56c800ac90 con 0x7f56d0103110 2024-03-27T06:36:24.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56b7fff700 1 -- 172.21.15.170:0/3843423645 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8011620 con 0x7f56d0103110 2024-03-27T06:36:24.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.373+0000 7f56d50c2700 1 -- 172.21.15.170:0/3843423645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d01a0d50 con 0x7f56d0103110 2024-03-27T06:36:24.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.374+0000 7f56d50c2700 1 -- 172.21.15.170:0/3843423645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d0069300 con 0x7f56d0103110 2024-03-27T06:36:24.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.375+0000 7f56b7fff700 1 -- 172.21.15.170:0/3843423645 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) ==== 138314+0+0 (secure 0 0 0) 0x7f56c8019020 con 0x7f56d0103110 2024-03-27T06:36:24.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.376+0000 7f56b7fff700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f56b80a3610 0x7f56b80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:24.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.376+0000 7f56b7fff700 1 -- 172.21.15.170:0/3843423645 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f56c80ce750 con 0x7f56d0103110 2024-03-27T06:36:24.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.376+0000 7f56ced9d700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f56b80a3610 0x7f56b80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:24.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.377+0000 7f56ced9d700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f56b80a3610 0x7f56b80a5ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f56c00039a0 tx=0x7f56c0003a70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:36:24.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.380+0000 7f56b7fff700 1 -- 172.21.15.170:0/3843423645 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f56c8020020 con 0x7f56d0103110 2024-03-27T06:36:24.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:24 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi156 2024-03-27T06:36:24.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:24 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi156 to 17080M 2024-03-27T06:36:24.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:24.583+0000 7f56d50c2700 1 -- 172.21.15.170:0/3843423645 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f56d0062a80 con 0x7f56b80a3610 2024-03-27T06:36:24.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:24 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi156 2024-03-27T06:36:24.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:24 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi156 to 17080M 2024-03-27T06:36:24.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:24 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi156 2024-03-27T06:36:24.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:24 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi156 to 17080M 2024-03-27T06:36:25.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:25 smithi156 ceph-mon[36261]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:25.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:25 smithi156 ceph-mon[36261]: osdmap e14: 2 total, 2 up, 2 in 2024-03-27T06:36:25.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:36:25.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:36:25.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:25.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:25 smithi170 ceph-mon[36438]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:25.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:25 smithi170 ceph-mon[36438]: osdmap e14: 2 total, 2 up, 2 in 2024-03-27T06:36:25.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:36:25.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:36:25.493 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:25.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:25 smithi016 ceph-mon[31908]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:25.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:25 smithi016 ceph-mon[31908]: osdmap e14: 2 total, 2 up, 2 in 2024-03-27T06:36:25.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:36:25.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:36:25.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:26.408 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:26 smithi170 ceph-mon[36438]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:36:26.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:26 smithi156 ceph-mon[36261]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:36:26.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:26 smithi016 ceph-mon[31908]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:36:27.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:27 smithi156 ceph-mon[36261]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:27.547 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:27 smithi170 ceph-mon[36438]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:27 smithi016 ceph-mon[31908]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:28.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:28 smithi016 ceph-mon[31908]: from='client.? 172.21.15.170:0/3035329435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c"}]: dispatch 2024-03-27T06:36:28.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:28 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c"}]: dispatch 2024-03-27T06:36:28.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:28 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c"}]': finished 2024-03-27T06:36:28.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:28 smithi016 ceph-mon[31908]: osdmap e15: 3 total, 2 up, 3 in 2024-03-27T06:36:28.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:28.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:28 smithi016 ceph-mon[31908]: from='client.? 172.21.15.170:0/3448593281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:36:28.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:28 smithi156 ceph-mon[36261]: from='client.? 172.21.15.170:0/3035329435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c"}]: dispatch 2024-03-27T06:36:28.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:28 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c"}]: dispatch 2024-03-27T06:36:28.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:28 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c"}]': finished 2024-03-27T06:36:28.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:28 smithi156 ceph-mon[36261]: osdmap e15: 3 total, 2 up, 3 in 2024-03-27T06:36:28.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:28.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:28 smithi156 ceph-mon[36261]: from='client.? 172.21.15.170:0/3448593281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:36:28.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:28 smithi170 ceph-mon[36438]: from='client.? 172.21.15.170:0/3035329435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c"}]: dispatch 2024-03-27T06:36:28.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:28 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c"}]: dispatch 2024-03-27T06:36:28.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:28 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c"}]': finished 2024-03-27T06:36:28.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:28 smithi170 ceph-mon[36438]: osdmap e15: 3 total, 2 up, 3 in 2024-03-27T06:36:28.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:28.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:28 smithi170 ceph-mon[36438]: from='client.? 172.21.15.170:0/3448593281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:36:29.494 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:29 smithi170 ceph-mon[36438]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:29.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:29 smithi016 ceph-mon[31908]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:29.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:29 smithi156 ceph-mon[36261]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:31.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:31 smithi170 ceph-mon[36438]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:31.523 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:31 smithi156 ceph-mon[36261]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:31.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:31 smithi016 ceph-mon[31908]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:33.296 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:33 smithi170 ceph-mon[36438]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:33.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:33 smithi016 ceph-mon[31908]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:33.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:33 smithi156 ceph-mon[36261]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:35.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:35 smithi170 ceph-mon[36438]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:35.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-27T06:36:35.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:35.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:35 smithi170 ceph-mon[36438]: Deploying daemon osd.2 on smithi170 2024-03-27T06:36:35.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:35 smithi016 ceph-mon[31908]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-27T06:36:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:35 smithi016 ceph-mon[31908]: Deploying daemon osd.2 on smithi170 2024-03-27T06:36:35.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:35 smithi156 ceph-mon[36261]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:35.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-27T06:36:35.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:35.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:35 smithi156 ceph-mon[36261]: Deploying daemon osd.2 on smithi170 2024-03-27T06:36:37.500 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:37 smithi170 ceph-mon[36438]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:37 smithi016 ceph-mon[31908]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:37.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:37 smithi156 ceph-mon[36261]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:39.355 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:39 smithi170 ceph-mon[36438]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:39.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:39 smithi016 ceph-mon[31908]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:39.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:39 smithi156 ceph-mon[36261]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:40.536 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:40 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:40.536 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:40 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:40.536 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:40 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:40.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:40 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:40.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:40 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:40.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:40 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:40.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:40 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:40 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:40 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:41.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:41 smithi156 ceph-mon[36261]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:41.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:41 smithi170 ceph-mon[36438]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:41.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:41 smithi016 ceph-mon[31908]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:43.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.351+0000 7f56b7fff700 1 -- 172.21.15.170:0/3843423645 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f56d0062a80 con 0x7f56b80a3610 2024-03-27T06:36:43.345 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 2 on host 'smithi170' 2024-03-27T06:36:43.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.353+0000 7f56d50c2700 1 -- 172.21.15.170:0/3843423645 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f56b80a3610 msgr2=0x7f56b80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:43.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.353+0000 7f56d50c2700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f56b80a3610 0x7f56b80a5ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f56c00039a0 tx=0x7f56c0003a70 comp rx=0 tx=0).stop 2024-03-27T06:36:43.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.353+0000 7f56d50c2700 1 -- 172.21.15.170:0/3843423645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56d0103110 msgr2=0x7f56d019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:43.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.353+0000 7f56d50c2700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56d0103110 0x7f56d019fca0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f56c8007d70 tx=0x7f56c800a340 comp rx=0 tx=0).stop 2024-03-27T06:36:43.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.353+0000 7f56d50c2700 1 -- 172.21.15.170:0/3843423645 shutdown_connections 2024-03-27T06:36:43.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.353+0000 7f56d50c2700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f56b80a3610 0x7f56b80a5ad0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:43.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.353+0000 7f56d50c2700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56d0103110 0x7f56d019fca0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:43.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.353+0000 7f56d50c2700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56d0101570 0x7f56d006b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:43.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.354+0000 7f56d50c2700 1 --2- 172.21.15.170:0/3843423645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56d0102770 0x7f56d019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:43.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.354+0000 7f56d50c2700 1 -- 172.21.15.170:0/3843423645 >> 172.21.15.170:0/3843423645 conn(0x7f56d00fcb30 msgr2=0x7f56d01059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:36:43.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.354+0000 7f56d50c2700 1 -- 172.21.15.170:0/3843423645 shutdown_connections 2024-03-27T06:36:43.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:36:43.354+0000 7f56d50c2700 1 -- 172.21.15.170:0/3843423645 wait complete. 2024-03-27T06:36:43.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:43 smithi170 ceph-mon[36438]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:43.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:43 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:43 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:43 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:43.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:43 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:43.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:43 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:43 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:43 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:43 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:43 smithi170 ceph-mon[36438]: from='osd.2 [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T06:36:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:43 smithi170 ceph-mon[36438]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T06:36:43.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:43 smithi156 ceph-mon[36261]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:43 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:43 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:43 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:43 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:43 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:43 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:43 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:43 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:43 smithi156 ceph-mon[36261]: from='osd.2 [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T06:36:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:43 smithi156 ceph-mon[36261]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T06:36:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:43 smithi016 ceph-mon[31908]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:36:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:43 smithi016 ceph-mon[31908]: from='osd.2 [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T06:36:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:43 smithi016 ceph-mon[31908]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-27T06:36:43.962 DEBUG:teuthology.orchestra.run.smithi170:osd.2> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.2.service 2024-03-27T06:36:43.964 INFO:tasks.cephadm:Deploying osd.3 on smithi016 with /dev/vg_nvme/lv_3... 2024-03-27T06:36:43.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-27T06:36:44.218 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:36:45.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:45 smithi170 ceph-mon[36438]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:45.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:45 smithi170 ceph-mon[36438]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-27T06:36:45.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:45 smithi170 ceph-mon[36438]: osdmap e16: 3 total, 2 up, 3 in 2024-03-27T06:36:45.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:45 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:45.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:45 smithi170 ceph-mon[36438]: from='osd.2 [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:36:45.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:45 smithi170 ceph-mon[36438]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:36:45.471 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:36:45 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2[41564]: 2024-03-27T06:36:45.317+0000 7fbe1cb98700 -1 osd.2 0 waiting for initial osdmap 2024-03-27T06:36:45.471 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:36:45 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2[41564]: 2024-03-27T06:36:45.321+0000 7fbe1898b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:36:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:45 smithi016 ceph-mon[31908]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:45 smithi016 ceph-mon[31908]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-27T06:36:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:45 smithi016 ceph-mon[31908]: osdmap e16: 3 total, 2 up, 3 in 2024-03-27T06:36:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:45 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:45 smithi016 ceph-mon[31908]: from='osd.2 [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:36:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:45 smithi016 ceph-mon[31908]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:36:45.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:45 smithi156 ceph-mon[36261]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:45.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:45 smithi156 ceph-mon[36261]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-27T06:36:45.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:45 smithi156 ceph-mon[36261]: osdmap e16: 3 total, 2 up, 3 in 2024-03-27T06:36:45.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:45 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:45.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:45 smithi156 ceph-mon[36261]: from='osd.2 [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:36:45.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:45 smithi156 ceph-mon[36261]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:36:46.341 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-27T06:36:46.341 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-27T06:36:46.341 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-03-27T06:36:46.341 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-03-27T06:36:46.341 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00884027 s, 1.2 GB/s 2024-03-27T06:36:46.341 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-03-27T06:36:46.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:46 smithi170 ceph-mon[36438]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:36:46.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:46 smithi170 ceph-mon[36438]: osdmap e17: 3 total, 2 up, 3 in 2024-03-27T06:36:46.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:46 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:46.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:46 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:46.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:46 smithi016 ceph-mon[31908]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:36:46.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:46 smithi016 ceph-mon[31908]: osdmap e17: 3 total, 2 up, 3 in 2024-03-27T06:36:46.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:46 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:46.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:46 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:46.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:46 smithi156 ceph-mon[36261]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:36:46.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:46 smithi156 ceph-mon[36261]: osdmap e17: 3 total, 2 up, 3 in 2024-03-27T06:36:46.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:46 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:46.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:46 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:47.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-03-27T06:36:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:36:47.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:36:47.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:36:47.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:47.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: osd.2 [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379] boot 2024-03-27T06:36:47.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: osdmap e18: 3 total, 3 up, 3 in 2024-03-27T06:36:47.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:47.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:47.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:47.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:36:47.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:47.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:47.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:47.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:47 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:47.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: osd.2 [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379] boot 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: osdmap e18: 3 total, 3 up, 3 in 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:47.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:47 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:47.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: osd.2 [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379] boot 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: osdmap e18: 3 total, 3 up, 3 in 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:36:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:47 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:48.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:48 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi170 2024-03-27T06:36:48.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:48 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi170 to 17080M 2024-03-27T06:36:48.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:48 smithi016 ceph-mon[31908]: osdmap e19: 3 total, 3 up, 3 in 2024-03-27T06:36:48.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:48 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-27T06:36:48.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:48 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi170 2024-03-27T06:36:48.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:48 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi170 to 17080M 2024-03-27T06:36:48.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:48 smithi170 ceph-mon[36438]: osdmap e19: 3 total, 3 up, 3 in 2024-03-27T06:36:48.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:48 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-27T06:36:48.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:48 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi170 2024-03-27T06:36:48.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:48 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi170 to 17080M 2024-03-27T06:36:48.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:48 smithi156 ceph-mon[36261]: osdmap e19: 3 total, 3 up, 3 in 2024-03-27T06:36:48.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:48 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-27T06:36:48.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.998+0000 7fb361393700 1 -- 172.21.15.16:0/351640399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c103180 msgr2=0x7fb35c109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:48.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.998+0000 7fb361393700 1 --2- 172.21.15.16:0/351640399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c103180 0x7fb35c109a30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0099c0 tx=0x7fb34c009cd0 comp rx=0 tx=0).stop 2024-03-27T06:36:48.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.998+0000 7fb361393700 1 -- 172.21.15.16:0/351640399 shutdown_connections 2024-03-27T06:36:48.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.998+0000 7fb361393700 1 --2- 172.21.15.16:0/351640399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c103180 0x7fb35c109a30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:48.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.998+0000 7fb361393700 1 --2- 172.21.15.16:0/351640399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c1015e0 0x7fb35c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:48.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.998+0000 7fb361393700 1 --2- 172.21.15.16:0/351640399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb35c1027e0 0x7fb35c102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:48.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.998+0000 7fb361393700 1 -- 172.21.15.16:0/351640399 >> 172.21.15.16:0/351640399 conn(0x7fb35c0fcba0 msgr2=0x7fb35c0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:36:48.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.998+0000 7fb361393700 1 -- 172.21.15.16:0/351640399 shutdown_connections 2024-03-27T06:36:48.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.998+0000 7fb361393700 1 -- 172.21.15.16:0/351640399 wait complete. 2024-03-27T06:36:48.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.998+0000 7fb361393700 1 Processor -- start 2024-03-27T06:36:48.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb361393700 1 -- start start 2024-03-27T06:36:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb361393700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c1015e0 0x7fb35c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb361393700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c1027e0 0x7fb35c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb361393700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb35c103180 0x7fb35c19fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb361393700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c10d2a0 con 0x7fb35c1027e0 2024-03-27T06:36:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb361393700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c10d120 con 0x7fb35c1015e0 2024-03-27T06:36:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb361393700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c10d420 con 0x7fb35c103180 2024-03-27T06:36:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb35a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c1027e0 0x7fb35c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb35a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c1027e0 0x7fb35c198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43180/0 (socket says 172.21.15.16:43180) 2024-03-27T06:36:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb35a7fc700 1 -- 172.21.15.16:0/3064742253 learned_addr learned my addr 172.21.15.16:0/3064742253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:36:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb35b7fe700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb35c103180 0x7fb35c19fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb35affd700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c1015e0 0x7fb35c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb35a7fc700 1 -- 172.21.15.16:0/3064742253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb35c103180 msgr2=0x7fb35c19fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb35a7fc700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb35c103180 0x7fb35c19fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb35a7fc700 1 -- 172.21.15.16:0/3064742253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c1015e0 msgr2=0x7fb35c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:36:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb35a7fc700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c1015e0 0x7fb35c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:36:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:48.999+0000 7fb35a7fc700 1 -- 172.21.15.16:0/3064742253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c1a0530 con 0x7fb35c1027e0 2024-03-27T06:36:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.000+0000 7fb35a7fc700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c1027e0 0x7fb35c198e10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb34400d980 tx=0x7fb34400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:36:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.000+0000 7fb353fff700 1 -- 172.21.15.16:0/3064742253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344016070 con 0x7fb35c1027e0 2024-03-27T06:36:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.000+0000 7fb353fff700 1 -- 172.21.15.16:0/3064742253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb34400de70 con 0x7fb35c1027e0 2024-03-27T06:36:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.000+0000 7fb361393700 1 -- 172.21.15.16:0/3064742253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35c1a0820 con 0x7fb35c1027e0 2024-03-27T06:36:49.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.000+0000 7fb353fff700 1 -- 172.21.15.16:0/3064742253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34400f620 con 0x7fb35c1027e0 2024-03-27T06:36:49.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.000+0000 7fb361393700 1 -- 172.21.15.16:0/3064742253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35c1a0db0 con 0x7fb35c1027e0 2024-03-27T06:36:49.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.001+0000 7fb361393700 1 -- 172.21.15.16:0/3064742253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35c069300 con 0x7fb35c1027e0 2024-03-27T06:36:49.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.002+0000 7fb353fff700 1 -- 172.21.15.16:0/3064742253 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 138314+0+0 (secure 0 0 0) 0x7fb344019070 con 0x7fb35c1027e0 2024-03-27T06:36:49.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.003+0000 7fb353fff700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb3480a3690 0x7fb3480a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:36:49.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.003+0000 7fb35affd700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb3480a3690 0x7fb3480a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:36:49.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.003+0000 7fb353fff700 1 -- 172.21.15.16:0/3064742253 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7fb3440ce650 con 0x7fb35c1027e0 2024-03-27T06:36:49.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.003+0000 7fb35affd700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb3480a3690 0x7fb3480a5b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb3540060b0 tx=0x7fb354007b80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:36:49.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.008+0000 7fb353fff700 1 -- 172.21.15.16:0/3064742253 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb34409c6e0 con 0x7fb35c1027e0 2024-03-27T06:36:49.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:49.224+0000 7fb361393700 1 -- 172.21.15.16:0/3064742253 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb35c0fec30 con 0x7fb3480a3690 2024-03-27T06:36:49.593 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45613]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T06:36:49.593 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45613]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:49.593 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45613]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:49.594 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45613]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:49.594 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45616]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T06:36:49.594 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45616]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:49.594 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45616]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:49.594 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45616]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:49.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 ceph-mon[36438]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:49.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-27T06:36:49.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 ceph-mon[36438]: osdmap e20: 3 total, 3 up, 3 in 2024-03-27T06:36:49.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-27T06:36:49.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:36:49.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:36:49.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:49.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 ceph-mon[31908]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:49.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-27T06:36:49.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 ceph-mon[31908]: osdmap e20: 3 total, 3 up, 3 in 2024-03-27T06:36:49.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-27T06:36:49.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:36:49.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:36:49.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:49.609 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51784]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T06:36:49.609 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51784]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:49.610 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51784]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:49.610 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51784]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:49.610 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51787]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T06:36:49.610 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51787]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:49.610 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51787]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:49.610 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51787]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:49.681 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44531]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T06:36:49.681 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44531]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:49.681 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44531]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:49.681 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44531]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:49.681 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44534]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T06:36:49.682 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44534]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:49.682 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44534]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:49.682 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44534]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:49.682 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 ceph-mon[36261]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:49.682 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-27T06:36:49.682 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 ceph-mon[36261]: osdmap e20: 3 total, 3 up, 3 in 2024-03-27T06:36:49.682 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-27T06:36:49.682 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:36:49.682 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:36:49.682 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:36:49.683 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44537]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T06:36:49.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51790]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T06:36:49.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:49.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51790]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:49.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51790]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:49.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51793]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T06:36:49.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51793]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:49.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51793]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:49.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:49 smithi016 sudo[51793]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:49.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44537]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:49.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44537]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:49.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44537]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:49.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44540]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T06:36:49.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44540]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:49.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44540]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:49.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:49 smithi156 sudo[44540]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:50.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45619]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-27T06:36:50.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:50.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45619]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:50.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45619]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:50.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45622]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-27T06:36:50.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45622]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-27T06:36:50.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45622]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-27T06:36:50.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:49 smithi170 sudo[45622]: pam_unix(sudo:session): session closed for user root 2024-03-27T06:36:50.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:36:50.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-27T06:36:50.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: osdmap e21: 3 total, 3 up, 3 in 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:36:50.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:36:50.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T06:36:50.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T06:36:50.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:36:50.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:36:50.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:36:50.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:50 smithi170 ceph-mon[36438]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T06:36:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:36:50.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: osdmap e21: 3 total, 3 up, 3 in 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:36:50.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T06:36:50.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T06:36:50.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:36:50.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:36:50.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:36:50.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:50 smithi016 ceph-mon[31908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T06:36:50.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:36:50.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-27T06:36:50.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: osdmap e21: 3 total, 3 up, 3 in 2024-03-27T06:36:50.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:50.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:50.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:36:50.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T06:36:50.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T06:36:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:36:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:36:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:36:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T06:36:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:36:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:36:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:36:50.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T06:36:50.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-27T06:36:50.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:36:50.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:36:50.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:36:50.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:50 smithi156 ceph-mon[36261]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-27T06:36:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:36:51.352+0000 7fb353fff700 1 -- 172.21.15.16:0/3064742253 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fb344098650 con 0x7fb35c1027e0 2024-03-27T06:36:51.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:51 smithi016 ceph-mon[31908]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:51.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:51 smithi016 ceph-mon[31908]: osdmap e22: 3 total, 3 up, 3 in 2024-03-27T06:36:51.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:51 smithi156 ceph-mon[36261]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:51.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:51 smithi156 ceph-mon[36261]: osdmap e22: 3 total, 3 up, 3 in 2024-03-27T06:36:51.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:51 smithi170 ceph-mon[36438]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:51.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:51 smithi170 ceph-mon[36438]: osdmap e22: 3 total, 3 up, 3 in 2024-03-27T06:36:52.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:52 smithi016 ceph-mon[31908]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-27T06:36:52.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:52 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2067791495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f77f4328-207f-48e0-bd4e-8c2cb212ae4f"}]: dispatch 2024-03-27T06:36:52.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:52 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2067791495' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f77f4328-207f-48e0-bd4e-8c2cb212ae4f"}]': finished 2024-03-27T06:36:52.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:52 smithi016 ceph-mon[31908]: osdmap e23: 4 total, 3 up, 4 in 2024-03-27T06:36:52.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:52 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:36:52.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:52 smithi156 ceph-mon[36261]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-27T06:36:52.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:52 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2067791495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f77f4328-207f-48e0-bd4e-8c2cb212ae4f"}]: dispatch 2024-03-27T06:36:52.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:52 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2067791495' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f77f4328-207f-48e0-bd4e-8c2cb212ae4f"}]': finished 2024-03-27T06:36:52.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:52 smithi156 ceph-mon[36261]: osdmap e23: 4 total, 3 up, 4 in 2024-03-27T06:36:52.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:52 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:36:52.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:52 smithi170 ceph-mon[36438]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-27T06:36:52.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:52 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2067791495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f77f4328-207f-48e0-bd4e-8c2cb212ae4f"}]: dispatch 2024-03-27T06:36:52.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:52 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2067791495' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f77f4328-207f-48e0-bd4e-8c2cb212ae4f"}]': finished 2024-03-27T06:36:52.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:52 smithi170 ceph-mon[36438]: osdmap e23: 4 total, 3 up, 4 in 2024-03-27T06:36:52.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:52 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:36:53.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:53 smithi016 ceph-mon[31908]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:53.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:53 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1126055093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:36:53.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:53 smithi156 ceph-mon[36261]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:53.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:53 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1126055093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:36:53.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:53 smithi170 ceph-mon[36438]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:53.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:53 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1126055093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:36:55.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:55 smithi016 ceph-mon[31908]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:55.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:55 smithi156 ceph-mon[36261]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:55.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:55 smithi170 ceph-mon[36438]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:57.704 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:57 smithi016 ceph-mon[31908]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:57.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:57 smithi156 ceph-mon[36261]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:57.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:57 smithi170 ceph-mon[36438]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:59.599 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:36:59 smithi016 ceph-mon[31908]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:59.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:36:59 smithi170 ceph-mon[36438]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:36:59.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:36:59 smithi156 ceph-mon[36261]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:00.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:00 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-27T06:37:00.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:00 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:00.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:00 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-27T06:37:00.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:00 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:00.839 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:00 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-27T06:37:00.839 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:00 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:01.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:01 smithi016 ceph-mon[31908]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:01.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:01 smithi016 ceph-mon[31908]: Deploying daemon osd.3 on smithi016 2024-03-27T06:37:01.622 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:01 smithi156 ceph-mon[36261]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:01.623 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:01 smithi156 ceph-mon[36261]: Deploying daemon osd.3 on smithi016 2024-03-27T06:37:01.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:01 smithi170 ceph-mon[36438]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:01.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:01 smithi170 ceph-mon[36438]: Deploying daemon osd.3 on smithi016 2024-03-27T06:37:03.693 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:03 smithi016 ceph-mon[31908]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:03 smithi156 ceph-mon[36261]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:03.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:03 smithi170 ceph-mon[36438]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:05.661 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:05 smithi016 ceph-mon[31908]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:05.661 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:05 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:05.661 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:05 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:05.661 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:05 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:05.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:05 smithi156 ceph-mon[36261]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:05.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:05 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:05.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:05 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:05.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:05 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:05.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:05 smithi170 ceph-mon[36438]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:05.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:05 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:05.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:05 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:05.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:05 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:07.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:07 smithi016 ceph-mon[31908]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:07.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:07 smithi156 ceph-mon[36261]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:07.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:07 smithi170 ceph-mon[36438]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:08.782 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 3 on host 'smithi016' 2024-03-27T06:37:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.787+0000 7fb353fff700 1 -- 172.21.15.16:0/3064742253 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb35c0fec30 con 0x7fb3480a3690 2024-03-27T06:37:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.789+0000 7fb361393700 1 -- 172.21.15.16:0/3064742253 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb3480a3690 msgr2=0x7fb3480a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.789+0000 7fb361393700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb3480a3690 0x7fb3480a5b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb3540060b0 tx=0x7fb354007b80 comp rx=0 tx=0).stop 2024-03-27T06:37:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.789+0000 7fb361393700 1 -- 172.21.15.16:0/3064742253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c1027e0 msgr2=0x7fb35c198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.789+0000 7fb361393700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c1027e0 0x7fb35c198e10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb34400d980 tx=0x7fb34400dc90 comp rx=0 tx=0).stop 2024-03-27T06:37:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.790+0000 7fb361393700 1 -- 172.21.15.16:0/3064742253 shutdown_connections 2024-03-27T06:37:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.790+0000 7fb361393700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb3480a3690 0x7fb3480a5b50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.790+0000 7fb361393700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb35c103180 0x7fb35c19fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.790+0000 7fb361393700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c1015e0 0x7fb35c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.790+0000 7fb361393700 1 --2- 172.21.15.16:0/3064742253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c1027e0 0x7fb35c198e10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.790+0000 7fb361393700 1 -- 172.21.15.16:0/3064742253 >> 172.21.15.16:0/3064742253 conn(0x7fb35c0fcba0 msgr2=0x7fb35c0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:37:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.790+0000 7fb361393700 1 -- 172.21.15.16:0/3064742253 shutdown_connections 2024-03-27T06:37:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:37:08.790+0000 7fb361393700 1 -- 172.21.15.16:0/3064742253 wait complete. 2024-03-27T06:37:09.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:09 smithi016 ceph-mon[31908]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:09.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:09 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:09.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:09 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:09.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:09 smithi016 ceph-mon[31908]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-27T06:37:09.520 DEBUG:teuthology.orchestra.run.smithi016:osd.3> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.3.service 2024-03-27T06:37:09.522 INFO:tasks.cephadm:Deploying osd.4 on smithi156 with /dev/vg_nvme/lv_3... 2024-03-27T06:37:09.522 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-27T06:37:09.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:09 smithi156 ceph-mon[36261]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:09.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:09 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:09.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:09 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:09.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:09 smithi156 ceph-mon[36261]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-27T06:37:09.768 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:37:09.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:09 smithi170 ceph-mon[36438]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:09.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:09 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:09.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:09 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:09.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:09 smithi170 ceph-mon[36438]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-27T06:37:10.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:10 smithi016 ceph-mon[31908]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:10.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:10 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:10.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:10 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:10.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:10 smithi016 ceph-mon[31908]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-27T06:37:10.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:10 smithi016 ceph-mon[31908]: osdmap e24: 4 total, 3 up, 4 in 2024-03-27T06:37:10.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:10 smithi016 ceph-mon[31908]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:37:10.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:10 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:10.905 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:10 smithi170 ceph-mon[36438]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:10.905 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:10 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:10.905 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:10 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:10.905 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:10 smithi170 ceph-mon[36438]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-27T06:37:10.905 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:10 smithi170 ceph-mon[36438]: osdmap e24: 4 total, 3 up, 4 in 2024-03-27T06:37:10.905 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:10 smithi170 ceph-mon[36438]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:37:10.905 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:10 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:10.906 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:10 smithi156 ceph-mon[36261]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:10.906 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:10 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:10.906 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:10 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:10.906 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:10 smithi156 ceph-mon[36261]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-27T06:37:10.907 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:10 smithi156 ceph-mon[36261]: osdmap e24: 4 total, 3 up, 4 in 2024-03-27T06:37:10.907 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:10 smithi156 ceph-mon[36261]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:37:10.907 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:10 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:11.441 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:37:11 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3[53978]: 2024-03-27T06:37:11.184+0000 7f9c3cc05700 -1 osd.3 0 waiting for initial osdmap 2024-03-27T06:37:11.441 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:37:11 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3[53978]: 2024-03-27T06:37:11.189+0000 7f9c349f0700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:37:11.782 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-27T06:37:11.782 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-27T06:37:11.782 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-03-27T06:37:11.782 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-03-27T06:37:11.782 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00831319 s, 1.3 GB/s 2024-03-27T06:37:11.783 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-03-27T06:37:12.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:11 smithi170 ceph-mon[36438]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:37:12.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:11 smithi170 ceph-mon[36438]: osdmap e25: 4 total, 3 up, 4 in 2024-03-27T06:37:12.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:12.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:12.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:11 smithi170 ceph-mon[36438]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' 2024-03-27T06:37:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:11 smithi016 ceph-mon[31908]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:37:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:11 smithi016 ceph-mon[31908]: osdmap e25: 4 total, 3 up, 4 in 2024-03-27T06:37:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:11 smithi016 ceph-mon[31908]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' 2024-03-27T06:37:12.165 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:11 smithi156 ceph-mon[36261]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:37:12.165 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:11 smithi156 ceph-mon[36261]: osdmap e25: 4 total, 3 up, 4 in 2024-03-27T06:37:12.165 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:12.165 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:12.165 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:11 smithi156 ceph-mon[36261]: from='osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606]' entity='osd.3' 2024-03-27T06:37:12.520 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_3 2024-03-27T06:37:12.763 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:37:13.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:12 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:37:13.003 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:12 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:37:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:12 smithi156 ceph-mon[36261]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:12 smithi156 ceph-mon[36261]: osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606] boot 2024-03-27T06:37:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:12 smithi156 ceph-mon[36261]: osdmap e26: 4 total, 4 up, 4 in 2024-03-27T06:37:13.004 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:12 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:13.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:12 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:37:13.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:12 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:37:13.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:12 smithi016 ceph-mon[31908]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:13.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:12 smithi016 ceph-mon[31908]: osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606] boot 2024-03-27T06:37:13.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:12 smithi016 ceph-mon[31908]: osdmap e26: 4 total, 4 up, 4 in 2024-03-27T06:37:13.081 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:12 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:13.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:12 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:37:13.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:12 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:37:13.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:12 smithi170 ceph-mon[36438]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-27T06:37:13.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:12 smithi170 ceph-mon[36438]: osd.3 [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606] boot 2024-03-27T06:37:13.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:12 smithi170 ceph-mon[36438]: osdmap e26: 4 total, 4 up, 4 in 2024-03-27T06:37:13.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:12 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:37:13.872 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: osdmap e27: 4 total, 4 up, 4 in 2024-03-27T06:37:13.873 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi016 2024-03-27T06:37:13.873 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:13.873 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:13.873 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:13.873 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:13.873 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi016 to 8546M 2024-03-27T06:37:13.874 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:13.874 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:13.874 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:13.874 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:13.874 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:13.874 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:13.874 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:13.875 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:14.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: osdmap e27: 4 total, 4 up, 4 in 2024-03-27T06:37:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi016 2024-03-27T06:37:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi016 to 8546M 2024-03-27T06:37:14.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:14.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:14.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:14.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:14.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:14.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:14.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:14.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: osdmap e27: 4 total, 4 up, 4 in 2024-03-27T06:37:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi016 2024-03-27T06:37:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi016 to 8546M 2024-03-27T06:37:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:14.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:14.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:14.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:14.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:14.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:14.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:14.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.341+0000 7f77cbb80700 1 -- 172.21.15.156:0/2196343263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77c4102670 msgr2=0x7f77c4102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:14.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.341+0000 7f77cbb80700 1 --2- 172.21.15.156:0/2196343263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77c4102670 0x7f77c4102a90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f77b40099c0 tx=0x7f77b4009cd0 comp rx=0 tx=0).stop 2024-03-27T06:37:14.339 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.341+0000 7f77cbb80700 1 -- 172.21.15.156:0/2196343263 shutdown_connections 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.341+0000 7f77cbb80700 1 --2- 172.21.15.156:0/2196343263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4104210 0x7f77c410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.341+0000 7f77cbb80700 1 --2- 172.21.15.156:0/2196343263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77c4102670 0x7f77c4102a90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.341+0000 7f77cbb80700 1 --2- 172.21.15.156:0/2196343263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c4103870 0x7f77c4103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.341+0000 7f77cbb80700 1 -- 172.21.15.156:0/2196343263 >> 172.21.15.156:0/2196343263 conn(0x7f77c40fdc50 msgr2=0x7f77c4100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.341+0000 7f77cbb80700 1 -- 172.21.15.156:0/2196343263 shutdown_connections 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.341+0000 7f77cbb80700 1 -- 172.21.15.156:0/2196343263 wait complete. 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77cbb80700 1 Processor -- start 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77cbb80700 1 -- start start 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77cbb80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4102670 0x7f77c40763e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77cbb80700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c4103870 0x7f77c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77cbb80700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77c4104210 0x7f77c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:37:14.340 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77cbb80700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77c410e370 con 0x7f77c4102670 2024-03-27T06:37:14.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77cbb80700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f77c410e1f0 con 0x7f77c4104210 2024-03-27T06:37:14.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77cbb80700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f77c410e4f0 con 0x7f77c4103870 2024-03-27T06:37:14.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77ca11d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77c4104210 0x7f77c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:37:14.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77c991c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4102670 0x7f77c40763e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:37:14.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77ca11d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77c4104210 0x7f77c4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:59580/0 (socket says 172.21.15.156:59580) 2024-03-27T06:37:14.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.342+0000 7f77ca11d700 1 -- 172.21.15.156:0/2960939627 learned_addr learned my addr 172.21.15.156:0/2960939627 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-27T06:37:14.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77c911b700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c4103870 0x7f77c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:37:14.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77c991c700 1 -- 172.21.15.156:0/2960939627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c4103870 msgr2=0x7f77c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:14.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77c991c700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c4103870 0x7f77c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:14.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77c991c700 1 -- 172.21.15.156:0/2960939627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77c4104210 msgr2=0x7f77c4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:14.341 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77c991c700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77c4104210 0x7f77c4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:14.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77c991c700 1 -- 172.21.15.156:0/2960939627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c4075820 con 0x7f77c4102670 2024-03-27T06:37:14.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77ca11d700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77c4104210 0x7f77c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:37:14.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77c991c700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4102670 0x7f77c40763e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f77b4003860 tx=0x7f77b4003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:37:14.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77baffd700 1 -- 172.21.15.156:0/2960939627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4027070 con 0x7f77c4102670 2024-03-27T06:37:14.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77cbb80700 1 -- 172.21.15.156:0/2960939627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c41a2a80 con 0x7f77c4102670 2024-03-27T06:37:14.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77baffd700 1 -- 172.21.15.156:0/2960939627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f77b400bda0 con 0x7f77c4102670 2024-03-27T06:37:14.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77baffd700 1 -- 172.21.15.156:0/2960939627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4022e40 con 0x7f77c4102670 2024-03-27T06:37:14.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.343+0000 7f77cbb80700 1 -- 172.21.15.156:0/2960939627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c41a2f30 con 0x7f77c4102670 2024-03-27T06:37:14.342 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.344+0000 7f77cbb80700 1 -- 172.21.15.156:0/2960939627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c4069300 con 0x7f77c4102670 2024-03-27T06:37:14.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.346+0000 7f77baffd700 1 -- 172.21.15.156:0/2960939627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f77b402a070 con 0x7f77c4102670 2024-03-27T06:37:14.343 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.346+0000 7f77baffd700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f77b00a36a0 0x7f77b00a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:37:14.344 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.346+0000 7f77baffd700 1 -- 172.21.15.156:0/2960939627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f77b40cf8d0 con 0x7f77c4102670 2024-03-27T06:37:14.344 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.346+0000 7f77c911b700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f77b00a36a0 0x7f77b00a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:37:14.344 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.347+0000 7f77c911b700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f77b00a36a0 0x7f77b00a5b60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005950 tx=0x7f77c000a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:37:14.345 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.351+0000 7f77baffd700 1 -- 172.21.15.156:0/2960939627 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f77b4013070 con 0x7f77c4102670 2024-03-27T06:37:14.555 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:14.560+0000 7f77cbb80700 1 -- 172.21.15.156:0/2960939627 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f77c4062a80 con 0x7f77b00a36a0 2024-03-27T06:37:14.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:14 smithi156 ceph-mon[36261]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:14.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:14 smithi156 ceph-mon[36261]: osdmap e28: 4 total, 4 up, 4 in 2024-03-27T06:37:14.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:14 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:37:14.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:14 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:37:14.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:14 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:15.014 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:14 smithi016 ceph-mon[31908]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:15.014 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:14 smithi016 ceph-mon[31908]: osdmap e28: 4 total, 4 up, 4 in 2024-03-27T06:37:15.014 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:14 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:37:15.014 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:14 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:37:15.014 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:14 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:15.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:14 smithi170 ceph-mon[36438]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:15.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:14 smithi170 ceph-mon[36438]: osdmap e28: 4 total, 4 up, 4 in 2024-03-27T06:37:15.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:14 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:37:15.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:14 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:37:15.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:14 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:16.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:15 smithi170 ceph-mon[36438]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:37:16.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:15 smithi170 ceph-mon[36438]: osdmap e29: 4 total, 4 up, 4 in 2024-03-27T06:37:16.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:15 smithi016 ceph-mon[31908]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:37:16.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:15 smithi016 ceph-mon[31908]: osdmap e29: 4 total, 4 up, 4 in 2024-03-27T06:37:16.132 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:15 smithi156 ceph-mon[36261]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:37:16.132 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:15 smithi156 ceph-mon[36261]: osdmap e29: 4 total, 4 up, 4 in 2024-03-27T06:37:17.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:16 smithi170 ceph-mon[36438]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:17.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:16 smithi170 ceph-mon[36438]: osdmap e30: 4 total, 4 up, 4 in 2024-03-27T06:37:17.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:16 smithi016 ceph-mon[31908]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:17.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:16 smithi016 ceph-mon[31908]: osdmap e30: 4 total, 4 up, 4 in 2024-03-27T06:37:17.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:16 smithi156 ceph-mon[36261]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:17.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:16 smithi156 ceph-mon[36261]: osdmap e30: 4 total, 4 up, 4 in 2024-03-27T06:37:17.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:17 smithi156 ceph-mon[36261]: from='client.? 172.21.15.156:0/3382482587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78647188-4493-4037-97d1-92e1f6f82fc2"}]: dispatch 2024-03-27T06:37:17.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:17 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78647188-4493-4037-97d1-92e1f6f82fc2"}]: dispatch 2024-03-27T06:37:17.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:17 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78647188-4493-4037-97d1-92e1f6f82fc2"}]': finished 2024-03-27T06:37:17.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:17 smithi156 ceph-mon[36261]: osdmap e31: 5 total, 4 up, 5 in 2024-03-27T06:37:17.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:17 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:18.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:17 smithi170 ceph-mon[36438]: from='client.? 172.21.15.156:0/3382482587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78647188-4493-4037-97d1-92e1f6f82fc2"}]: dispatch 2024-03-27T06:37:18.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:17 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78647188-4493-4037-97d1-92e1f6f82fc2"}]: dispatch 2024-03-27T06:37:18.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:17 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78647188-4493-4037-97d1-92e1f6f82fc2"}]': finished 2024-03-27T06:37:18.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:17 smithi170 ceph-mon[36438]: osdmap e31: 5 total, 4 up, 5 in 2024-03-27T06:37:18.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:17 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:18.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:17 smithi016 ceph-mon[31908]: from='client.? 172.21.15.156:0/3382482587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78647188-4493-4037-97d1-92e1f6f82fc2"}]: dispatch 2024-03-27T06:37:18.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:17 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78647188-4493-4037-97d1-92e1f6f82fc2"}]: dispatch 2024-03-27T06:37:18.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:17 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78647188-4493-4037-97d1-92e1f6f82fc2"}]': finished 2024-03-27T06:37:18.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:17 smithi016 ceph-mon[31908]: osdmap e31: 5 total, 4 up, 5 in 2024-03-27T06:37:18.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:17 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:19.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:18 smithi170 ceph-mon[36438]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:19.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:18 smithi170 ceph-mon[36438]: from='client.? 172.21.15.156:0/3852896960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:37:19.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:18 smithi016 ceph-mon[31908]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:19.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:18 smithi016 ceph-mon[31908]: from='client.? 172.21.15.156:0/3852896960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:37:19.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:18 smithi156 ceph-mon[36261]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:19.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:18 smithi156 ceph-mon[36261]: from='client.? 172.21.15.156:0/3852896960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:37:21.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:20 smithi170 ceph-mon[36438]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-03-27T06:37:21.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:20 smithi016 ceph-mon[31908]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-03-27T06:37:21.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:20 smithi156 ceph-mon[36261]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-03-27T06:37:22.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:22 smithi156 ceph-mon[36261]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-27T06:37:23.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:22 smithi170 ceph-mon[36438]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-27T06:37:23.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:22 smithi016 ceph-mon[31908]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-27T06:37:25.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:24 smithi170 ceph-mon[36438]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:37:25.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:24 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-27T06:37:25.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:24 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:25.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:24 smithi016 ceph-mon[31908]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:37:25.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:24 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-27T06:37:25.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:24 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:25.142 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:24 smithi156 ceph-mon[36261]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:37:25.142 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:24 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-27T06:37:25.142 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:24 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:26.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:25 smithi170 ceph-mon[36438]: Deploying daemon osd.4 on smithi156 2024-03-27T06:37:26.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:25 smithi016 ceph-mon[31908]: Deploying daemon osd.4 on smithi156 2024-03-27T06:37:26.142 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:25 smithi156 ceph-mon[36261]: Deploying daemon osd.4 on smithi156 2024-03-27T06:37:27.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:26 smithi170 ceph-mon[36438]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-27T06:37:27.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:26 smithi016 ceph-mon[31908]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-27T06:37:27.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:26 smithi156 ceph-mon[36261]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-27T06:37:29.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:28 smithi170 ceph-mon[36438]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-27T06:37:29.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:28 smithi016 ceph-mon[31908]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-27T06:37:29.200 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:28 smithi156 ceph-mon[36261]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-27T06:37:31.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:31 smithi156 ceph-mon[36261]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-27T06:37:31.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:31.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:31.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:31.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:31 smithi170 ceph-mon[36438]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-27T06:37:31.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:31.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:31.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:31 smithi016 ceph-mon[31908]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-27T06:37:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:33.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:33 smithi156 ceph-mon[36261]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:33.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:33 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:33.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:33 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:33.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:33 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:33.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:33 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:33.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:33 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:33.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:33 smithi170 ceph-mon[36438]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:33.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:33 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:33.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:33 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:33.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:33 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:33.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:33 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:33.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:33 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:33.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:33 smithi016 ceph-mon[31908]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:33.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:33 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:33.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:33 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:33.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:33 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:33.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:33 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:33.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:33 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:34.073 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 4 on host 'smithi156' 2024-03-27T06:37:34.074 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.079+0000 7f77baffd700 1 -- 172.21.15.156:0/2960939627 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f77c4062a80 con 0x7f77b00a36a0 2024-03-27T06:37:34.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.081+0000 7f77cbb80700 1 -- 172.21.15.156:0/2960939627 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f77b00a36a0 msgr2=0x7f77b00a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:34.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.081+0000 7f77cbb80700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f77b00a36a0 0x7f77b00a5b60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005950 tx=0x7f77c000a300 comp rx=0 tx=0).stop 2024-03-27T06:37:34.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.082+0000 7f77cbb80700 1 -- 172.21.15.156:0/2960939627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4102670 msgr2=0x7f77c40763e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:34.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.082+0000 7f77cbb80700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4102670 0x7f77c40763e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f77b4003860 tx=0x7f77b4003c60 comp rx=0 tx=0).stop 2024-03-27T06:37:34.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.082+0000 7f77cbb80700 1 -- 172.21.15.156:0/2960939627 shutdown_connections 2024-03-27T06:37:34.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.082+0000 7f77cbb80700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77c4104210 0x7f77c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:34.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.082+0000 7f77cbb80700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4102670 0x7f77c40763e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:34.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.082+0000 7f77cbb80700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f77b00a36a0 0x7f77b00a5b60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:34.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.082+0000 7f77cbb80700 1 --2- 172.21.15.156:0/2960939627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c4103870 0x7f77c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:34.076 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.082+0000 7f77cbb80700 1 -- 172.21.15.156:0/2960939627 >> 172.21.15.156:0/2960939627 conn(0x7f77c40fdc50 msgr2=0x7f77c4106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:37:34.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.082+0000 7f77cbb80700 1 -- 172.21.15.156:0/2960939627 shutdown_connections 2024-03-27T06:37:34.077 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:37:34.082+0000 7f77cbb80700 1 -- 172.21.15.156:0/2960939627 wait complete. 2024-03-27T06:37:34.319 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:34 smithi156 ceph-mon[36261]: from='osd.4 [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T06:37:34.319 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:34 smithi156 ceph-mon[36261]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T06:37:34.320 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:34 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:34.320 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:34 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:34.320 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:34 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:34.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:34 smithi170 ceph-mon[36438]: from='osd.4 [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T06:37:34.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:34 smithi170 ceph-mon[36438]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T06:37:34.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:34 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:34.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:34 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:34.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:34 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:34.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:34 smithi016 ceph-mon[31908]: from='osd.4 [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T06:37:34.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:34 smithi016 ceph-mon[31908]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-27T06:37:34.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:34 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:34.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:34 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:34.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:34 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:34.720 DEBUG:teuthology.orchestra.run.smithi156:osd.4> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.4.service 2024-03-27T06:37:34.722 INFO:tasks.cephadm:Deploying osd.5 on smithi170 with /dev/vg_nvme/lv_3... 2024-03-27T06:37:34.723 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-27T06:37:34.968 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:37:35.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:35 smithi170 ceph-mon[36438]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:35.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:35 smithi170 ceph-mon[36438]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-27T06:37:35.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:35 smithi170 ceph-mon[36438]: from='osd.4 [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:37:35.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:35 smithi170 ceph-mon[36438]: osdmap e32: 5 total, 4 up, 5 in 2024-03-27T06:37:35.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:35.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:35 smithi170 ceph-mon[36438]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:37:35.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:35 smithi156 ceph-mon[36261]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:35.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:35 smithi156 ceph-mon[36261]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-27T06:37:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:35 smithi156 ceph-mon[36261]: from='osd.4 [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:37:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:35 smithi156 ceph-mon[36261]: osdmap e32: 5 total, 4 up, 5 in 2024-03-27T06:37:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:35 smithi156 ceph-mon[36261]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:37:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:35 smithi016 ceph-mon[31908]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:35 smithi016 ceph-mon[31908]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-27T06:37:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:35 smithi016 ceph-mon[31908]: from='osd.4 [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:37:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:35 smithi016 ceph-mon[31908]: osdmap e32: 5 total, 4 up, 5 in 2024-03-27T06:37:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:35 smithi016 ceph-mon[31908]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:37:35.971 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:37:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4[47353]: 2024-03-27T06:37:35.569+0000 7efd6df25700 -1 osd.4 0 waiting for initial osdmap 2024-03-27T06:37:35.971 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:37:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4[47353]: 2024-03-27T06:37:35.575+0000 7efd65d10700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:37:36.297 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:36 smithi170 ceph-mon[36438]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:37:36.297 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:36 smithi170 ceph-mon[36438]: osdmap e33: 5 total, 4 up, 5 in 2024-03-27T06:37:36.297 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:36 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:36.297 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:36 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:36.297 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:36 smithi170 ceph-mon[36438]: from='osd.4 ' entity='osd.4' 2024-03-27T06:37:36.313 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:36 smithi156 ceph-mon[36261]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:37:36.313 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:36 smithi156 ceph-mon[36261]: osdmap e33: 5 total, 4 up, 5 in 2024-03-27T06:37:36.313 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:36 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:36.313 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:36 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:36.313 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:36 smithi156 ceph-mon[36261]: from='osd.4 ' entity='osd.4' 2024-03-27T06:37:36.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:36 smithi016 ceph-mon[31908]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:37:36.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:36 smithi016 ceph-mon[31908]: osdmap e33: 5 total, 4 up, 5 in 2024-03-27T06:37:36.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:36.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:36 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:36.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:36 smithi016 ceph-mon[31908]: from='osd.4 ' entity='osd.4' 2024-03-27T06:37:37.002 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-27T06:37:37.003 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-27T06:37:37.003 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-03-27T06:37:37.003 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-03-27T06:37:37.003 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00827303 s, 1.3 GB/s 2024-03-27T06:37:37.003 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-03-27T06:37:37.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:37 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:37:37.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:37 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:37:37.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:37 smithi156 ceph-mon[36261]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:37.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:37 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:37.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:37 smithi156 ceph-mon[36261]: osd.4 [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974] boot 2024-03-27T06:37:37.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:37 smithi156 ceph-mon[36261]: osdmap e34: 5 total, 5 up, 5 in 2024-03-27T06:37:37.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:37 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:37.309 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:37 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:37:37.309 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:37 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:37:37.309 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:37 smithi170 ceph-mon[36438]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:37.309 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:37 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:37.309 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:37 smithi170 ceph-mon[36438]: osd.4 [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974] boot 2024-03-27T06:37:37.309 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:37 smithi170 ceph-mon[36438]: osdmap e34: 5 total, 5 up, 5 in 2024-03-27T06:37:37.309 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:37 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:37.596 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi170:vg_nvme/lv_3 2024-03-27T06:37:37.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:37 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:37:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:37 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:37:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:37 smithi016 ceph-mon[31908]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-27T06:37:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:37 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:37 smithi016 ceph-mon[31908]: osd.4 [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974] boot 2024-03-27T06:37:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:37 smithi016 ceph-mon[31908]: osdmap e34: 5 total, 5 up, 5 in 2024-03-27T06:37:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:37 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:37:37.850 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:37:38.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:38.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: osdmap e35: 5 total, 5 up, 5 in 2024-03-27T06:37:38.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi156 2024-03-27T06:37:38.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:38.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:38.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:38.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:38.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi156 to 8540M 2024-03-27T06:37:38.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:38.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:38.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:38.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:38.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:38.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: osdmap e35: 5 total, 5 up, 5 in 2024-03-27T06:37:38.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi156 2024-03-27T06:37:38.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:38.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:38.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:38.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:38.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi156 to 8540M 2024-03-27T06:37:38.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:38.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:38.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:38.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:38.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: osdmap e35: 5 total, 5 up, 5 in 2024-03-27T06:37:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi156 2024-03-27T06:37:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:37:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi156 to 8540M 2024-03-27T06:37:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:39.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.503+0000 7ff44cd87700 1 -- 172.21.15.170:0/876455842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff448102770 msgr2=0x7ff448102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:39.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.503+0000 7ff44cd87700 1 --2- 172.21.15.170:0/876455842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff448102770 0x7ff448102bd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009990 tx=0x7ff43c009ca0 comp rx=0 tx=0).stop 2024-03-27T06:37:39.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.504+0000 7ff44cd87700 1 -- 172.21.15.170:0/876455842 shutdown_connections 2024-03-27T06:37:39.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.504+0000 7ff44cd87700 1 --2- 172.21.15.170:0/876455842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff448103110 0x7ff4481099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:39.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.504+0000 7ff44cd87700 1 --2- 172.21.15.170:0/876455842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448101570 0x7ff448101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:39.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.504+0000 7ff44cd87700 1 --2- 172.21.15.170:0/876455842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff448102770 0x7ff448102bd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:39.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.504+0000 7ff44cd87700 1 -- 172.21.15.170:0/876455842 >> 172.21.15.170:0/876455842 conn(0x7ff4480fcb30 msgr2=0x7ff4480fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:37:39.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.504+0000 7ff44cd87700 1 -- 172.21.15.170:0/876455842 shutdown_connections 2024-03-27T06:37:39.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.504+0000 7ff44cd87700 1 -- 172.21.15.170:0/876455842 wait complete. 2024-03-27T06:37:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.504+0000 7ff44cd87700 1 Processor -- start 2024-03-27T06:37:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.504+0000 7ff44cd87700 1 -- start start 2024-03-27T06:37:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.504+0000 7ff44cd87700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff448101570 0x7ff448198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:37:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff44cd87700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448102770 0x7ff448198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:37:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff44cd87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff448103110 0x7ff44819fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:37:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff44cd87700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810d2f0 con 0x7ff448102770 2024-03-27T06:37:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff44cd87700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810d170 con 0x7ff448101570 2024-03-27T06:37:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff44cd87700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810d470 con 0x7ff448103110 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff446d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff448103110 0x7ff44819fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff446d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff448103110 0x7ff44819fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49066/0 (socket says 172.21.15.170:49066) 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff446d9d700 1 -- 172.21.15.170:0/3425004238 learned_addr learned my addr 172.21.15.170:0/3425004238 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff44659c700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff448101570 0x7ff448198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff445d9b700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448102770 0x7ff448198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff446d9d700 1 -- 172.21.15.170:0/3425004238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff448101570 msgr2=0x7ff448198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff446d9d700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff448101570 0x7ff448198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff446d9d700 1 -- 172.21.15.170:0/3425004238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448102770 msgr2=0x7ff448198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff446d9d700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448102770 0x7ff448198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff446d9d700 1 -- 172.21.15.170:0/3425004238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4481a0530 con 0x7ff448103110 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff445d9b700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448102770 0x7ff448198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff446d9d700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff448103110 0x7ff44819fe90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b9a0 tx=0x7ff44000bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:37:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff4377fe700 1 -- 172.21.15.170:0/3425004238 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff440019070 con 0x7ff448103110 2024-03-27T06:37:39.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff44cd87700 1 -- 172.21.15.170:0/3425004238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4481a0820 con 0x7ff448103110 2024-03-27T06:37:39.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff4377fe700 1 -- 172.21.15.170:0/3425004238 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff44000ce30 con 0x7ff448103110 2024-03-27T06:37:39.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.505+0000 7ff4377fe700 1 -- 172.21.15.170:0/3425004238 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4400075d0 con 0x7ff448103110 2024-03-27T06:37:39.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.506+0000 7ff44cd87700 1 -- 172.21.15.170:0/3425004238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4481a0d10 con 0x7ff448103110 2024-03-27T06:37:39.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.506+0000 7ff44cd87700 1 -- 172.21.15.170:0/3425004238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff448069300 con 0x7ff448103110 2024-03-27T06:37:39.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.508+0000 7ff4377fe700 1 -- 172.21.15.170:0/3425004238 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7ff44001c070 con 0x7ff448103110 2024-03-27T06:37:39.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.508+0000 7ff4377fe700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4300a36a0 0x7ff4300a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:37:39.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.508+0000 7ff4377fe700 1 -- 172.21.15.170:0/3425004238 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7ff4400cea90 con 0x7ff448103110 2024-03-27T06:37:39.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.508+0000 7ff44659c700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4300a36a0 0x7ff4300a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:37:39.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.509+0000 7ff44659c700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4300a36a0 0x7ff4300a5b60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4481025d0 tx=0x7ff43800a3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:37:39.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.512+0000 7ff4377fe700 1 -- 172.21.15.170:0/3425004238 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff440023070 con 0x7ff448103110 2024-03-27T06:37:39.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:39.718+0000 7ff44cd87700 1 -- 172.21.15.170:0/3425004238 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff4480feb80 con 0x7ff4300a36a0 2024-03-27T06:37:40.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:40 smithi170 ceph-mon[36438]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:40.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:40 smithi170 ceph-mon[36438]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:37:40.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:40 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:37:40.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:40 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:37:40.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:40 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:40 smithi016 ceph-mon[31908]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:40 smithi016 ceph-mon[31908]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:37:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:40 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:37:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:40 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:37:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:40 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:40.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:40 smithi156 ceph-mon[36261]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:40.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:40 smithi156 ceph-mon[36261]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:37:40.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:40 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:37:40.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:40 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:37:40.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:40 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:42.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:42 smithi170 ceph-mon[36438]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:42.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:42 smithi170 ceph-mon[36438]: from='client.? 172.21.15.170:0/887240035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4efed7-91d9-4adc-881a-ce3f44f80813"}]: dispatch 2024-03-27T06:37:42.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:42 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4efed7-91d9-4adc-881a-ce3f44f80813"}]: dispatch 2024-03-27T06:37:42.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:42 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f4efed7-91d9-4adc-881a-ce3f44f80813"}]': finished 2024-03-27T06:37:42.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:42 smithi170 ceph-mon[36438]: osdmap e36: 6 total, 5 up, 6 in 2024-03-27T06:37:42.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:37:42.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:42 smithi016 ceph-mon[31908]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:42.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:42 smithi016 ceph-mon[31908]: from='client.? 172.21.15.170:0/887240035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4efed7-91d9-4adc-881a-ce3f44f80813"}]: dispatch 2024-03-27T06:37:42.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:42 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4efed7-91d9-4adc-881a-ce3f44f80813"}]: dispatch 2024-03-27T06:37:42.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:42 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f4efed7-91d9-4adc-881a-ce3f44f80813"}]': finished 2024-03-27T06:37:42.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:42 smithi016 ceph-mon[31908]: osdmap e36: 6 total, 5 up, 6 in 2024-03-27T06:37:42.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:37:42.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:42 smithi156 ceph-mon[36261]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:42.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:42 smithi156 ceph-mon[36261]: from='client.? 172.21.15.170:0/887240035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4efed7-91d9-4adc-881a-ce3f44f80813"}]: dispatch 2024-03-27T06:37:42.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:42 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f4efed7-91d9-4adc-881a-ce3f44f80813"}]: dispatch 2024-03-27T06:37:42.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:42 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f4efed7-91d9-4adc-881a-ce3f44f80813"}]': finished 2024-03-27T06:37:42.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:42 smithi156 ceph-mon[36261]: osdmap e36: 6 total, 5 up, 6 in 2024-03-27T06:37:42.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:37:43.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:43 smithi170 ceph-mon[36438]: from='client.? 172.21.15.170:0/541895749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:37:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:43 smithi016 ceph-mon[31908]: from='client.? 172.21.15.170:0/541895749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:37:43.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:43 smithi156 ceph-mon[36261]: from='client.? 172.21.15.170:0/541895749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:37:44.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:44 smithi156 ceph-mon[36261]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:44.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:44 smithi170 ceph-mon[36438]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:44.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:44 smithi016 ceph-mon[31908]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:46.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:46 smithi170 ceph-mon[36438]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:46.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:46 smithi016 ceph-mon[31908]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:46.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:46 smithi156 ceph-mon[36261]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:48.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:48 smithi170 ceph-mon[36438]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:48.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:48 smithi016 ceph-mon[31908]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:48.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:48 smithi156 ceph-mon[36261]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:50.721 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:50 smithi170 ceph-mon[36438]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:50.721 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-27T06:37:50.721 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:50.721 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:50 smithi170 ceph-mon[36438]: Deploying daemon osd.5 on smithi170 2024-03-27T06:37:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:50 smithi016 ceph-mon[31908]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-27T06:37:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:50 smithi016 ceph-mon[31908]: Deploying daemon osd.5 on smithi170 2024-03-27T06:37:50.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:50 smithi156 ceph-mon[36261]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:50.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-27T06:37:50.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:50.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:50 smithi156 ceph-mon[36261]: Deploying daemon osd.5 on smithi170 2024-03-27T06:37:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:52 smithi016 ceph-mon[31908]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:52.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:52 smithi170 ceph-mon[36438]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:52.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:52 smithi156 ceph-mon[36261]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:54.840 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:54 smithi170 ceph-mon[36438]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:54.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:54 smithi016 ceph-mon[31908]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:54.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:54 smithi156 ceph-mon[36261]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:56.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:56.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:56.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:56.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:56.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:56.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:56.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:56.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:37:56.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:57.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:57 smithi170 ceph-mon[36438]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:57.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:57 smithi016 ceph-mon[31908]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:57.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:57 smithi156 ceph-mon[36261]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:59.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:59 smithi170 ceph-mon[36438]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:59.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:59.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:59.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:59.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:59.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:37:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:59.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.389+0000 7ff4377fe700 1 -- 172.21.15.170:0/3425004238 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff4480feb80 con 0x7ff4300a36a0 2024-03-27T06:37:59.383 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 5 on host 'smithi170' 2024-03-27T06:37:59.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 -- 172.21.15.170:0/3425004238 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4300a36a0 msgr2=0x7ff4300a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:59.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4300a36a0 0x7ff4300a5b60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4481025d0 tx=0x7ff43800a3b0 comp rx=0 tx=0).stop 2024-03-27T06:37:59.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 -- 172.21.15.170:0/3425004238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff448103110 msgr2=0x7ff44819fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:37:59.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff448103110 0x7ff44819fe90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b9a0 tx=0x7ff44000bcb0 comp rx=0 tx=0).stop 2024-03-27T06:37:59.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 -- 172.21.15.170:0/3425004238 shutdown_connections 2024-03-27T06:37:59.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff4300a36a0 0x7ff4300a5b60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:59.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff448103110 0x7ff44819fe90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:59.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff448101570 0x7ff448198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:59.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 --2- 172.21.15.170:0/3425004238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448102770 0x7ff448198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:37:59.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 -- 172.21.15.170:0/3425004238 >> 172.21.15.170:0/3425004238 conn(0x7ff4480fcb30 msgr2=0x7ff4480fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:37:59.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 -- 172.21.15.170:0/3425004238 shutdown_connections 2024-03-27T06:37:59.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:37:59.391+0000 7ff44cd87700 1 -- 172.21.15.170:0/3425004238 wait complete. 2024-03-27T06:37:59.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:59 smithi156 ceph-mon[36261]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:59.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:59.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:59.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:59.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:59.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:37:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:59 smithi016 ceph-mon[31908]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:37:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:37:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:37:59.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:37:59.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:37:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:00.054 DEBUG:teuthology.orchestra.run.smithi170:osd.5> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.5.service 2024-03-27T06:38:00.056 INFO:tasks.cephadm:Deploying osd.6 on smithi016 with /dev/vg_nvme/lv_2... 2024-03-27T06:38:00.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-27T06:38:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:38:00.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:00 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:00.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:00 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:00.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:00 smithi170 ceph-mon[36438]: from='osd.5 [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T06:38:00.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:00 smithi170 ceph-mon[36438]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T06:38:00.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:00 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:00.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:00 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:00.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:00 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:00.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:00 smithi016 ceph-mon[31908]: from='osd.5 [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T06:38:00.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:00 smithi016 ceph-mon[31908]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T06:38:00.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:00 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:00.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:00 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:00.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:00 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:00.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:00 smithi156 ceph-mon[36261]: from='osd.5 [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T06:38:00.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:00 smithi156 ceph-mon[36261]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-27T06:38:00.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:00 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:01.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:01 smithi016 ceph-mon[31908]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:38:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:01 smithi016 ceph-mon[31908]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-27T06:38:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:01 smithi016 ceph-mon[31908]: osdmap e37: 6 total, 5 up, 6 in 2024-03-27T06:38:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:01 smithi016 ceph-mon[31908]: from='osd.5 [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:38:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:01 smithi016 ceph-mon[31908]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:38:01.623 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:01 smithi156 ceph-mon[36261]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:38:01.623 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:01 smithi156 ceph-mon[36261]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-27T06:38:01.623 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:01 smithi156 ceph-mon[36261]: osdmap e37: 6 total, 5 up, 6 in 2024-03-27T06:38:01.623 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:01.623 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:01 smithi156 ceph-mon[36261]: from='osd.5 [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:38:01.623 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:01 smithi156 ceph-mon[36261]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:38:01.712 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:01 smithi170 ceph-mon[36438]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:38:01.712 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:01 smithi170 ceph-mon[36438]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-27T06:38:01.712 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:01 smithi170 ceph-mon[36438]: osdmap e37: 6 total, 5 up, 6 in 2024-03-27T06:38:01.712 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:01.712 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:01 smithi170 ceph-mon[36438]: from='osd.5 [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:38:01.712 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:01 smithi170 ceph-mon[36438]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:38:01.713 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:38:01 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:38:01.369+0000 7f7eaa637700 -1 osd.5 0 waiting for initial osdmap 2024-03-27T06:38:01.713 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:38:01 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:38:01.378+0000 7f7ea642a700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:38:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-27T06:38:02.452 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-27T06:38:02.452 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-03-27T06:38:02.452 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-03-27T06:38:02.452 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00834966 s, 1.3 GB/s 2024-03-27T06:38:02.452 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 2024-03-27T06:38:02.452 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-03-27T06:38:02.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:02 smithi156 ceph-mon[36261]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:38:02.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:02 smithi156 ceph-mon[36261]: osdmap e38: 6 total, 5 up, 6 in 2024-03-27T06:38:02.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:02 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:02.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:02 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:02.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:02 smithi156 ceph-mon[36261]: osd.5 [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882] boot 2024-03-27T06:38:02.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:02 smithi156 ceph-mon[36261]: osdmap e39: 6 total, 6 up, 6 in 2024-03-27T06:38:02.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:02 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:02.835 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:02 smithi170 ceph-mon[36438]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:38:02.835 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:02 smithi170 ceph-mon[36438]: osdmap e38: 6 total, 5 up, 6 in 2024-03-27T06:38:02.835 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:02 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:02.836 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:02 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:02.836 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:02 smithi170 ceph-mon[36438]: osd.5 [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882] boot 2024-03-27T06:38:02.836 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:02 smithi170 ceph-mon[36438]: osdmap e39: 6 total, 6 up, 6 in 2024-03-27T06:38:02.836 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:02 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:02.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:02 smithi016 ceph-mon[31908]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:38:02.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:02 smithi016 ceph-mon[31908]: osdmap e38: 6 total, 5 up, 6 in 2024-03-27T06:38:02.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:02 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:02.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:02 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:02.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:02 smithi016 ceph-mon[31908]: osd.5 [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882] boot 2024-03-27T06:38:02.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:02 smithi016 ceph-mon[31908]: osdmap e39: 6 total, 6 up, 6 in 2024-03-27T06:38:02.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:02 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:38:03.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-03-27T06:38:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:38:03.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:38:03.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:38:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:38:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:03 smithi170 ceph-mon[36438]: osdmap e40: 6 total, 6 up, 6 in 2024-03-27T06:38:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:38:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:38:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:38:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:03.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:03 smithi016 ceph-mon[31908]: osdmap e40: 6 total, 6 up, 6 in 2024-03-27T06:38:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:38:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:38:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 158 MiB used, 447 GiB / 447 GiB avail 2024-03-27T06:38:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:03.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:03.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:03.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:03 smithi156 ceph-mon[36261]: osdmap e40: 6 total, 6 up, 6 in 2024-03-27T06:38:04.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:04 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi170 2024-03-27T06:38:04.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:04 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi170 to 8540M 2024-03-27T06:38:04.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:04 smithi156 ceph-mon[36261]: osdmap e41: 6 total, 6 up, 6 in 2024-03-27T06:38:04.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:04 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi170 2024-03-27T06:38:04.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:04 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi170 to 8540M 2024-03-27T06:38:04.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:04 smithi016 ceph-mon[31908]: osdmap e41: 6 total, 6 up, 6 in 2024-03-27T06:38:04.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:04 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi170 2024-03-27T06:38:04.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:04 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi170 to 8540M 2024-03-27T06:38:04.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:04 smithi170 ceph-mon[36438]: osdmap e41: 6 total, 6 up, 6 in 2024-03-27T06:38:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.311+0000 7f6cca57b700 1 -- 172.21.15.16:0/2890562313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc41031a0 msgr2=0x7f6cc4105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.311+0000 7f6cca57b700 1 --2- 172.21.15.16:0/2890562313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc41031a0 0x7f6cc4105590 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009990 tx=0x7f6cb4009ca0 comp rx=0 tx=0).stop 2024-03-27T06:38:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.311+0000 7f6cca57b700 1 -- 172.21.15.16:0/2890562313 shutdown_connections 2024-03-27T06:38:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.311+0000 7f6cca57b700 1 --2- 172.21.15.16:0/2890562313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cc4108400 0x7f6cc410a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.311+0000 7f6cca57b700 1 --2- 172.21.15.16:0/2890562313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc41031a0 0x7f6cc4105590 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.311+0000 7f6cca57b700 1 --2- 172.21.15.16:0/2890562313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cc4105ad0 0x7f6cc4107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.311+0000 7f6cca57b700 1 -- 172.21.15.16:0/2890562313 >> 172.21.15.16:0/2890562313 conn(0x7f6cc40fcbd0 msgr2=0x7f6cc40feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.311+0000 7f6cca57b700 1 -- 172.21.15.16:0/2890562313 shutdown_connections 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.311+0000 7f6cca57b700 1 -- 172.21.15.16:0/2890562313 wait complete. 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.311+0000 7f6cca57b700 1 Processor -- start 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.311+0000 7f6cca57b700 1 -- start start 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cca57b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cc41031a0 0x7f6cc41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cca57b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cc4105ad0 0x7f6cc4198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cca57b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4108400 0x7f6cc419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cca57b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc410e3b0 con 0x7f6cc4108400 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cca57b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc410e230 con 0x7f6cc41031a0 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cca57b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc410e530 con 0x7f6cc4105ad0 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc8b18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4108400 0x7f6cc419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc8b18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4108400 0x7f6cc419fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42304/0 (socket says 172.21.15.16:42304) 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc8b18700 1 -- 172.21.15.16:0/559872557 learned_addr learned my addr 172.21.15.16:0/559872557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc3fff700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cc41031a0 0x7f6cc41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc37fe700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cc4105ad0 0x7f6cc4198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc8b18700 1 -- 172.21.15.16:0/559872557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cc4105ad0 msgr2=0x7f6cc4198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc8b18700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cc4105ad0 0x7f6cc4198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc8b18700 1 -- 172.21.15.16:0/559872557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cc41031a0 msgr2=0x7f6cc41988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc8b18700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cc41031a0 0x7f6cc41988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc8b18700 1 -- 172.21.15.16:0/559872557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc41a0530 con 0x7f6cc4108400 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc8b18700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4108400 0x7f6cc419fe90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc007d70 tx=0x7f6cbc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc17fa700 1 -- 172.21.15.16:0/559872557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc016020 con 0x7f6cc4108400 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.312+0000 7f6cc17fa700 1 -- 172.21.15.16:0/559872557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6cbc00ac50 con 0x7f6cc4108400 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.313+0000 7f6cca57b700 1 -- 172.21.15.16:0/559872557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc41a0820 con 0x7f6cc4108400 2024-03-27T06:38:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.313+0000 7f6cc17fa700 1 -- 172.21.15.16:0/559872557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc0115e0 con 0x7f6cc4108400 2024-03-27T06:38:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.313+0000 7f6cca57b700 1 -- 172.21.15.16:0/559872557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc41a0d10 con 0x7f6cc4108400 2024-03-27T06:38:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.313+0000 7f6cca57b700 1 -- 172.21.15.16:0/559872557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc4069300 con 0x7f6cc4108400 2024-03-27T06:38:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.315+0000 7f6cc17fa700 1 -- 172.21.15.16:0/559872557 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f6cbc019020 con 0x7f6cc4108400 2024-03-27T06:38:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.315+0000 7f6cc17fa700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6cac0a36a0 0x7f6cac0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.316+0000 7f6cc3fff700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6cac0a36a0 0x7f6cac0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.316+0000 7f6cc17fa700 1 -- 172.21.15.16:0/559872557 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f6cbc0cf140 con 0x7f6cc4108400 2024-03-27T06:38:05.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.316+0000 7f6cc3fff700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6cac0a36a0 0x7f6cac0a5b60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400b5c0 tx=0x7f6cb4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:38:05.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.319+0000 7f6cc17fa700 1 -- 172.21.15.16:0/559872557 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f6cbc020020 con 0x7f6cc4108400 2024-03-27T06:38:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:05.538+0000 7f6cca57b700 1 -- 172.21.15.16:0/559872557 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6cc4062a80 con 0x7f6cac0a36a0 2024-03-27T06:38:05.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:05 smithi156 ceph-mon[36261]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:05.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:05 smithi170 ceph-mon[36438]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:05.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:05 smithi016 ceph-mon[31908]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:06.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:06 smithi016 ceph-mon[31908]: osdmap e42: 6 total, 6 up, 6 in 2024-03-27T06:38:06.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:06 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:38:06.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:06 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:38:06.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:06 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:06.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:06 smithi156 ceph-mon[36261]: osdmap e42: 6 total, 6 up, 6 in 2024-03-27T06:38:06.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:06 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:38:06.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:06 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:38:06.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:06 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:06.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:06 smithi170 ceph-mon[36438]: osdmap e42: 6 total, 6 up, 6 in 2024-03-27T06:38:06.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:06 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:38:06.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:06 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:38:06.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:06 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:07.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:07 smithi016 ceph-mon[31908]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:07.497 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:07 smithi016 ceph-mon[31908]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:38:07.497 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:07 smithi016 ceph-mon[31908]: osdmap e43: 6 total, 6 up, 6 in 2024-03-27T06:38:07.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:07 smithi156 ceph-mon[36261]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:07.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:07 smithi156 ceph-mon[36261]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:38:07.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:07 smithi156 ceph-mon[36261]: osdmap e43: 6 total, 6 up, 6 in 2024-03-27T06:38:07.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:07 smithi170 ceph-mon[36438]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:07.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:07 smithi170 ceph-mon[36438]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:38:07.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:07 smithi170 ceph-mon[36438]: osdmap e43: 6 total, 6 up, 6 in 2024-03-27T06:38:08.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:08 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1429105575' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffbf31b8-a44e-4f81-819b-48213bf4cc84"}]: dispatch 2024-03-27T06:38:08.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:08 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1429105575' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffbf31b8-a44e-4f81-819b-48213bf4cc84"}]': finished 2024-03-27T06:38:08.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:08 smithi156 ceph-mon[36261]: osdmap e44: 7 total, 6 up, 7 in 2024-03-27T06:38:08.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:08 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:08.732 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:08 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1429105575' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffbf31b8-a44e-4f81-819b-48213bf4cc84"}]: dispatch 2024-03-27T06:38:08.732 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:08 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1429105575' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffbf31b8-a44e-4f81-819b-48213bf4cc84"}]': finished 2024-03-27T06:38:08.732 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:08 smithi016 ceph-mon[31908]: osdmap e44: 7 total, 6 up, 7 in 2024-03-27T06:38:08.732 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:08 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:08.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:08 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1429105575' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffbf31b8-a44e-4f81-819b-48213bf4cc84"}]: dispatch 2024-03-27T06:38:08.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:08 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1429105575' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffbf31b8-a44e-4f81-819b-48213bf4cc84"}]': finished 2024-03-27T06:38:08.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:08 smithi170 ceph-mon[36438]: osdmap e44: 7 total, 6 up, 7 in 2024-03-27T06:38:08.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:08 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:09.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:09 smithi156 ceph-mon[36261]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:09.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:09 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2262585333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:38:09.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:09 smithi170 ceph-mon[36438]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:09.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:09 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2262585333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:38:09.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:09 smithi016 ceph-mon[31908]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:09.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:09 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2262585333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:38:11.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:11 smithi156 ceph-mon[36261]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-27T06:38:11.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:11 smithi170 ceph-mon[36438]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-27T06:38:11.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:11 smithi016 ceph-mon[31908]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-27T06:38:13.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:13 smithi170 ceph-mon[36438]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-27T06:38:13.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:13 smithi156 ceph-mon[36261]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-27T06:38:13.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:13 smithi016 ceph-mon[31908]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-27T06:38:15.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:15 smithi016 ceph-mon[31908]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:38:15.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:15 smithi156 ceph-mon[36261]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:38:15.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:15 smithi170 ceph-mon[36438]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:38:16.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:16 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-27T06:38:16.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:16 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:16.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:16 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-27T06:38:16.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:16 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:16.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:16 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-27T06:38:16.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:16 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:17.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:17 smithi016 ceph-mon[31908]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-27T06:38:17.684 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:17 smithi016 ceph-mon[31908]: Deploying daemon osd.6 on smithi016 2024-03-27T06:38:17.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:17 smithi156 ceph-mon[36261]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-27T06:38:17.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:17 smithi156 ceph-mon[36261]: Deploying daemon osd.6 on smithi016 2024-03-27T06:38:17.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:17 smithi170 ceph-mon[36438]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-27T06:38:17.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:17 smithi170 ceph-mon[36438]: Deploying daemon osd.6 on smithi016 2024-03-27T06:38:19.551 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:19 smithi016 ceph-mon[31908]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T06:38:19.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:19 smithi156 ceph-mon[36261]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T06:38:19.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:19 smithi170 ceph-mon[36438]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T06:38:21.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:21 smithi016 ceph-mon[31908]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:21.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:21 smithi156 ceph-mon[36261]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:21.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:21 smithi170 ceph-mon[36438]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:22.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:22.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:22.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:22.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:22.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:23.688 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:23 smithi016 ceph-mon[31908]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:23.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:23 smithi156 ceph-mon[36261]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:23.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:23 smithi170 ceph-mon[36438]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:25.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:25 smithi016 ceph-mon[31908]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:25.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:25.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:25.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:25.566 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:25.567 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:25 smithi156 ceph-mon[36261]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:25.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:25 smithi170 ceph-mon[36438]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:25.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.949+0000 7f6cc17fa700 1 -- 172.21.15.16:0/559872557 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6cc4062a80 con 0x7f6cac0a36a0 2024-03-27T06:38:25.944 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 6 on host 'smithi016' 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 -- 172.21.15.16:0/559872557 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6cac0a36a0 msgr2=0x7f6cac0a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6cac0a36a0 0x7f6cac0a5b60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400b5c0 tx=0x7f6cb4005fb0 comp rx=0 tx=0).stop 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 -- 172.21.15.16:0/559872557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4108400 msgr2=0x7f6cc419fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4108400 0x7f6cc419fe90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc007d70 tx=0x7f6cbc00a340 comp rx=0 tx=0).stop 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 -- 172.21.15.16:0/559872557 shutdown_connections 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4108400 0x7f6cc419fe90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cc41031a0 0x7f6cc41988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6cac0a36a0 0x7f6cac0a5b60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 --2- 172.21.15.16:0/559872557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cc4105ad0 0x7f6cc4198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 -- 172.21.15.16:0/559872557 >> 172.21.15.16:0/559872557 conn(0x7f6cc40fcbd0 msgr2=0x7f6cc4103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 -- 172.21.15.16:0/559872557 shutdown_connections 2024-03-27T06:38:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:38:25.951+0000 7f6cca57b700 1 -- 172.21.15.16:0/559872557 wait complete. 2024-03-27T06:38:26.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:26 smithi016 ceph-mon[31908]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-27T06:38:26.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:26.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:26.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:26.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:26 smithi156 ceph-mon[36261]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-27T06:38:26.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:26.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:26.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:26.772 DEBUG:teuthology.orchestra.run.smithi016:osd.6> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.6.service 2024-03-27T06:38:26.774 INFO:tasks.cephadm:Deploying osd.7 on smithi156 with /dev/vg_nvme/lv_2... 2024-03-27T06:38:26.774 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-27T06:38:26.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:26 smithi170 ceph-mon[36438]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-27T06:38:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:27.023 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:38:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:27 smithi016 ceph-mon[31908]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:27 smithi016 ceph-mon[31908]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-27T06:38:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:27 smithi016 ceph-mon[31908]: osdmap e45: 7 total, 6 up, 7 in 2024-03-27T06:38:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:27 smithi016 ceph-mon[31908]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:38:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:27 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:27.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:38:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:38:27.430+0000 7f615fd53700 -1 osd.6 0 waiting for initial osdmap 2024-03-27T06:38:27.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:38:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:38:27.439+0000 7f615bb46700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:38:27.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:27 smithi156 ceph-mon[36261]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:27 smithi156 ceph-mon[36261]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-27T06:38:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:27 smithi156 ceph-mon[36261]: osdmap e45: 7 total, 6 up, 7 in 2024-03-27T06:38:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:27 smithi156 ceph-mon[36261]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:38:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:27 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:27.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:27 smithi170 ceph-mon[36438]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:27 smithi170 ceph-mon[36438]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-27T06:38:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:27 smithi170 ceph-mon[36438]: osdmap e45: 7 total, 6 up, 7 in 2024-03-27T06:38:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:27 smithi170 ceph-mon[36438]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:38:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:27 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:28.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:28 smithi156 ceph-mon[36261]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:38:28.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:28 smithi156 ceph-mon[36261]: osdmap e46: 7 total, 6 up, 7 in 2024-03-27T06:38:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:28.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:28 smithi170 ceph-mon[36438]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:38:28.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:28 smithi170 ceph-mon[36438]: osdmap e46: 7 total, 6 up, 7 in 2024-03-27T06:38:28.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:28.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:28 smithi016 ceph-mon[31908]: from='osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:38:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:28 smithi016 ceph-mon[31908]: osdmap e46: 7 total, 6 up, 7 in 2024-03-27T06:38:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:28.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:29.242 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-27T06:38:29.242 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-27T06:38:29.243 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-03-27T06:38:29.243 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-03-27T06:38:29.243 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00848347 s, 1.2 GB/s 2024-03-27T06:38:29.243 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-03-27T06:38:29.553 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:29 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:38:29.553 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:29 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:38:29.553 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:29 smithi156 ceph-mon[36261]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:29.553 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:29 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:29.553 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:29 smithi156 ceph-mon[36261]: osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547] boot 2024-03-27T06:38:29.553 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:29 smithi156 ceph-mon[36261]: osdmap e47: 7 total, 7 up, 7 in 2024-03-27T06:38:29.553 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:29 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:29.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:29 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:38:29.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:29 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:38:29.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:29 smithi016 ceph-mon[31908]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:29.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:29 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:29.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:29 smithi016 ceph-mon[31908]: osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547] boot 2024-03-27T06:38:29.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:29 smithi016 ceph-mon[31908]: osdmap e47: 7 total, 7 up, 7 in 2024-03-27T06:38:29.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:29 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:29.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:29 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:38:29.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:29 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:38:29.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:29 smithi170 ceph-mon[36438]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-27T06:38:29.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:29 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:29 smithi170 ceph-mon[36438]: osd.6 [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547] boot 2024-03-27T06:38:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:29 smithi170 ceph-mon[36438]: osdmap e47: 7 total, 7 up, 7 in 2024-03-27T06:38:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:29 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:38:29.975 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_2 2024-03-27T06:38:30.229 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:38:30.632 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:30 smithi156 ceph-mon[36261]: osdmap e48: 7 total, 7 up, 7 in 2024-03-27T06:38:30.633 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:30 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:30.633 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:30 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:30.633 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:30 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:30.633 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:30 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:30.633 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:30 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:30.633 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:30 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:30.633 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:30 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:30.633 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:30 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:30.633 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:30 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:30.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:30 smithi170 ceph-mon[36438]: osdmap e48: 7 total, 7 up, 7 in 2024-03-27T06:38:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:30 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:30 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:30 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:30 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:30 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:30 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:30 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:30 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:30 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:30 smithi016 ceph-mon[31908]: osdmap e48: 7 total, 7 up, 7 in 2024-03-27T06:38:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:30 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:30 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:30 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:30 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:30 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:30 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:30 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:30 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:30 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:31.682 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:31 smithi156 ceph-mon[36261]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:31.682 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:31 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi016 2024-03-27T06:38:31.682 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:31 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi016 to 5697M 2024-03-27T06:38:31.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:31 smithi170 ceph-mon[36438]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:31.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:31 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi016 2024-03-27T06:38:31.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:31 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi016 to 5697M 2024-03-27T06:38:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:31 smithi016 ceph-mon[31908]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:31 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi016 2024-03-27T06:38:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:31 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi016 to 5697M 2024-03-27T06:38:31.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.890+0000 7f8d2a002700 1 -- 172.21.15.156:0/950388812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d24102670 msgr2=0x7f8d24102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:31.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.890+0000 7f8d2a002700 1 --2- 172.21.15.156:0/950388812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d24102670 0x7f8d24102a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8d140099c0 tx=0x7f8d14009cd0 comp rx=0 tx=0).stop 2024-03-27T06:38:31.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.890+0000 7f8d2a002700 1 -- 172.21.15.156:0/950388812 shutdown_connections 2024-03-27T06:38:31.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.890+0000 7f8d2a002700 1 --2- 172.21.15.156:0/950388812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24104210 0x7f8d2410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:31.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.890+0000 7f8d2a002700 1 --2- 172.21.15.156:0/950388812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d24102670 0x7f8d24102a90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:31.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.890+0000 7f8d2a002700 1 --2- 172.21.15.156:0/950388812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d24103870 0x7f8d24103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:31.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.890+0000 7f8d2a002700 1 -- 172.21.15.156:0/950388812 >> 172.21.15.156:0/950388812 conn(0x7f8d240fdc50 msgr2=0x7f8d24100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:38:31.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.890+0000 7f8d2a002700 1 -- 172.21.15.156:0/950388812 shutdown_connections 2024-03-27T06:38:31.890 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.890+0000 7f8d2a002700 1 -- 172.21.15.156:0/950388812 wait complete. 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.890+0000 7f8d2a002700 1 Processor -- start 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.890+0000 7f8d2a002700 1 -- start start 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d2a002700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d24102670 0x7f8d24198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d2a002700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d24103870 0x7f8d24198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d2a002700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24104210 0x7f8d2419ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d2a002700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2410e370 con 0x7f8d24103870 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d2a002700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2410e1f0 con 0x7f8d24102670 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d2a002700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2410e4f0 con 0x7f8d24104210 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d237fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d24102670 0x7f8d24198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d237fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d24102670 0x7f8d24198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58512/0 (socket says 172.21.15.156:58512) 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d237fe700 1 -- 172.21.15.156:0/1814977806 learned_addr learned my addr 172.21.15.156:0/1814977806 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-27T06:38:31.891 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d23fff700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24104210 0x7f8d2419ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d237fe700 1 -- 172.21.15.156:0/1814977806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24104210 msgr2=0x7f8d2419ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d237fe700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24104210 0x7f8d2419ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d237fe700 1 -- 172.21.15.156:0/1814977806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d24103870 msgr2=0x7f8d24198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d1affd700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d24103870 0x7f8d24198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d237fe700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d24103870 0x7f8d24198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d237fe700 1 -- 172.21.15.156:0/1814977806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d241a05b0 con 0x7f8d24102670 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d237fe700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d24102670 0x7f8d24198950 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009f50 tx=0x7f8d14009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d217fa700 1 -- 172.21.15.156:0/1814977806 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14027070 con 0x7f8d24102670 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.891+0000 7f8d2a002700 1 -- 172.21.15.156:0/1814977806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d241a0840 con 0x7f8d24102670 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.892+0000 7f8d217fa700 1 -- 172.21.15.156:0/1814977806 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8d1400bda0 con 0x7f8d24102670 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.892+0000 7f8d217fa700 1 -- 172.21.15.156:0/1814977806 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400f460 con 0x7f8d24102670 2024-03-27T06:38:31.892 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.892+0000 7f8d2a002700 1 -- 172.21.15.156:0/1814977806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d241a0d70 con 0x7f8d24102670 2024-03-27T06:38:31.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.892+0000 7f8d2a002700 1 -- 172.21.15.156:0/1814977806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d24069300 con 0x7f8d24102670 2024-03-27T06:38:31.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.894+0000 7f8d217fa700 1 -- 172.21.15.156:0/1814977806 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f8d1402a070 con 0x7f8d24102670 2024-03-27T06:38:31.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.894+0000 7f8d217fa700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100a36a0 0x7f8d100a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:31.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.894+0000 7f8d217fa700 1 -- 172.21.15.156:0/1814977806 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f8d140cff70 con 0x7f8d24102670 2024-03-27T06:38:31.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.894+0000 7f8d1affd700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100a36a0 0x7f8d100a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:31.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.895+0000 7f8d1affd700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100a36a0 0x7f8d100a5b60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005950 tx=0x7f8d0c005cf0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:38:31.893 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:31.898+0000 7f8d217fa700 1 -- 172.21.15.156:0/1814977806 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8d14013070 con 0x7f8d24102670 2024-03-27T06:38:32.094 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:32.100+0000 7f8d2a002700 1 -- 172.21.15.156:0/1814977806 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8d24062b40 con 0x7f8d100a36a0 2024-03-27T06:38:32.704 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:32 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:38:32.704 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:32 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:38:32.704 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:32 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:32.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:32 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:38:32.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:32 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:38:32.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:32 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:32.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:32 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:38:32.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:32 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:38:32.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:32 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:33.710 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:33 smithi156 ceph-mon[36261]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:33.711 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:33 smithi156 ceph-mon[36261]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:38:33.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:33 smithi170 ceph-mon[36438]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:33.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:33 smithi170 ceph-mon[36438]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:38:33.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:33 smithi016 ceph-mon[31908]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:33.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:33 smithi016 ceph-mon[31908]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:38:35.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:35 smithi156 ceph-mon[36261]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:35.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:35 smithi156 ceph-mon[36261]: from='client.? 172.21.15.156:0/2324982848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb83f843-12b4-4b70-9a85-dd2ca56ed9d8"}]: dispatch 2024-03-27T06:38:35.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:35 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb83f843-12b4-4b70-9a85-dd2ca56ed9d8"}]: dispatch 2024-03-27T06:38:35.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:35 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb83f843-12b4-4b70-9a85-dd2ca56ed9d8"}]': finished 2024-03-27T06:38:35.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:35 smithi156 ceph-mon[36261]: osdmap e49: 8 total, 7 up, 8 in 2024-03-27T06:38:35.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:35.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:35 smithi170 ceph-mon[36438]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:35.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:35 smithi170 ceph-mon[36438]: from='client.? 172.21.15.156:0/2324982848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb83f843-12b4-4b70-9a85-dd2ca56ed9d8"}]: dispatch 2024-03-27T06:38:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:35 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb83f843-12b4-4b70-9a85-dd2ca56ed9d8"}]: dispatch 2024-03-27T06:38:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:35 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb83f843-12b4-4b70-9a85-dd2ca56ed9d8"}]': finished 2024-03-27T06:38:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:35 smithi170 ceph-mon[36438]: osdmap e49: 8 total, 7 up, 8 in 2024-03-27T06:38:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:35.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:35 smithi016 ceph-mon[31908]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:35.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:35 smithi016 ceph-mon[31908]: from='client.? 172.21.15.156:0/2324982848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb83f843-12b4-4b70-9a85-dd2ca56ed9d8"}]: dispatch 2024-03-27T06:38:35.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:35 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb83f843-12b4-4b70-9a85-dd2ca56ed9d8"}]: dispatch 2024-03-27T06:38:35.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:35 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb83f843-12b4-4b70-9a85-dd2ca56ed9d8"}]': finished 2024-03-27T06:38:35.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:35 smithi016 ceph-mon[31908]: osdmap e49: 8 total, 7 up, 8 in 2024-03-27T06:38:35.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:36.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:36 smithi156 ceph-mon[36261]: from='client.? 172.21.15.156:0/1560995728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:38:36.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:36 smithi170 ceph-mon[36438]: from='client.? 172.21.15.156:0/1560995728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:38:36.860 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:36 smithi016 ceph-mon[31908]: from='client.? 172.21.15.156:0/1560995728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:38:37.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:37 smithi156 ceph-mon[36261]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:37.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:37 smithi170 ceph-mon[36438]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:37.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:37 smithi016 ceph-mon[31908]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:39.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:39 smithi156 ceph-mon[36261]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:39.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:39 smithi170 ceph-mon[36438]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:39.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:39 smithi016 ceph-mon[31908]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:41.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:41 smithi156 ceph-mon[36261]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:41.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:41 smithi170 ceph-mon[36438]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:41.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:41 smithi016 ceph-mon[31908]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:42.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-27T06:38:42.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:42.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-27T06:38:42.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:42.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-27T06:38:42.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:43 smithi156 ceph-mon[36261]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:43 smithi156 ceph-mon[36261]: Deploying daemon osd.7 on smithi156 2024-03-27T06:38:43.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:43 smithi170 ceph-mon[36438]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:43.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:43 smithi170 ceph-mon[36438]: Deploying daemon osd.7 on smithi156 2024-03-27T06:38:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:43 smithi016 ceph-mon[31908]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:43 smithi016 ceph-mon[31908]: Deploying daemon osd.7 on smithi156 2024-03-27T06:38:45.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:45 smithi156 ceph-mon[36261]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:45.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:45 smithi170 ceph-mon[36438]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:45.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:45 smithi016 ceph-mon[31908]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:47.681 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:47 smithi156 ceph-mon[36261]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:47.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:47 smithi170 ceph-mon[36438]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:47 smithi016 ceph-mon[31908]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:49.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:48 smithi170 ceph-mon[36438]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:49.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:48 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:49.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:48 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:49.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:48 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:49.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:48 smithi016 ceph-mon[31908]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:49.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:48 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:49.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:48 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:49.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:48 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:49.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:48 smithi156 ceph-mon[36261]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:49.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:48 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:49.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:48 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:49.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:48 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:51.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:50 smithi170 ceph-mon[36438]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:51.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:50 smithi016 ceph-mon[31908]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:51.215 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:50 smithi156 ceph-mon[36261]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:51.563 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.569+0000 7f8d217fa700 1 -- 172.21.15.156:0/1814977806 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8d24062b40 con 0x7f8d100a36a0 2024-03-27T06:38:51.564 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 7 on host 'smithi156' 2024-03-27T06:38:51.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 -- 172.21.15.156:0/1814977806 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100a36a0 msgr2=0x7f8d100a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:51.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100a36a0 0x7f8d100a5b60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005950 tx=0x7f8d0c005cf0 comp rx=0 tx=0).stop 2024-03-27T06:38:51.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 -- 172.21.15.156:0/1814977806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d24102670 msgr2=0x7f8d24198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:51.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d24102670 0x7f8d24198950 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009f50 tx=0x7f8d14009580 comp rx=0 tx=0).stop 2024-03-27T06:38:51.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 -- 172.21.15.156:0/1814977806 shutdown_connections 2024-03-27T06:38:51.566 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8d100a36a0 0x7f8d100a5b60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:51.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24104210 0x7f8d2419ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:51.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d24102670 0x7f8d24198950 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:51.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 --2- 172.21.15.156:0/1814977806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d24103870 0x7f8d24198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:51.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 -- 172.21.15.156:0/1814977806 >> 172.21.15.156:0/1814977806 conn(0x7f8d240fdc50 msgr2=0x7f8d24106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:38:51.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 -- 172.21.15.156:0/1814977806 shutdown_connections 2024-03-27T06:38:51.567 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:38:51.572+0000 7f8d2a002700 1 -- 172.21.15.156:0/1814977806 wait complete. 2024-03-27T06:38:52.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:51 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:51 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:51 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:52.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:51 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:52.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:51 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:51 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:51 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:51 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:52.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:51 smithi170 ceph-mon[36438]: from='osd.7 [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T06:38:52.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:51 smithi170 ceph-mon[36438]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T06:38:52.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:51 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:51 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:51 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:52.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:51 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:52.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:51 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:51 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:51 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:51 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:52.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:51 smithi016 ceph-mon[31908]: from='osd.7 [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T06:38:52.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:51 smithi016 ceph-mon[31908]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T06:38:52.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:51 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:51 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:51 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:52.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:51 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:52.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:51 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:51 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:51 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:52.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:51 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:38:52.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:51 smithi156 ceph-mon[36261]: from='osd.7 [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T06:38:52.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:51 smithi156 ceph-mon[36261]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-27T06:38:52.245 DEBUG:teuthology.orchestra.run.smithi156:osd.7> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.7.service 2024-03-27T06:38:52.247 INFO:tasks.cephadm:Deploying osd.8 on smithi170 with /dev/vg_nvme/lv_2... 2024-03-27T06:38:52.248 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-27T06:38:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:38:52.908 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:52 smithi170 ceph-mon[36438]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:52.908 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:52 smithi170 ceph-mon[36438]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-27T06:38:52.908 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:52 smithi170 ceph-mon[36438]: osdmap e50: 8 total, 7 up, 8 in 2024-03-27T06:38:52.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:52 smithi170 ceph-mon[36438]: from='osd.7 [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:38:52.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:52 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:52.909 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:52 smithi170 ceph-mon[36438]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:38:53.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:52 smithi016 ceph-mon[31908]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:53.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:52 smithi016 ceph-mon[31908]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-27T06:38:53.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:52 smithi016 ceph-mon[31908]: osdmap e50: 8 total, 7 up, 8 in 2024-03-27T06:38:53.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:52 smithi016 ceph-mon[31908]: from='osd.7 [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:38:53.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:52 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:53.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:52 smithi016 ceph-mon[31908]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:38:53.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:52 smithi156 ceph-mon[36261]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:53.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:52 smithi156 ceph-mon[36261]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-27T06:38:53.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:52 smithi156 ceph-mon[36261]: osdmap e50: 8 total, 7 up, 8 in 2024-03-27T06:38:53.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:52 smithi156 ceph-mon[36261]: from='osd.7 [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:38:53.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:52 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:53.178 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:52 smithi156 ceph-mon[36261]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:38:53.882 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:38:53 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:38:53.568+0000 7f3a96843700 -1 osd.7 0 waiting for initial osdmap 2024-03-27T06:38:53.883 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:38:53 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:38:53.577+0000 7f3a92636700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:38:54.711 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-27T06:38:54.712 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-27T06:38:54.712 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-03-27T06:38:54.712 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-03-27T06:38:54.712 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00828556 s, 1.3 GB/s 2024-03-27T06:38:54.712 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-03-27T06:38:54.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:54 smithi170 ceph-mon[36438]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:54.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:54 smithi170 ceph-mon[36438]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:38:54.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:54 smithi170 ceph-mon[36438]: osdmap e51: 8 total, 7 up, 8 in 2024-03-27T06:38:54.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:54 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:54.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:54 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:54.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:54 smithi016 ceph-mon[31908]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:54.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:54 smithi016 ceph-mon[31908]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:38:54.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:54 smithi016 ceph-mon[31908]: osdmap e51: 8 total, 7 up, 8 in 2024-03-27T06:38:54.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:54 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:54.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:54 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:54.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:54 smithi156 ceph-mon[36261]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-27T06:38:54.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:54 smithi156 ceph-mon[36261]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:38:54.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:54 smithi156 ceph-mon[36261]: osdmap e51: 8 total, 7 up, 8 in 2024-03-27T06:38:54.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:54 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:54.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:54 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:55.451 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi170:vg_nvme/lv_2 2024-03-27T06:38:55.692 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:38:55.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:38:55.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:38:55.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: osd.7 [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476] boot 2024-03-27T06:38:55.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: osdmap e52: 8 total, 8 up, 8 in 2024-03-27T06:38:55.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:55.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: osdmap e53: 8 total, 8 up, 8 in 2024-03-27T06:38:55.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi156 2024-03-27T06:38:55.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:55.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:55.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:55.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:55.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:55.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi156 to 5693M 2024-03-27T06:38:55.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:55.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:55.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:55.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:55 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:55.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:38:55.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:38:55.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: osd.7 [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476] boot 2024-03-27T06:38:55.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: osdmap e52: 8 total, 8 up, 8 in 2024-03-27T06:38:55.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:55.816 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: osdmap e53: 8 total, 8 up, 8 in 2024-03-27T06:38:55.816 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi156 2024-03-27T06:38:55.816 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:55.816 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:55.816 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:55.816 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:55.817 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:55.817 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi156 to 5693M 2024-03-27T06:38:55.817 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:55.817 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:55.817 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:55.817 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:55 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:55.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:38:55.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:38:55.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: osd.7 [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476] boot 2024-03-27T06:38:55.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: osdmap e52: 8 total, 8 up, 8 in 2024-03-27T06:38:55.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:38:55.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: osdmap e53: 8 total, 8 up, 8 in 2024-03-27T06:38:55.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi156 2024-03-27T06:38:55.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:55.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:55.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:55.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:55.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:38:55.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi156 to 5693M 2024-03-27T06:38:55.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:55.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:55.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:38:55.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:55 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:38:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.441+0000 7f6c7977c700 1 -- 172.21.15.170:0/4018554896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c74105a30 msgr2=0x7f6c74107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.441+0000 7f6c7977c700 1 --2- 172.21.15.170:0/4018554896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c74105a30 0x7f6c74107e20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009990 tx=0x7f6c68009ca0 comp rx=0 tx=0).stop 2024-03-27T06:38:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.441+0000 7f6c7977c700 1 -- 172.21.15.170:0/4018554896 shutdown_connections 2024-03-27T06:38:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.441+0000 7f6c7977c700 1 --2- 172.21.15.170:0/4018554896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c74108360 0x7f6c7410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.441+0000 7f6c7977c700 1 --2- 172.21.15.170:0/4018554896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c74103100 0x7f6c741054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.441+0000 7f6c7977c700 1 --2- 172.21.15.170:0/4018554896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c74105a30 0x7f6c74107e20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:57.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.441+0000 7f6c7977c700 1 -- 172.21.15.170:0/4018554896 >> 172.21.15.170:0/4018554896 conn(0x7f6c740fcb10 msgr2=0x7f6c740fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:38:57.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.441+0000 7f6c7977c700 1 -- 172.21.15.170:0/4018554896 shutdown_connections 2024-03-27T06:38:57.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.441+0000 7f6c7977c700 1 -- 172.21.15.170:0/4018554896 wait complete. 2024-03-27T06:38:57.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.442+0000 7f6c7977c700 1 Processor -- start 2024-03-27T06:38:57.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.442+0000 7f6c7977c700 1 -- start start 2024-03-27T06:38:57.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.442+0000 7f6c7977c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c74103100 0x7f6c74198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:57.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.442+0000 7f6c7977c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c74105a30 0x7f6c74198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:57.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.442+0000 7f6c7977c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c74108360 0x7f6c7419ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:57.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c7977c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410e3f0 con 0x7f6c74105a30 2024-03-27T06:38:57.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c7977c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410e270 con 0x7f6c74103100 2024-03-27T06:38:57.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c7977c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410e570 con 0x7f6c74108360 2024-03-27T06:38:57.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c737fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c74108360 0x7f6c7419ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:57.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c72ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c74103100 0x7f6c74198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c737fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c74108360 0x7f6c7419ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57266/0 (socket says 172.21.15.170:57266) 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c737fe700 1 -- 172.21.15.170:0/1323769934 learned_addr learned my addr 172.21.15.170:0/1323769934 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c727fc700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c74105a30 0x7f6c74198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c737fe700 1 -- 172.21.15.170:0/1323769934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c74103100 msgr2=0x7f6c74198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c737fe700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c74103100 0x7f6c74198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c737fe700 1 -- 172.21.15.170:0/1323769934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c74105a30 msgr2=0x7f6c74198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c737fe700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c74105a30 0x7f6c74198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c737fe700 1 -- 172.21.15.170:0/1323769934 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c741a0660 con 0x7f6c74108360 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c737fe700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c74108360 0x7f6c7419ffc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00efc0 tx=0x7f6c6c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c5bfff700 1 -- 172.21.15.170:0/1323769934 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0092e0 con 0x7f6c74108360 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c5bfff700 1 -- 172.21.15.170:0/1323769934 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6c6c004bb0 con 0x7f6c74108360 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c7977c700 1 -- 172.21.15.170:0/1323769934 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c74100ef0 con 0x7f6c74108360 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c5bfff700 1 -- 172.21.15.170:0/1323769934 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c004280 con 0x7f6c74108360 2024-03-27T06:38:57.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.443+0000 7f6c7977c700 1 -- 172.21.15.170:0/1323769934 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c74101430 con 0x7f6c74108360 2024-03-27T06:38:57.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.444+0000 7f6c7977c700 1 -- 172.21.15.170:0/1323769934 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c74069300 con 0x7f6c74108360 2024-03-27T06:38:57.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.445+0000 7f6c5bfff700 1 -- 172.21.15.170:0/1323769934 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f6c6c01b030 con 0x7f6c74108360 2024-03-27T06:38:57.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.446+0000 7f6c5bfff700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6c5c0a36a0 0x7f6c5c0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:38:57.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.446+0000 7f6c5bfff700 1 -- 172.21.15.170:0/1323769934 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f6c6c0cf660 con 0x7f6c74108360 2024-03-27T06:38:57.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.446+0000 7f6c72ffd700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6c5c0a36a0 0x7f6c5c0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:38:57.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.447+0000 7f6c72ffd700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6c5c0a36a0 0x7f6c5c0a5b60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009c60 tx=0x7f6c64009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:38:57.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.450+0000 7f6c5bfff700 1 -- 172.21.15.170:0/1323769934 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f6c6c022020 con 0x7f6c74108360 2024-03-27T06:38:57.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:57 smithi170 ceph-mon[36438]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:38:57.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:57 smithi170 ceph-mon[36438]: osdmap e54: 8 total, 8 up, 8 in 2024-03-27T06:38:57.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:38:57.655+0000 7f6c7977c700 1 -- 172.21.15.170:0/1323769934 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6c74062b40 con 0x7f6c5c0a36a0 2024-03-27T06:38:57.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:57 smithi156 ceph-mon[36261]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:38:57.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:57 smithi156 ceph-mon[36261]: osdmap e54: 8 total, 8 up, 8 in 2024-03-27T06:38:57.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:57 smithi016 ceph-mon[31908]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:38:57.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:57 smithi016 ceph-mon[31908]: osdmap e54: 8 total, 8 up, 8 in 2024-03-27T06:38:58.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:58 smithi170 ceph-mon[36438]: osdmap e55: 8 total, 8 up, 8 in 2024-03-27T06:38:58.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:58 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:38:58.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:58 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:38:58.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:58 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:58.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:58 smithi156 ceph-mon[36261]: osdmap e55: 8 total, 8 up, 8 in 2024-03-27T06:38:58.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:58 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:38:58.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:58 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:38:58.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:58 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:58.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:58 smithi016 ceph-mon[31908]: osdmap e55: 8 total, 8 up, 8 in 2024-03-27T06:38:58.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:38:58.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:38:58.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:58 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:38:59.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:59 smithi170 ceph-mon[36438]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:38:59.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:59 smithi170 ceph-mon[36438]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:38:59.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:38:59 smithi170 ceph-mon[36438]: osdmap e56: 8 total, 8 up, 8 in 2024-03-27T06:38:59.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:59 smithi156 ceph-mon[36261]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:38:59.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:59 smithi156 ceph-mon[36261]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:38:59.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:38:59 smithi156 ceph-mon[36261]: osdmap e56: 8 total, 8 up, 8 in 2024-03-27T06:38:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:59 smithi016 ceph-mon[31908]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:38:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:59 smithi016 ceph-mon[31908]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:38:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:38:59 smithi016 ceph-mon[31908]: osdmap e56: 8 total, 8 up, 8 in 2024-03-27T06:39:00.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:00 smithi156 ceph-mon[36261]: from='client.? 172.21.15.170:0/827527744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4849c1e0-60fa-4333-b423-20b759447717"}]: dispatch 2024-03-27T06:39:00.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:00 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4849c1e0-60fa-4333-b423-20b759447717"}]: dispatch 2024-03-27T06:39:00.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:00 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4849c1e0-60fa-4333-b423-20b759447717"}]': finished 2024-03-27T06:39:00.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:00 smithi156 ceph-mon[36261]: osdmap e57: 9 total, 8 up, 9 in 2024-03-27T06:39:00.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:00 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:00.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:00 smithi170 ceph-mon[36438]: from='client.? 172.21.15.170:0/827527744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4849c1e0-60fa-4333-b423-20b759447717"}]: dispatch 2024-03-27T06:39:00.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:00 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4849c1e0-60fa-4333-b423-20b759447717"}]: dispatch 2024-03-27T06:39:00.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:00 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4849c1e0-60fa-4333-b423-20b759447717"}]': finished 2024-03-27T06:39:00.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:00 smithi170 ceph-mon[36438]: osdmap e57: 9 total, 8 up, 9 in 2024-03-27T06:39:00.798 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:00 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:00.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:00 smithi016 ceph-mon[31908]: from='client.? 172.21.15.170:0/827527744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4849c1e0-60fa-4333-b423-20b759447717"}]: dispatch 2024-03-27T06:39:00.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:00 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4849c1e0-60fa-4333-b423-20b759447717"}]: dispatch 2024-03-27T06:39:00.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:00 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4849c1e0-60fa-4333-b423-20b759447717"}]': finished 2024-03-27T06:39:00.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:00 smithi016 ceph-mon[31908]: osdmap e57: 9 total, 8 up, 9 in 2024-03-27T06:39:00.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:00 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:01.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:01 smithi156 ceph-mon[36261]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:01.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:01 smithi156 ceph-mon[36261]: from='client.? 172.21.15.170:0/985084210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:39:01.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:01 smithi170 ceph-mon[36438]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:01.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:01 smithi170 ceph-mon[36438]: from='client.? 172.21.15.170:0/985084210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:39:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:01 smithi016 ceph-mon[31908]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:01 smithi016 ceph-mon[31908]: from='client.? 172.21.15.170:0/985084210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:39:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:03 smithi156 ceph-mon[36261]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:03.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:03 smithi170 ceph-mon[36438]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:03.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:03 smithi016 ceph-mon[31908]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:05.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:05 smithi156 ceph-mon[36261]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:05.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:05 smithi170 ceph-mon[36438]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:05.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:05 smithi016 ceph-mon[31908]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:07.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:07 smithi156 ceph-mon[36261]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:07.830 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:07 smithi170 ceph-mon[36438]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:07.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:07 smithi016 ceph-mon[31908]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:08.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:08 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-27T06:39:08.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:08 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:08.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:08 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-27T06:39:08.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:08 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:08.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:08 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-27T06:39:08.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:08 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:09.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:09 smithi156 ceph-mon[36261]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:09.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:09 smithi156 ceph-mon[36261]: Deploying daemon osd.8 on smithi170 2024-03-27T06:39:09.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:09 smithi170 ceph-mon[36438]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:09.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:09 smithi170 ceph-mon[36438]: Deploying daemon osd.8 on smithi170 2024-03-27T06:39:09.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:09 smithi016 ceph-mon[31908]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:09.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:09 smithi016 ceph-mon[31908]: Deploying daemon osd.8 on smithi170 2024-03-27T06:39:11.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:11 smithi156 ceph-mon[36261]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:11.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:11 smithi170 ceph-mon[36438]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:11.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:11 smithi016 ceph-mon[31908]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:13.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:13 smithi156 ceph-mon[36261]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:13.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:13 smithi170 ceph-mon[36438]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:13.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:13 smithi016 ceph-mon[31908]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:15.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:14 smithi170 ceph-mon[36438]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:15.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:14 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:15.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:14 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:15.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:14 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:14 smithi016 ceph-mon[31908]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:14 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:14 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:14 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:15.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:14 smithi156 ceph-mon[36261]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:15.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:14 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:15.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:14 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:15.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:14 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:17.085 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:16 smithi170 ceph-mon[36438]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:17.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:16 smithi016 ceph-mon[31908]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:17.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:16 smithi156 ceph-mon[36261]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:17.470 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 8 on host 'smithi170' 2024-03-27T06:39:17.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.475+0000 7f6c5bfff700 1 -- 172.21.15.170:0/1323769934 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6c74062b40 con 0x7f6c5c0a36a0 2024-03-27T06:39:17.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 -- 172.21.15.170:0/1323769934 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6c5c0a36a0 msgr2=0x7f6c5c0a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:39:17.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6c5c0a36a0 0x7f6c5c0a5b60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009c60 tx=0x7f6c64009260 comp rx=0 tx=0).stop 2024-03-27T06:39:17.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 -- 172.21.15.170:0/1323769934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c74108360 msgr2=0x7f6c7419ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:39:17.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c74108360 0x7f6c7419ffc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00efc0 tx=0x7f6c6c00c3a0 comp rx=0 tx=0).stop 2024-03-27T06:39:17.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 -- 172.21.15.170:0/1323769934 shutdown_connections 2024-03-27T06:39:17.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6c5c0a36a0 0x7f6c5c0a5b60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:17.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c74108360 0x7f6c7419ffc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:17.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c74103100 0x7f6c74198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:17.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 --2- 172.21.15.170:0/1323769934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c74105a30 0x7f6c74198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:17.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 -- 172.21.15.170:0/1323769934 >> 172.21.15.170:0/1323769934 conn(0x7f6c740fcb10 msgr2=0x7f6c74103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:39:17.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 -- 172.21.15.170:0/1323769934 shutdown_connections 2024-03-27T06:39:17.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:39:17.477+0000 7f6c7977c700 1 -- 172.21.15.170:0/1323769934 wait complete. 2024-03-27T06:39:18.117 DEBUG:teuthology.orchestra.run.smithi170:osd.8> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.8.service 2024-03-27T06:39:18.120 INFO:tasks.cephadm:Deploying osd.9 on smithi016 with /dev/vg_nvme/lv_1... 2024-03-27T06:39:18.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-27T06:39:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:39:18.702 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:18 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:18.702 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:18 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:18.702 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:18 smithi016 ceph-mon[31908]: from='osd.8 [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T06:39:18.702 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:18 smithi016 ceph-mon[31908]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T06:39:18.702 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:18 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:18.703 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:18 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:18.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:18 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:18.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:18 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:18.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:18 smithi156 ceph-mon[36261]: from='osd.8 [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T06:39:18.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:18 smithi156 ceph-mon[36261]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T06:39:18.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:18 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:18.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:18 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:18.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:18 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:18.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:18 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:18.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:18 smithi170 ceph-mon[36438]: from='osd.8 [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T06:39:18.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:18 smithi170 ceph-mon[36438]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-27T06:39:18.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:18 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:18.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:18 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:19.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:19 smithi156 ceph-mon[36261]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:19.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:19 smithi156 ceph-mon[36261]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-27T06:39:19.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:19 smithi156 ceph-mon[36261]: osdmap e58: 9 total, 8 up, 9 in 2024-03-27T06:39:19.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:19.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:19 smithi156 ceph-mon[36261]: from='osd.8 [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:39:19.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:19 smithi156 ceph-mon[36261]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:39:19.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:19 smithi016 ceph-mon[31908]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:19.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:19 smithi016 ceph-mon[31908]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-27T06:39:19.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:19 smithi016 ceph-mon[31908]: osdmap e58: 9 total, 8 up, 9 in 2024-03-27T06:39:19.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:19.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:19 smithi016 ceph-mon[31908]: from='osd.8 [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:39:19.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:19 smithi016 ceph-mon[31908]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:39:19.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:19 smithi170 ceph-mon[36438]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:19.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:19 smithi170 ceph-mon[36438]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-27T06:39:19.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:19 smithi170 ceph-mon[36438]: osdmap e58: 9 total, 8 up, 9 in 2024-03-27T06:39:19.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:19.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:19 smithi170 ceph-mon[36438]: from='osd.8 [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:39:19.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:19 smithi170 ceph-mon[36438]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:39:19.844 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:39:19 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:39:19.450+0000 7f92e07ce700 -1 osd.8 0 waiting for initial osdmap 2024-03-27T06:39:19.844 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:39:19 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:39:19.461+0000 7f92dc5c1700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:39:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-27T06:39:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-27T06:39:20.546 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-03-27T06:39:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-03-27T06:39:20.546 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00832473 s, 1.3 GB/s 2024-03-27T06:39:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-03-27T06:39:20.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:20 smithi170 ceph-mon[36438]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:39:20.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:20 smithi170 ceph-mon[36438]: osdmap e59: 9 total, 8 up, 9 in 2024-03-27T06:39:20.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:20.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:20.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:20 smithi170 ceph-mon[36438]: osd.8 [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551] boot 2024-03-27T06:39:20.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:20 smithi170 ceph-mon[36438]: osdmap e60: 9 total, 9 up, 9 in 2024-03-27T06:39:20.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:20.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:20 smithi156 ceph-mon[36261]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:39:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:20 smithi156 ceph-mon[36261]: osdmap e59: 9 total, 8 up, 9 in 2024-03-27T06:39:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:20 smithi156 ceph-mon[36261]: osd.8 [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551] boot 2024-03-27T06:39:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:20 smithi156 ceph-mon[36261]: osdmap e60: 9 total, 9 up, 9 in 2024-03-27T06:39:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:20 smithi016 ceph-mon[31908]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:39:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:20 smithi016 ceph-mon[31908]: osdmap e59: 9 total, 8 up, 9 in 2024-03-27T06:39:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:20 smithi016 ceph-mon[31908]: osd.8 [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551] boot 2024-03-27T06:39:20.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:20 smithi016 ceph-mon[31908]: osdmap e60: 9 total, 9 up, 9 in 2024-03-27T06:39:20.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:39:21.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-03-27T06:39:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:39:21.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:21 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:39:21.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:21 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:39:21.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:21 smithi156 ceph-mon[36261]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:21.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:21 smithi156 ceph-mon[36261]: osdmap e61: 9 total, 9 up, 9 in 2024-03-27T06:39:21.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:21 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:39:21.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:21 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:39:21.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:21 smithi170 ceph-mon[36438]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:21.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:21 smithi170 ceph-mon[36438]: osdmap e61: 9 total, 9 up, 9 in 2024-03-27T06:39:21.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:21 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:39:21.813 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:21 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:39:21.813 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:21 smithi016 ceph-mon[31908]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-27T06:39:21.813 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:21 smithi016 ceph-mon[31908]: osdmap e61: 9 total, 9 up, 9 in 2024-03-27T06:39:23.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:23.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi170 2024-03-27T06:39:23.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:23.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:23.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:23.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi170 to 5693M 2024-03-27T06:39:23.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:23.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:23.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:23.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:23.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:23.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:22 smithi170 ceph-mon[36438]: osdmap e62: 9 total, 9 up, 9 in 2024-03-27T06:39:23.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:23.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi170 2024-03-27T06:39:23.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:23.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:23.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:23.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi170 to 5693M 2024-03-27T06:39:23.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.097 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:23.097 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:23.097 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.097 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:23.097 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:23.097 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:23.097 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.097 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:22 smithi016 ceph-mon[31908]: osdmap e62: 9 total, 9 up, 9 in 2024-03-27T06:39:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi170 2024-03-27T06:39:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi170 to 5693M 2024-03-27T06:39:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:23.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:22 smithi156 ceph-mon[36261]: osdmap e62: 9 total, 9 up, 9 in 2024-03-27T06:39:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.497+0000 7f915754c700 1 -- 172.21.15.16:0/3522956157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9150103180 msgr2=0x7f9150105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:39:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.497+0000 7f915754c700 1 --2- 172.21.15.16:0/3522956157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9150103180 0x7f9150105570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f914c009990 tx=0x7f914c009ca0 comp rx=0 tx=0).stop 2024-03-27T06:39:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.497+0000 7f915754c700 1 -- 172.21.15.16:0/3522956157 shutdown_connections 2024-03-27T06:39:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.497+0000 7f915754c700 1 --2- 172.21.15.16:0/3522956157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91501083e0 0x7f915010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.497+0000 7f915754c700 1 --2- 172.21.15.16:0/3522956157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9150103180 0x7f9150105570 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.497+0000 7f915754c700 1 --2- 172.21.15.16:0/3522956157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9150105ab0 0x7f9150107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.497+0000 7f915754c700 1 -- 172.21.15.16:0/3522956157 >> 172.21.15.16:0/3522956157 conn(0x7f91500fcb10 msgr2=0x7f91500fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:39:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.497+0000 7f915754c700 1 -- 172.21.15.16:0/3522956157 shutdown_connections 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.497+0000 7f915754c700 1 -- 172.21.15.16:0/3522956157 wait complete. 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.497+0000 7f915754c700 1 Processor -- start 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.497+0000 7f915754c700 1 -- start start 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f915754c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9150103180 0x7f9150102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f915754c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9150105ab0 0x7f9150101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f915754c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91501083e0 0x7f9150101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f915754c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f915010e400 con 0x7f91501083e0 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f915754c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f915010e280 con 0x7f9150103180 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f915754c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f915010e580 con 0x7f9150105ab0 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f9155ae9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91501083e0 0x7f9150101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f9155ae9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91501083e0 0x7f9150101670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35002/0 (socket says 172.21.15.16:35002) 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f9155ae9700 1 -- 172.21.15.16:0/94743727 learned_addr learned my addr 172.21.15.16:0/94743727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f91552e8700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9150103180 0x7f9150102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:39:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f9154ae7700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9150105ab0 0x7f9150101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:39:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f9155ae9700 1 -- 172.21.15.16:0/94743727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9150105ab0 msgr2=0x7f9150101130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:39:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f9155ae9700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9150105ab0 0x7f9150101130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f9155ae9700 1 -- 172.21.15.16:0/94743727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9150103180 msgr2=0x7f9150102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:39:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f9155ae9700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9150103180 0x7f9150102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f9155ae9700 1 -- 172.21.15.16:0/94743727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9150101f20 con 0x7f91501083e0 2024-03-27T06:39:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f9155ae9700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91501083e0 0x7f9150101670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9148007d40 tx=0x7f914800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:39:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f91467fc700 1 -- 172.21.15.16:0/94743727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148016020 con 0x7f91501083e0 2024-03-27T06:39:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f915754c700 1 -- 172.21.15.16:0/94743727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91501a2b40 con 0x7f91501083e0 2024-03-27T06:39:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f91467fc700 1 -- 172.21.15.16:0/94743727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f914800ac90 con 0x7f91501083e0 2024-03-27T06:39:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f915754c700 1 -- 172.21.15.16:0/94743727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91501a3050 con 0x7f91501083e0 2024-03-27T06:39:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.498+0000 7f91467fc700 1 -- 172.21.15.16:0/94743727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91480115e0 con 0x7f91501083e0 2024-03-27T06:39:23.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.499+0000 7f915754c700 1 -- 172.21.15.16:0/94743727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9150069300 con 0x7f91501083e0 2024-03-27T06:39:23.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.501+0000 7f91467fc700 1 -- 172.21.15.16:0/94743727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f9148019020 con 0x7f91501083e0 2024-03-27T06:39:23.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.501+0000 7f91467fc700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f913c0a36a0 0x7f913c0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:39:23.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.501+0000 7f91552e8700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f913c0a36a0 0x7f913c0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:39:23.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.501+0000 7f91467fc700 1 -- 172.21.15.16:0/94743727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f91480cf320 con 0x7f91501083e0 2024-03-27T06:39:23.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.502+0000 7f91552e8700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f913c0a36a0 0x7f913c0a5b60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f914c00b5c0 tx=0x7f914c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:39:23.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.506+0000 7f91467fc700 1 -- 172.21.15.16:0/94743727 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f9148020020 con 0x7f91501083e0 2024-03-27T06:39:23.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:23.726+0000 7f915754c700 1 -- 172.21.15.16:0/94743727 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9150062a80 con 0x7f913c0a36a0 2024-03-27T06:39:24.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:24 smithi016 ceph-mon[31908]: osdmap e63: 9 total, 9 up, 9 in 2024-03-27T06:39:24.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:24 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:39:24.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:24 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:39:24.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:24 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:24.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:24 smithi156 ceph-mon[36261]: osdmap e63: 9 total, 9 up, 9 in 2024-03-27T06:39:24.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:24 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:39:24.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:24 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:39:24.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:24 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:24.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:24 smithi170 ceph-mon[36438]: osdmap e63: 9 total, 9 up, 9 in 2024-03-27T06:39:24.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:24 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:39:24.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:24 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:39:24.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:24 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:25 smithi156 ceph-mon[36261]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:25 smithi156 ceph-mon[36261]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:39:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:25 smithi156 ceph-mon[36261]: osdmap e64: 9 total, 9 up, 9 in 2024-03-27T06:39:25.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:25 smithi016 ceph-mon[31908]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:25.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:25 smithi016 ceph-mon[31908]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:39:25.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:25 smithi016 ceph-mon[31908]: osdmap e64: 9 total, 9 up, 9 in 2024-03-27T06:39:25.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:25 smithi170 ceph-mon[36438]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:25.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:25 smithi170 ceph-mon[36438]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:39:25.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:25 smithi170 ceph-mon[36438]: osdmap e64: 9 total, 9 up, 9 in 2024-03-27T06:39:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:27 smithi156 ceph-mon[36261]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-03-27T06:39:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:27 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2751292865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6bb6987-e0df-4d98-9044-63946f8567cd"}]: dispatch 2024-03-27T06:39:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:27 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2751292865' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6bb6987-e0df-4d98-9044-63946f8567cd"}]': finished 2024-03-27T06:39:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:27 smithi156 ceph-mon[36261]: osdmap e65: 10 total, 9 up, 10 in 2024-03-27T06:39:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:27 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:27 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1328373798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:39:27.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:27 smithi170 ceph-mon[36438]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-03-27T06:39:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:27 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2751292865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6bb6987-e0df-4d98-9044-63946f8567cd"}]: dispatch 2024-03-27T06:39:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:27 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2751292865' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6bb6987-e0df-4d98-9044-63946f8567cd"}]': finished 2024-03-27T06:39:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:27 smithi170 ceph-mon[36438]: osdmap e65: 10 total, 9 up, 10 in 2024-03-27T06:39:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:27 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:27 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1328373798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:39:27.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:27 smithi016 ceph-mon[31908]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-03-27T06:39:27.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:27 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2751292865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6bb6987-e0df-4d98-9044-63946f8567cd"}]: dispatch 2024-03-27T06:39:27.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:27 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2751292865' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6bb6987-e0df-4d98-9044-63946f8567cd"}]': finished 2024-03-27T06:39:27.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:27 smithi016 ceph-mon[31908]: osdmap e65: 10 total, 9 up, 10 in 2024-03-27T06:39:27.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:27 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:27.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:27 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1328373798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:39:29.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:29 smithi156 ceph-mon[36261]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-27T06:39:29.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:29 smithi170 ceph-mon[36438]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-27T06:39:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:29 smithi016 ceph-mon[31908]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-27T06:39:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:31 smithi156 ceph-mon[36261]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-27T06:39:31.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:31 smithi016 ceph-mon[31908]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-27T06:39:31.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:31 smithi170 ceph-mon[36438]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-27T06:39:33.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:33 smithi016 ceph-mon[31908]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:39:33.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:33 smithi156 ceph-mon[36261]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:39:33.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:33 smithi170 ceph-mon[36438]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:39:35.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:35 smithi156 ceph-mon[36261]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-27T06:39:35.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-27T06:39:35.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:35 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:35.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:35 smithi170 ceph-mon[36438]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-27T06:39:35.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-27T06:39:35.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:35 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:35.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:35 smithi016 ceph-mon[31908]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-27T06:39:35.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-27T06:39:35.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:35 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:36.776 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:36 smithi016 ceph-mon[31908]: Deploying daemon osd.9 on smithi016 2024-03-27T06:39:36.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:36 smithi170 ceph-mon[36438]: Deploying daemon osd.9 on smithi016 2024-03-27T06:39:36.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:36 smithi156 ceph-mon[36261]: Deploying daemon osd.9 on smithi016 2024-03-27T06:39:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:37 smithi016 ceph-mon[31908]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:37.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:37 smithi170 ceph-mon[36438]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:37.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:37 smithi156 ceph-mon[36261]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:39.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:39 smithi016 ceph-mon[31908]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:39.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:39 smithi170 ceph-mon[36438]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:39.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:39 smithi156 ceph-mon[36261]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:41.522 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:41 smithi016 ceph-mon[31908]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:41.522 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:41 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:41.522 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:41 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:41.522 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:41 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:41.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:41 smithi170 ceph-mon[36438]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:41.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:41 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:41.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:41 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:41.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:41 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:41.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:41 smithi156 ceph-mon[36261]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:41.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:41 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:41.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:41 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:41.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:41 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:43.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:43 smithi170 ceph-mon[36438]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:43 smithi016 ceph-mon[31908]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:43.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:43 smithi156 ceph-mon[36261]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:44.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.793+0000 7f91467fc700 1 -- 172.21.15.16:0/94743727 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9150062a80 con 0x7f913c0a36a0 2024-03-27T06:39:44.789 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 9 on host 'smithi016' 2024-03-27T06:39:44.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.795+0000 7f915754c700 1 -- 172.21.15.16:0/94743727 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f913c0a36a0 msgr2=0x7f913c0a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:39:44.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.795+0000 7f915754c700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f913c0a36a0 0x7f913c0a5b60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f914c00b5c0 tx=0x7f914c005fb0 comp rx=0 tx=0).stop 2024-03-27T06:39:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.795+0000 7f915754c700 1 -- 172.21.15.16:0/94743727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91501083e0 msgr2=0x7f9150101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:39:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.795+0000 7f915754c700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91501083e0 0x7f9150101670 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9148007d40 tx=0x7f914800a340 comp rx=0 tx=0).stop 2024-03-27T06:39:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.795+0000 7f915754c700 1 -- 172.21.15.16:0/94743727 shutdown_connections 2024-03-27T06:39:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.795+0000 7f915754c700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91501083e0 0x7f9150101670 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.795+0000 7f915754c700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9150103180 0x7f9150102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:44.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.795+0000 7f915754c700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f913c0a36a0 0x7f913c0a5b60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.795+0000 7f915754c700 1 --2- 172.21.15.16:0/94743727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9150105ab0 0x7f9150101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.795+0000 7f915754c700 1 -- 172.21.15.16:0/94743727 >> 172.21.15.16:0/94743727 conn(0x7f91500fcb10 msgr2=0x7f9150103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:39:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.796+0000 7f915754c700 1 -- 172.21.15.16:0/94743727 shutdown_connections 2024-03-27T06:39:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:39:44.796+0000 7f915754c700 1 -- 172.21.15.16:0/94743727 wait complete. 2024-03-27T06:39:45.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:44 smithi170 ceph-mon[36438]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:45.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:44 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:44 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:44 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:45.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:44 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:45.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:44 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:44 smithi170 ceph-mon[36438]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-27T06:39:45.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:44 smithi016 ceph-mon[31908]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:45.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:44 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:44 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:44 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:45.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:44 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:45.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:44 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:44 smithi016 ceph-mon[31908]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-27T06:39:45.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:44 smithi156 ceph-mon[36261]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:45.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:44 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:44 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:44 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:45.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:44 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:45.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:44 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:44 smithi156 ceph-mon[36261]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-27T06:39:45.613 DEBUG:teuthology.orchestra.run.smithi016:osd.9> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.9.service 2024-03-27T06:39:45.616 INFO:tasks.cephadm:Deploying osd.10 on smithi156 with /dev/vg_nvme/lv_1... 2024-03-27T06:39:45.616 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-27T06:39:45.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:45 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:45 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:45 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:45.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:45 smithi016 ceph-mon[31908]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-27T06:39:45.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:45 smithi016 ceph-mon[31908]: osdmap e66: 10 total, 9 up, 10 in 2024-03-27T06:39:45.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:45 smithi016 ceph-mon[31908]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:39:45.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:45 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:45.865 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:39:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:45 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:45 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:45 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:45 smithi156 ceph-mon[36261]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-27T06:39:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:45 smithi156 ceph-mon[36261]: osdmap e66: 10 total, 9 up, 10 in 2024-03-27T06:39:45.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:45 smithi156 ceph-mon[36261]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:39:45.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:45 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:46.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:45 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:46.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:45 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:46.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:45 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:39:46.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:45 smithi170 ceph-mon[36438]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-27T06:39:46.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:45 smithi170 ceph-mon[36438]: osdmap e66: 10 total, 9 up, 10 in 2024-03-27T06:39:46.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:45 smithi170 ceph-mon[36438]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-27T06:39:46.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:45 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:46.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:39:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:39:46.501+0000 7f4b6432b700 -1 osd.9 0 waiting for initial osdmap 2024-03-27T06:39:46.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:39:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:39:46.513+0000 7f4b6011e700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:39:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:47 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:39:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:47 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:39:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:47 smithi156 ceph-mon[36261]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:47 smithi156 ceph-mon[36261]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:39:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:47 smithi156 ceph-mon[36261]: osdmap e67: 10 total, 9 up, 10 in 2024-03-27T06:39:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:47 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:47 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:47.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:47 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:39:47.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:47 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:39:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:47 smithi170 ceph-mon[36438]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:47 smithi170 ceph-mon[36438]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:39:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:47 smithi170 ceph-mon[36438]: osdmap e67: 10 total, 9 up, 10 in 2024-03-27T06:39:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:47 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:47 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:47 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:39:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:47 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:39:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:47 smithi016 ceph-mon[31908]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-27T06:39:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:47 smithi016 ceph-mon[31908]: from='osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-27T06:39:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:47 smithi016 ceph-mon[31908]: osdmap e67: 10 total, 9 up, 10 in 2024-03-27T06:39:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:47 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:47 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:47.916 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-27T06:39:47.916 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-27T06:39:47.916 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2024-03-27T06:39:47.916 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2024-03-27T06:39:47.916 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00855247 s, 1.2 GB/s 2024-03-27T06:39:47.916 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2024-03-27T06:39:48.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:48 smithi016 ceph-mon[31908]: osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184] boot 2024-03-27T06:39:48.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:48 smithi016 ceph-mon[31908]: osdmap e68: 10 total, 10 up, 10 in 2024-03-27T06:39:48.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:48 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:48.677 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi156:vg_nvme/lv_1 2024-03-27T06:39:48.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:48 smithi170 ceph-mon[36438]: osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184] boot 2024-03-27T06:39:48.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:48 smithi170 ceph-mon[36438]: osdmap e68: 10 total, 10 up, 10 in 2024-03-27T06:39:48.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:48 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:48.921 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:39:48.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:48 smithi156 ceph-mon[36261]: osd.9 [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184] boot 2024-03-27T06:39:48.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:48 smithi156 ceph-mon[36261]: osdmap e68: 10 total, 10 up, 10 in 2024-03-27T06:39:48.956 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:48 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:39:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:49 smithi016 ceph-mon[31908]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:49.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:49.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:49.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:49.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:49 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:49 smithi156 ceph-mon[36261]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:49 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:49.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:49 smithi170 ceph-mon[36438]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:49.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:49.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:39:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:39:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:49 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:39:50.593 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 -- 172.21.15.156:0/3930635225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1720102670 msgr2=0x7f1720102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:39:50.594 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 --2- 172.21.15.156:0/3930635225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1720102670 0x7f1720102a90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f171c0099c0 tx=0x7f171c009cd0 comp rx=0 tx=0).stop 2024-03-27T06:39:50.594 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 -- 172.21.15.156:0/3930635225 shutdown_connections 2024-03-27T06:39:50.594 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 --2- 172.21.15.156:0/3930635225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1720104210 0x7f172010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:50.594 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 --2- 172.21.15.156:0/3930635225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1720102670 0x7f1720102a90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:50.594 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 --2- 172.21.15.156:0/3930635225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1720103870 0x7f1720103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:50.594 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 -- 172.21.15.156:0/3930635225 >> 172.21.15.156:0/3930635225 conn(0x7f17200fdc50 msgr2=0x7f1720100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:39:50.594 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 -- 172.21.15.156:0/3930635225 shutdown_connections 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 -- 172.21.15.156:0/3930635225 wait complete. 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 Processor -- start 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 -- start start 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.597+0000 7f17277dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1720102670 0x7f1720198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f17277dc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1720103870 0x7f1720198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f17277dc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1720104210 0x7f172019ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f17277dc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f172010e3f0 con 0x7f1720102670 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f17277dc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f172010e270 con 0x7f1720104210 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f17277dc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f172010e570 con 0x7f1720103870 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1725d79700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1720104210 0x7f172019ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1725d79700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1720104210 0x7f172019ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:33702/0 (socket says 172.21.15.156:33702) 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1725d79700 1 -- 172.21.15.156:0/2278949816 learned_addr learned my addr 172.21.15.156:0/2278949816 (peer_addr_for_me v2:172.21.15.156:0/0) 2024-03-27T06:39:50.595 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1724d77700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1720103870 0x7f1720198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1725578700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1720102670 0x7f1720198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1725d79700 1 -- 172.21.15.156:0/2278949816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1720103870 msgr2=0x7f1720198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1725d79700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1720103870 0x7f1720198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1725d79700 1 -- 172.21.15.156:0/2278949816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1720102670 msgr2=0x7f1720198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1725d79700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1720102670 0x7f1720198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1725d79700 1 -- 172.21.15.156:0/2278949816 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17201a05d0 con 0x7f1720104210 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1724d77700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1720103870 0x7f1720198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f1725d79700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1720104210 0x7f172019ff30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1718007d70 tx=0x7f171800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f17167fc700 1 -- 172.21.15.156:0/2278949816 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718016020 con 0x7f1720104210 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f17277dc700 1 -- 172.21.15.156:0/2278949816 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17201a08c0 con 0x7f1720104210 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f17167fc700 1 -- 172.21.15.156:0/2278949816 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f171800ad40 con 0x7f1720104210 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f17167fc700 1 -- 172.21.15.156:0/2278949816 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718011780 con 0x7f1720104210 2024-03-27T06:39:50.596 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.598+0000 7f17277dc700 1 -- 172.21.15.156:0/2278949816 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17201a0e50 con 0x7f1720104210 2024-03-27T06:39:50.597 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.599+0000 7f17277dc700 1 -- 172.21.15.156:0/2278949816 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1720069300 con 0x7f1720104210 2024-03-27T06:39:50.597 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.600+0000 7f17167fc700 1 -- 172.21.15.156:0/2278949816 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f1718019020 con 0x7f1720104210 2024-03-27T06:39:50.598 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.601+0000 7f17167fc700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f170c0a36a0 0x7f170c0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:39:50.598 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.601+0000 7f17167fc700 1 -- 172.21.15.156:0/2278949816 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f17180cf4e0 con 0x7f1720104210 2024-03-27T06:39:50.598 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.601+0000 7f1725578700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f170c0a36a0 0x7f170c0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:39:50.598 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.602+0000 7f1725578700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f170c0a36a0 0x7f170c0a5b60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b580 tx=0x7f171c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:39:50.600 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.606+0000 7f17167fc700 1 -- 172.21.15.156:0/2278949816 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f1718020020 con 0x7f1720104210 2024-03-27T06:39:50.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:50 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi016 2024-03-27T06:39:50.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:50 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi016 to 4273M 2024-03-27T06:39:50.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:50 smithi156 ceph-mon[36261]: osdmap e69: 10 total, 10 up, 10 in 2024-03-27T06:39:50.810 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:39:50.815+0000 7f17277dc700 1 -- 172.21.15.156:0/2278949816 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1720062a80 con 0x7f170c0a36a0 2024-03-27T06:39:50.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:50 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi016 2024-03-27T06:39:50.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:50 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi016 to 4273M 2024-03-27T06:39:50.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:50 smithi170 ceph-mon[36438]: osdmap e69: 10 total, 10 up, 10 in 2024-03-27T06:39:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:50 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi016 2024-03-27T06:39:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:50 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi016 to 4273M 2024-03-27T06:39:50.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:50 smithi016 ceph-mon[31908]: osdmap e69: 10 total, 10 up, 10 in 2024-03-27T06:39:51.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:51 smithi156 ceph-mon[36261]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:51.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:51 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:39:51.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:51 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:39:51.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:51 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:51.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:51 smithi170 ceph-mon[36438]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:51.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:51 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:39:51.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:51 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:39:51.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:51 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:51 smithi016 ceph-mon[31908]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:51 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:39:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:51 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:39:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:51 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:39:52.672 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:52 smithi156 ceph-mon[36261]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:39:52.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:52 smithi170 ceph-mon[36438]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:39:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:52 smithi016 ceph-mon[31908]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:39:53.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:53 smithi170 ceph-mon[36438]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:53.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:53 smithi170 ceph-mon[36438]: from='client.? 172.21.15.156:0/3505716986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7e2309-4c4e-496e-9043-ff264a805f66"}]: dispatch 2024-03-27T06:39:53.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:53 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7e2309-4c4e-496e-9043-ff264a805f66"}]: dispatch 2024-03-27T06:39:53.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:53 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb7e2309-4c4e-496e-9043-ff264a805f66"}]': finished 2024-03-27T06:39:53.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:53 smithi170 ceph-mon[36438]: osdmap e70: 11 total, 10 up, 11 in 2024-03-27T06:39:53.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:53 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:39:53.857 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:53 smithi156 ceph-mon[36261]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:53.857 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:53 smithi156 ceph-mon[36261]: from='client.? 172.21.15.156:0/3505716986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7e2309-4c4e-496e-9043-ff264a805f66"}]: dispatch 2024-03-27T06:39:53.857 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:53 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7e2309-4c4e-496e-9043-ff264a805f66"}]: dispatch 2024-03-27T06:39:53.857 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:53 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb7e2309-4c4e-496e-9043-ff264a805f66"}]': finished 2024-03-27T06:39:53.857 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:53 smithi156 ceph-mon[36261]: osdmap e70: 11 total, 10 up, 11 in 2024-03-27T06:39:53.857 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:53 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:39:53.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:53 smithi016 ceph-mon[31908]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:53.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:53 smithi016 ceph-mon[31908]: from='client.? 172.21.15.156:0/3505716986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7e2309-4c4e-496e-9043-ff264a805f66"}]: dispatch 2024-03-27T06:39:53.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:53 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7e2309-4c4e-496e-9043-ff264a805f66"}]: dispatch 2024-03-27T06:39:53.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:53 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb7e2309-4c4e-496e-9043-ff264a805f66"}]': finished 2024-03-27T06:39:53.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:53 smithi016 ceph-mon[31908]: osdmap e70: 11 total, 10 up, 11 in 2024-03-27T06:39:53.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:53 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:39:54.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:54 smithi170 ceph-mon[36438]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:54.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:54 smithi170 ceph-mon[36438]: from='client.? 172.21.15.156:0/465600921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:39:54.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:54 smithi016 ceph-mon[31908]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:54.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:54 smithi016 ceph-mon[31908]: from='client.? 172.21.15.156:0/465600921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:39:54.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:54 smithi156 ceph-mon[36261]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:54.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:54 smithi156 ceph-mon[36261]: from='client.? 172.21.15.156:0/465600921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:39:56.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:56 smithi170 ceph-mon[36438]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:56.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:56 smithi016 ceph-mon[31908]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:56.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:56 smithi156 ceph-mon[36261]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:58.658 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:39:58 smithi156 ceph-mon[36261]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:58.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:39:58 smithi170 ceph-mon[36438]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:39:58.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:39:58 smithi016 ceph-mon[31908]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:00.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:00 smithi156 ceph-mon[36261]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:00.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:00 smithi156 ceph-mon[36261]: overall HEALTH_OK 2024-03-27T06:40:00.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:00 smithi016 ceph-mon[31908]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:00.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:00 smithi016 ceph-mon[31908]: overall HEALTH_OK 2024-03-27T06:40:01.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:00 smithi170 ceph-mon[36438]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:01.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:00 smithi170 ceph-mon[36438]: overall HEALTH_OK 2024-03-27T06:40:01.732 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-27T06:40:01.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:01.733 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:01 smithi156 ceph-mon[36261]: Deploying daemon osd.10 on smithi156 2024-03-27T06:40:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-27T06:40:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:01 smithi016 ceph-mon[31908]: Deploying daemon osd.10 on smithi156 2024-03-27T06:40:01.995 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-27T06:40:01.995 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:01.995 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:01 smithi170 ceph-mon[36438]: Deploying daemon osd.10 on smithi156 2024-03-27T06:40:02.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:02 smithi156 ceph-mon[36261]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:02.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:02 smithi016 ceph-mon[31908]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:03.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:02 smithi170 ceph-mon[36438]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:04.854 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:04 smithi156 ceph-mon[36261]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:04.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:04 smithi016 ceph-mon[31908]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:05.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:04 smithi170 ceph-mon[36438]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:06.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:06 smithi156 ceph-mon[36261]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:06.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:06 smithi016 ceph-mon[31908]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:07.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:06 smithi170 ceph-mon[36438]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:07.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:07 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:07.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:07 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:07.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:07 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:08.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:07 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:08.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:07 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:08.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:07 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:07 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:07 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:07 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:09.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:08 smithi170 ceph-mon[36438]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:09.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:08 smithi016 ceph-mon[31908]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:09.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:08 smithi156 ceph-mon[36261]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:10.675 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.680+0000 7f17167fc700 1 -- 172.21.15.156:0/2278949816 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1720062a80 con 0x7f170c0a36a0 2024-03-27T06:40:10.675 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 10 on host 'smithi156' 2024-03-27T06:40:10.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.682+0000 7f17277dc700 1 -- 172.21.15.156:0/2278949816 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f170c0a36a0 msgr2=0x7f170c0a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:10.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.682+0000 7f17277dc700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f170c0a36a0 0x7f170c0a5b60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b580 tx=0x7f171c005fb0 comp rx=0 tx=0).stop 2024-03-27T06:40:10.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.682+0000 7f17277dc700 1 -- 172.21.15.156:0/2278949816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1720104210 msgr2=0x7f172019ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:10.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.682+0000 7f17277dc700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1720104210 0x7f172019ff30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1718007d70 tx=0x7f171800a340 comp rx=0 tx=0).stop 2024-03-27T06:40:10.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.682+0000 7f17277dc700 1 -- 172.21.15.156:0/2278949816 shutdown_connections 2024-03-27T06:40:10.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.682+0000 7f17277dc700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1720104210 0x7f172019ff30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:10.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.682+0000 7f17277dc700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1720102670 0x7f1720198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:10.677 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.682+0000 7f17277dc700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f170c0a36a0 0x7f170c0a5b60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:10.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.682+0000 7f17277dc700 1 --2- 172.21.15.156:0/2278949816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1720103870 0x7f1720198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:10.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.683+0000 7f17277dc700 1 -- 172.21.15.156:0/2278949816 >> 172.21.15.156:0/2278949816 conn(0x7f17200fdc50 msgr2=0x7f1720106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:10.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.683+0000 7f17277dc700 1 -- 172.21.15.156:0/2278949816 shutdown_connections 2024-03-27T06:40:10.678 INFO:teuthology.orchestra.run.smithi156.stderr:2024-03-27T06:40:10.683+0000 7f17277dc700 1 -- 172.21.15.156:0/2278949816 wait complete. 2024-03-27T06:40:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:10 smithi156 ceph-mon[36261]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:10 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:10 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:10 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:10 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:10 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:10 smithi156 ceph-mon[36261]: from='osd.10 [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T06:40:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:10 smithi156 ceph-mon[36261]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T06:40:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:10 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:10 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:10 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:10 smithi170 ceph-mon[36438]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:10 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:10 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:10 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:10 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:10 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:10 smithi170 ceph-mon[36438]: from='osd.10 [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T06:40:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:10 smithi170 ceph-mon[36438]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T06:40:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:10 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:10 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:10 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:10 smithi016 ceph-mon[31908]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:10 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:10 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:10 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:10 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:10 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:10 smithi016 ceph-mon[31908]: from='osd.10 [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T06:40:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:10 smithi016 ceph-mon[31908]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-27T06:40:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:10 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:10 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:11.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:10 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:11.378 DEBUG:teuthology.orchestra.run.smithi156:osd.10> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.10.service 2024-03-27T06:40:11.381 INFO:tasks.cephadm:Deploying osd.11 on smithi170 with /dev/vg_nvme/lv_1... 2024-03-27T06:40:11.381 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-27T06:40:11.629 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:40:11.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:11 smithi156 ceph-mon[36261]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-27T06:40:11.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:11 smithi156 ceph-mon[36261]: osdmap e71: 11 total, 10 up, 11 in 2024-03-27T06:40:11.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:11.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:11 smithi156 ceph-mon[36261]: from='osd.10 [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:40:11.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:11 smithi156 ceph-mon[36261]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:40:12.090 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:11 smithi170 ceph-mon[36438]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-27T06:40:12.090 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:11 smithi170 ceph-mon[36438]: osdmap e71: 11 total, 10 up, 11 in 2024-03-27T06:40:12.090 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:12.090 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:11 smithi170 ceph-mon[36438]: from='osd.10 [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:40:12.090 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:11 smithi170 ceph-mon[36438]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:40:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:11 smithi016 ceph-mon[31908]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-27T06:40:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:11 smithi016 ceph-mon[31908]: osdmap e71: 11 total, 10 up, 11 in 2024-03-27T06:40:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:11 smithi016 ceph-mon[31908]: from='osd.10 [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:40:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:11 smithi016 ceph-mon[31908]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2024-03-27T06:40:12.722 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:40:12 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:40:12.297+0000 7fca3075e700 -1 osd.10 0 waiting for initial osdmap 2024-03-27T06:40:12.722 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:40:12 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:40:12.310+0000 7fca28549700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:40:13.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:12 smithi170 ceph-mon[36438]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:13.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:12 smithi170 ceph-mon[36438]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:40:13.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:12 smithi170 ceph-mon[36438]: osdmap e72: 11 total, 10 up, 11 in 2024-03-27T06:40:13.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:12 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:13.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:12 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:13.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:12 smithi170 ceph-mon[36438]: from='osd.10 ' entity='osd.10' 2024-03-27T06:40:13.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:12 smithi016 ceph-mon[31908]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:13.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:12 smithi016 ceph-mon[31908]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:40:13.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:12 smithi016 ceph-mon[31908]: osdmap e72: 11 total, 10 up, 11 in 2024-03-27T06:40:13.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:12 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:13.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:12 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:13.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:12 smithi016 ceph-mon[31908]: from='osd.10 ' entity='osd.10' 2024-03-27T06:40:13.139 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:12 smithi156 ceph-mon[36261]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-27T06:40:13.140 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:12 smithi156 ceph-mon[36261]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2024-03-27T06:40:13.140 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:12 smithi156 ceph-mon[36261]: osdmap e72: 11 total, 10 up, 11 in 2024-03-27T06:40:13.140 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:12 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:13.140 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:12 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:13.140 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:12 smithi156 ceph-mon[36261]: from='osd.10 ' entity='osd.10' 2024-03-27T06:40:13.769 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-27T06:40:13.770 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-27T06:40:13.770 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-03-27T06:40:13.770 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-03-27T06:40:13.770 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.008425 s, 1.2 GB/s 2024-03-27T06:40:13.770 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-03-27T06:40:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:13 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:40:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:13 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:40:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:13 smithi170 ceph-mon[36438]: osd.10 [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561] boot 2024-03-27T06:40:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:13 smithi170 ceph-mon[36438]: osdmap e73: 11 total, 11 up, 11 in 2024-03-27T06:40:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:13 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:40:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:13 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:40:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:13 smithi016 ceph-mon[31908]: osd.10 [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561] boot 2024-03-27T06:40:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:13 smithi016 ceph-mon[31908]: osdmap e73: 11 total, 11 up, 11 in 2024-03-27T06:40:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:14.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:13 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:40:14.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:13 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:40:14.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:13 smithi156 ceph-mon[36261]: osd.10 [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561] boot 2024-03-27T06:40:14.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:13 smithi156 ceph-mon[36261]: osdmap e73: 11 total, 11 up, 11 in 2024-03-27T06:40:14.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:40:14.637 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi170:vg_nvme/lv_1 2024-03-27T06:40:14.894 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:40:14.917 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:14 smithi170 ceph-mon[36438]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:14.918 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:14 smithi170 ceph-mon[36438]: osdmap e74: 11 total, 11 up, 11 in 2024-03-27T06:40:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:14 smithi016 ceph-mon[31908]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:14 smithi016 ceph-mon[31908]: osdmap e74: 11 total, 11 up, 11 in 2024-03-27T06:40:15.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:14 smithi156 ceph-mon[36261]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:15.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:14 smithi156 ceph-mon[36261]: osdmap e74: 11 total, 11 up, 11 in 2024-03-27T06:40:16.082 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: osdmap e75: 11 total, 11 up, 11 in 2024-03-27T06:40:16.082 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi156 2024-03-27T06:40:16.082 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.082 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.082 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.082 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.082 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.082 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.082 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi156 to 4270M 2024-03-27T06:40:16.083 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.083 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:16.083 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:16.083 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.083 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:15 smithi170 ceph-mon[36438]: osdmap e76: 11 total, 11 up, 11 in 2024-03-27T06:40:16.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: osdmap e75: 11 total, 11 up, 11 in 2024-03-27T06:40:16.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi156 2024-03-27T06:40:16.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi156 to 4270M 2024-03-27T06:40:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:15 smithi016 ceph-mon[31908]: osdmap e76: 11 total, 11 up, 11 in 2024-03-27T06:40:16.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: osdmap e75: 11 total, 11 up, 11 in 2024-03-27T06:40:16.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi156 2024-03-27T06:40:16.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:16.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi156 to 4270M 2024-03-27T06:40:16.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:16.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:16.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:16.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:15 smithi156 ceph-mon[36261]: osdmap e76: 11 total, 11 up, 11 in 2024-03-27T06:40:16.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.643+0000 7f766af17700 1 -- 172.21.15.170:0/4201323183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7664104240 msgr2=0x7f766410aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:16.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.643+0000 7f766af17700 1 --2- 172.21.15.170:0/4201323183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7664104240 0x7f766410aaf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f765c009990 tx=0x7f765c009ca0 comp rx=0 tx=0).stop 2024-03-27T06:40:16.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.643+0000 7f766af17700 1 -- 172.21.15.170:0/4201323183 shutdown_connections 2024-03-27T06:40:16.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.643+0000 7f766af17700 1 --2- 172.21.15.170:0/4201323183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7664104240 0x7f766410aaf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.643+0000 7f766af17700 1 --2- 172.21.15.170:0/4201323183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76641026a0 0x7f7664102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.643+0000 7f766af17700 1 --2- 172.21.15.170:0/4201323183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76641038a0 0x7f7664103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.643+0000 7f766af17700 1 -- 172.21.15.170:0/4201323183 >> 172.21.15.170:0/4201323183 conn(0x7f76640fdc40 msgr2=0x7f7664100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.643+0000 7f766af17700 1 -- 172.21.15.170:0/4201323183 shutdown_connections 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.643+0000 7f766af17700 1 -- 172.21.15.170:0/4201323183 wait complete. 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f766af17700 1 Processor -- start 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f766af17700 1 -- start start 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f766af17700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76641026a0 0x7f76641988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f766af17700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76641038a0 0x7f7664198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f766af17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7664104240 0x7f766419feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f766af17700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f766410e3d0 con 0x7f7664104240 2024-03-27T06:40:16.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f766af17700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f766410e250 con 0x7f76641026a0 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f766af17700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f766410e550 con 0x7f76641038a0 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f765bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76641038a0 0x7f7664198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f765bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76641038a0 0x7f7664198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38518/0 (socket says 172.21.15.170:38518) 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f765bfff700 1 -- 172.21.15.170:0/2821270079 learned_addr learned my addr 172.21.15.170:0/2821270079 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f7668cb3700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76641026a0 0x7f76641988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f76694b4700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7664104240 0x7f766419feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f765bfff700 1 -- 172.21.15.170:0/2821270079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76641026a0 msgr2=0x7f76641988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f765bfff700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76641026a0 0x7f76641988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f765bfff700 1 -- 172.21.15.170:0/2821270079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7664104240 msgr2=0x7f766419feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f765bfff700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7664104240 0x7f766419feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f765bfff700 1 -- 172.21.15.170:0/2821270079 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76641a0550 con 0x7f76641038a0 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f765bfff700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76641038a0 0x7f7664198e30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f765400c950 tx=0x7f765400cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.644+0000 7f7659ffb700 1 -- 172.21.15.170:0/2821270079 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7654007ac0 con 0x7f76641038a0 2024-03-27T06:40:16.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.645+0000 7f766af17700 1 -- 172.21.15.170:0/2821270079 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76641a0840 con 0x7f76641038a0 2024-03-27T06:40:16.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.645+0000 7f7659ffb700 1 -- 172.21.15.170:0/2821270079 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7654004d30 con 0x7f76641038a0 2024-03-27T06:40:16.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.645+0000 7f7659ffb700 1 -- 172.21.15.170:0/2821270079 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765400f450 con 0x7f76641038a0 2024-03-27T06:40:16.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.645+0000 7f766af17700 1 -- 172.21.15.170:0/2821270079 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76641a0d80 con 0x7f76641038a0 2024-03-27T06:40:16.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.646+0000 7f766af17700 1 -- 172.21.15.170:0/2821270079 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7664069300 con 0x7f76641038a0 2024-03-27T06:40:16.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.647+0000 7f7659ffb700 1 -- 172.21.15.170:0/2821270079 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f7654004750 con 0x7f76641038a0 2024-03-27T06:40:16.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.647+0000 7f7659ffb700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f764c0a36a0 0x7f764c0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:16.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.647+0000 7f7659ffb700 1 -- 172.21.15.170:0/2821270079 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6694+0+0 (secure 0 0 0) 0x7f76540cf5c0 con 0x7f76641038a0 2024-03-27T06:40:16.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.648+0000 7f7668cb3700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f764c0a36a0 0x7f764c0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:16.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.648+0000 7f7668cb3700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f764c0a36a0 0x7f764c0a5b60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f76600060b0 tx=0x7f766000c040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:16.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.652+0000 7f7659ffb700 1 -- 172.21.15.170:0/2821270079 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7654014030 con 0x7f76641038a0 2024-03-27T06:40:16.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:16.858+0000 7f766af17700 1 -- 172.21.15.170:0/2821270079 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7664062a80 con 0x7f764c0a36a0 2024-03-27T06:40:17.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:17 smithi156 ceph-mon[36261]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:17.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:17 smithi156 ceph-mon[36261]: osdmap e77: 11 total, 11 up, 11 in 2024-03-27T06:40:17.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:17 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:40:17.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:17 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:40:17.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:17 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:17.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:17 smithi170 ceph-mon[36438]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:17.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:17 smithi170 ceph-mon[36438]: osdmap e77: 11 total, 11 up, 11 in 2024-03-27T06:40:17.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:17 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:40:17.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:17 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:40:17.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:17 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:17.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:17 smithi016 ceph-mon[31908]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:17.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:17 smithi016 ceph-mon[31908]: osdmap e77: 11 total, 11 up, 11 in 2024-03-27T06:40:17.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:17 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-27T06:40:17.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:17 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-27T06:40:17.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:17 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:18.709 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:18 smithi170 ceph-mon[36438]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:40:18.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:18 smithi156 ceph-mon[36261]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:40:18.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:18 smithi016 ceph-mon[31908]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:40:19.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:19 smithi156 ceph-mon[36261]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:19.780 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:19 smithi016 ceph-mon[31908]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:19.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:19 smithi170 ceph-mon[36438]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:20 smithi156 ceph-mon[36261]: from='client.? 172.21.15.170:0/204783335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73e6132-619a-4b5c-85a7-e5c3ee40188e"}]: dispatch 2024-03-27T06:40:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:20 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73e6132-619a-4b5c-85a7-e5c3ee40188e"}]: dispatch 2024-03-27T06:40:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:20 smithi156 ceph-mon[36261]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a73e6132-619a-4b5c-85a7-e5c3ee40188e"}]': finished 2024-03-27T06:40:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:20 smithi156 ceph-mon[36261]: osdmap e78: 12 total, 11 up, 12 in 2024-03-27T06:40:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:20 smithi156 ceph-mon[36261]: from='client.? 172.21.15.170:0/840210269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:40:20.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:20 smithi170 ceph-mon[36438]: from='client.? 172.21.15.170:0/204783335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73e6132-619a-4b5c-85a7-e5c3ee40188e"}]: dispatch 2024-03-27T06:40:20.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:20 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73e6132-619a-4b5c-85a7-e5c3ee40188e"}]: dispatch 2024-03-27T06:40:20.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:20 smithi170 ceph-mon[36438]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a73e6132-619a-4b5c-85a7-e5c3ee40188e"}]': finished 2024-03-27T06:40:20.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:20 smithi170 ceph-mon[36438]: osdmap e78: 12 total, 11 up, 12 in 2024-03-27T06:40:20.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:20.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:20 smithi170 ceph-mon[36438]: from='client.? 172.21.15.170:0/840210269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:40:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:20 smithi016 ceph-mon[31908]: from='client.? 172.21.15.170:0/204783335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73e6132-619a-4b5c-85a7-e5c3ee40188e"}]: dispatch 2024-03-27T06:40:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:20 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a73e6132-619a-4b5c-85a7-e5c3ee40188e"}]: dispatch 2024-03-27T06:40:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:20 smithi016 ceph-mon[31908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a73e6132-619a-4b5c-85a7-e5c3ee40188e"}]': finished 2024-03-27T06:40:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:20 smithi016 ceph-mon[31908]: osdmap e78: 12 total, 11 up, 12 in 2024-03-27T06:40:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:20 smithi016 ceph-mon[31908]: from='client.? 172.21.15.170:0/840210269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-27T06:40:21.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:21 smithi156 ceph-mon[36261]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:21.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:21 smithi170 ceph-mon[36438]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:21.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:21 smithi016 ceph-mon[31908]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:23.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:23 smithi156 ceph-mon[36261]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-27T06:40:23.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:23 smithi170 ceph-mon[36438]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-27T06:40:23.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:23 smithi016 ceph-mon[31908]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-27T06:40:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:25 smithi156 ceph-mon[36261]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:40:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:25 smithi170 ceph-mon[36438]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:40:25.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:25 smithi016 ceph-mon[31908]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-27T06:40:27.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:27 smithi170 ceph-mon[36438]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-27T06:40:27.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:27 smithi016 ceph-mon[31908]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-27T06:40:27.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:27 smithi156 ceph-mon[36261]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-27T06:40:28.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-27T06:40:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-27T06:40:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:28.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-27T06:40:28.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:29.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:29 smithi170 ceph-mon[36438]: Deploying daemon osd.11 on smithi170 2024-03-27T06:40:29.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:29 smithi170 ceph-mon[36438]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T06:40:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:29 smithi016 ceph-mon[31908]: Deploying daemon osd.11 on smithi170 2024-03-27T06:40:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:29 smithi016 ceph-mon[31908]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T06:40:29.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:29 smithi156 ceph-mon[36261]: Deploying daemon osd.11 on smithi170 2024-03-27T06:40:29.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:29 smithi156 ceph-mon[36261]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T06:40:31.735 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:31 smithi170 ceph-mon[36438]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T06:40:31.774 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:31 smithi156 ceph-mon[36261]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T06:40:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:31 smithi016 ceph-mon[31908]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-27T06:40:33.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:33 smithi170 ceph-mon[36438]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:33.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:33 smithi016 ceph-mon[31908]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:33 smithi156 ceph-mon[36261]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:34.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:34 smithi156 ceph-mon[36261]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:34.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:34 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:34.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:34 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:34.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:34 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:35.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:34 smithi170 ceph-mon[36438]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:35.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:34 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:35.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:34 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:35.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:34 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:35.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:34 smithi016 ceph-mon[31908]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:35.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:34 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:35.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:34 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:35.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:34 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:36.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:36 smithi156 ceph-mon[36261]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:37.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:36 smithi170 ceph-mon[36438]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:37.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:36 smithi016 ceph-mon[31908]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:37.275 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 11 on host 'smithi170' 2024-03-27T06:40:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.280+0000 7f7659ffb700 1 -- 172.21.15.170:0/2821270079 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7664062a80 con 0x7f764c0a36a0 2024-03-27T06:40:37.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.282+0000 7f766af17700 1 -- 172.21.15.170:0/2821270079 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f764c0a36a0 msgr2=0x7f764c0a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.283+0000 7f766af17700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f764c0a36a0 0x7f764c0a5b60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f76600060b0 tx=0x7f766000c040 comp rx=0 tx=0).stop 2024-03-27T06:40:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.283+0000 7f766af17700 1 -- 172.21.15.170:0/2821270079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76641038a0 msgr2=0x7f7664198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.283+0000 7f766af17700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76641038a0 0x7f7664198e30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f765400c950 tx=0x7f765400cc60 comp rx=0 tx=0).stop 2024-03-27T06:40:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.283+0000 7f766af17700 1 -- 172.21.15.170:0/2821270079 shutdown_connections 2024-03-27T06:40:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.283+0000 7f766af17700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7664104240 0x7f766419feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.283+0000 7f766af17700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76641026a0 0x7f76641988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.283+0000 7f766af17700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f764c0a36a0 0x7f764c0a5b60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.283+0000 7f766af17700 1 --2- 172.21.15.170:0/2821270079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76641038a0 0x7f7664198e30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.283+0000 7f766af17700 1 -- 172.21.15.170:0/2821270079 >> 172.21.15.170:0/2821270079 conn(0x7f76640fdc40 msgr2=0x7f7664106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.283+0000 7f766af17700 1 -- 172.21.15.170:0/2821270079 shutdown_connections 2024-03-27T06:40:37.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:37.283+0000 7f766af17700 1 -- 172.21.15.170:0/2821270079 wait complete. 2024-03-27T06:40:37.970 DEBUG:teuthology.orchestra.run.smithi170:osd.11> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.11.service 2024-03-27T06:40:37.973 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-27T06:40:37.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-27T06:40:38.230 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:40:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:38 smithi016 ceph-mon[31908]: from='osd.11 [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T06:40:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:38 smithi016 ceph-mon[31908]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T06:40:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:38 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:38.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:38.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:38.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:38 smithi170 ceph-mon[36438]: from='osd.11 [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T06:40:38.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:38 smithi170 ceph-mon[36438]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T06:40:38.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:38.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:38 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:38.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:38.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:38.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:38 smithi156 ceph-mon[36261]: from='osd.11 [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T06:40:38.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:38 smithi156 ceph-mon[36261]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-27T06:40:38.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:38.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:38 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:39 smithi170 ceph-mon[36438]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:39 smithi170 ceph-mon[36438]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-27T06:40:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:39 smithi170 ceph-mon[36438]: osdmap e79: 12 total, 11 up, 12 in 2024-03-27T06:40:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:39 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:39 smithi170 ceph-mon[36438]: from='osd.11 [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:40:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:39 smithi170 ceph-mon[36438]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:40:39.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:40:39 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:40:39.270+0000 7f60d93ca700 -1 osd.11 0 waiting for initial osdmap 2024-03-27T06:40:39.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:40:39 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:40:39.282+0000 7f60d51bd700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-27T06:40:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:39 smithi016 ceph-mon[31908]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:39 smithi016 ceph-mon[31908]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-27T06:40:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:39 smithi016 ceph-mon[31908]: osdmap e79: 12 total, 11 up, 12 in 2024-03-27T06:40:39.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:39 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:39.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:39 smithi016 ceph-mon[31908]: from='osd.11 [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:40:39.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:39 smithi016 ceph-mon[31908]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:40:39.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:39 smithi156 ceph-mon[36261]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:39.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:39 smithi156 ceph-mon[36261]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-27T06:40:39.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:39 smithi156 ceph-mon[36261]: osdmap e79: 12 total, 11 up, 12 in 2024-03-27T06:40:39.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:39 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:39.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:39 smithi156 ceph-mon[36261]: from='osd.11 [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:40:39.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:39 smithi156 ceph-mon[36261]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-27T06:40:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2428145654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa408101570 msgr2=0x7fa408101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 --2- 172.21.15.16:0/2428145654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa408101570 0x7fa408101990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa4040099c0 tx=0x7fa404009cd0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2428145654 shutdown_connections 2024-03-27T06:40:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 --2- 172.21.15.16:0/2428145654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa408103110 0x7fa4081099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 --2- 172.21.15.16:0/2428145654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa408101570 0x7fa408101990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 --2- 172.21.15.16:0/2428145654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa408102770 0x7fa408102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2428145654 >> 172.21.15.16:0/2428145654 conn(0x7fa4080fcb10 msgr2=0x7fa4080fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2428145654 shutdown_connections 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2428145654 wait complete. 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 Processor -- start 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 -- start start 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa408101570 0x7fa4081989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa408102770 0x7fa408198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.169+0000 7fa40f4ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa408103110 0x7fa40819ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40f4ef700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa40810d2a0 con 0x7fa408101570 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40f4ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fa40810d120 con 0x7fa408103110 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40f4ef700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa40810d420 con 0x7fa408102770 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40d28b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa408101570 0x7fa4081989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40d28b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa408101570 0x7fa4081989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36718/0 (socket says 172.21.15.16:36718) 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40d28b700 1 -- 172.21.15.16:0/2014130101 learned_addr learned my addr 172.21.15.16:0/2014130101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40ca8a700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa408102770 0x7fa408198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40da8c700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa408103110 0x7fa40819ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40d28b700 1 -- 172.21.15.16:0/2014130101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa408102770 msgr2=0x7fa408198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40d28b700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa408102770 0x7fa408198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40d28b700 1 -- 172.21.15.16:0/2014130101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa408103110 msgr2=0x7fa40819ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40d28b700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa408103110 0x7fa40819ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40d28b700 1 -- 172.21.15.16:0/2014130101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4081a0610 con 0x7fa408101570 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40d28b700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa408101570 0x7fa4081989b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa40400b580 tx=0x7fa404005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa3fe7fc700 1 -- 172.21.15.16:0/2014130101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404027070 con 0x7fa408101570 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa3fe7fc700 1 -- 172.21.15.16:0/2014130101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa404004140 con 0x7fa408101570 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2014130101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4081a08a0 con 0x7fa408101570 2024-03-27T06:40:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa3fe7fc700 1 -- 172.21.15.16:0/2014130101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40400f460 con 0x7fa408101570 2024-03-27T06:40:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.170+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2014130101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4081a0dd0 con 0x7fa408101570 2024-03-27T06:40:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.171+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2014130101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa408069300 con 0x7fa408101570 2024-03-27T06:40:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.172+0000 7fa3fe7fc700 1 -- 172.21.15.16:0/2014130101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fa40402a070 con 0x7fa408101570 2024-03-27T06:40:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.173+0000 7fa3fe7fc700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3f40a3720 0x7fa3f40a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.173+0000 7fa40ca8a700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3f40a3720 0x7fa3f40a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.173+0000 7fa3fe7fc700 1 -- 172.21.15.16:0/2014130101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7fa4040d07a0 con 0x7fa408101570 2024-03-27T06:40:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.173+0000 7fa40ca8a700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3f40a3720 0x7fa3f40a5be0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8007c60 tx=0x7fa3f80073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.177+0000 7fa3fe7fc700 1 -- 172.21.15.16:0/2014130101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fa404031070 con 0x7fa408101570 2024-03-27T06:40:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.386+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2014130101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa4081a1190 con 0x7fa408101570 2024-03-27T06:40:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.387+0000 7fa3fe7fc700 1 -- 172.21.15.16:0/2014130101 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fa404013070 con 0x7fa408101570 2024-03-27T06:40:40.382 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:40:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2014130101 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3f40a3720 msgr2=0x7fa3f40a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3f40a3720 0x7fa3f40a5be0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8007c60 tx=0x7fa3f80073d0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2014130101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa408101570 msgr2=0x7fa4081989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa408101570 0x7fa4081989b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa40400b580 tx=0x7fa404005af0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2014130101 shutdown_connections 2024-03-27T06:40:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa408103110 0x7fa40819ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa408101570 0x7fa4081989b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa3f40a3720 0x7fa3f40a5be0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 --2- 172.21.15.16:0/2014130101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa408102770 0x7fa408198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2014130101 >> 172.21.15.16:0/2014130101 conn(0x7fa4080fcb10 msgr2=0x7fa4080fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2014130101 shutdown_connections 2024-03-27T06:40:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:40.389+0000 7fa40f4ef700 1 -- 172.21.15.16:0/2014130101 wait complete. 2024-03-27T06:40:40.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:40 smithi170 ceph-mon[36438]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:40:40.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:40 smithi170 ceph-mon[36438]: osdmap e80: 12 total, 11 up, 12 in 2024-03-27T06:40:40.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:40 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:40.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:40 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:40.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:40 smithi016 ceph-mon[31908]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:40:40.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:40 smithi016 ceph-mon[31908]: osdmap e80: 12 total, 11 up, 12 in 2024-03-27T06:40:40.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:40 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:40.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:40 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:40.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:40 smithi156 ceph-mon[36261]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-27T06:40:40.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:40 smithi156 ceph-mon[36261]: osdmap e80: 12 total, 11 up, 12 in 2024-03-27T06:40:40.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:40 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:40.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:40 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:41.089 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1711521640,"num_in_osds":12,"osd_in_since":1711521619,"num_remapped_pgs":0} 2024-03-27T06:40:41.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-27T06:40:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:40:41.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:41 smithi170 ceph-mon[36438]: purged_snaps scrub starts 2024-03-27T06:40:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:41 smithi170 ceph-mon[36438]: purged_snaps scrub ok 2024-03-27T06:40:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:41 smithi170 ceph-mon[36438]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:41 smithi170 ceph-mon[36438]: osd.11 [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164] boot 2024-03-27T06:40:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:41 smithi170 ceph-mon[36438]: osdmap e81: 12 total, 12 up, 12 in 2024-03-27T06:40:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:41 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:41 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2014130101' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-27T06:40:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:41 smithi016 ceph-mon[31908]: purged_snaps scrub starts 2024-03-27T06:40:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:41 smithi016 ceph-mon[31908]: purged_snaps scrub ok 2024-03-27T06:40:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:41 smithi016 ceph-mon[31908]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:41 smithi016 ceph-mon[31908]: osd.11 [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164] boot 2024-03-27T06:40:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:41 smithi016 ceph-mon[31908]: osdmap e81: 12 total, 12 up, 12 in 2024-03-27T06:40:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:41 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:41 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2014130101' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-27T06:40:41.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:41 smithi156 ceph-mon[36261]: purged_snaps scrub starts 2024-03-27T06:40:41.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:41 smithi156 ceph-mon[36261]: purged_snaps scrub ok 2024-03-27T06:40:41.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:41 smithi156 ceph-mon[36261]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-27T06:40:41.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:41 smithi156 ceph-mon[36261]: osd.11 [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164] boot 2024-03-27T06:40:41.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:41 smithi156 ceph-mon[36261]: osdmap e81: 12 total, 12 up, 12 in 2024-03-27T06:40:41.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:41 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:40:41.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:41 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2014130101' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-27T06:40:42.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: osdmap e82: 12 total, 12 up, 12 in 2024-03-27T06:40:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:42 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:42.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: osdmap e82: 12 total, 12 up, 12 in 2024-03-27T06:40:42.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:42.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:42.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:42 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:42.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: osdmap e82: 12 total, 12 up, 12 in 2024-03-27T06:40:42.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:40:42.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:42.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:42.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:42.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:42 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.231+0000 7fd1a48c3700 1 -- 172.21.15.16:0/2316066878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a0103080 msgr2=0x7fd1a010f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.231+0000 7fd1a48c3700 1 --2- 172.21.15.16:0/2316066878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a0103080 0x7fd1a010f480 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd19800b0a0 tx=0x7fd19800b3b0 comp rx=0 tx=0).stop 2024-03-27T06:40:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.231+0000 7fd1a48c3700 1 -- 172.21.15.16:0/2316066878 shutdown_connections 2024-03-27T06:40:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.231+0000 7fd1a48c3700 1 --2- 172.21.15.16:0/2316066878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a0103080 0x7fd1a010f480 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.231+0000 7fd1a48c3700 1 --2- 172.21.15.16:0/2316066878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1a01086b0 0x7fd1a0108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.231+0000 7fd1a48c3700 1 --2- 172.21.15.16:0/2316066878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1a01026c0 0x7fd1a0102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.231+0000 7fd1a48c3700 1 -- 172.21.15.16:0/2316066878 >> 172.21.15.16:0/2316066878 conn(0x7fd1a00796d0 msgr2=0x7fd1a0079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.232+0000 7fd1a48c3700 1 -- 172.21.15.16:0/2316066878 shutdown_connections 2024-03-27T06:40:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.232+0000 7fd1a48c3700 1 -- 172.21.15.16:0/2316066878 wait complete. 2024-03-27T06:40:43.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.232+0000 7fd1a48c3700 1 Processor -- start 2024-03-27T06:40:43.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.232+0000 7fd1a48c3700 1 -- start start 2024-03-27T06:40:43.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.232+0000 7fd1a48c3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1a01026c0 0x7fd1a0198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:43.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.232+0000 7fd1a48c3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1a0103080 0x7fd1a0199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:43.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd1a48c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a01086b0 0x7fd1a019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:43.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd1a48c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a0112a80 con 0x7fd1a01086b0 2024-03-27T06:40:43.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd1a48c3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a0112900 con 0x7fd1a0103080 2024-03-27T06:40:43.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd1a48c3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a0112c00 con 0x7fd1a01026c0 2024-03-27T06:40:43.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a01086b0 0x7fd1a019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:43.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a01086b0 0x7fd1a019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36730/0 (socket says 172.21.15.16:36730) 2024-03-27T06:40:43.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1a0103080 0x7fd1a0199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19ed9d700 1 -- 172.21.15.16:0/616751854 learned_addr learned my addr 172.21.15.16:0/616751854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19e59c700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1a01026c0 0x7fd1a0198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19ed9d700 1 -- 172.21.15.16:0/616751854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1a01026c0 msgr2=0x7fd1a0198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19ed9d700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1a01026c0 0x7fd1a0198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19ed9d700 1 -- 172.21.15.16:0/616751854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1a0103080 msgr2=0x7fd1a0199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19ed9d700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1a0103080 0x7fd1a0199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19ed9d700 1 -- 172.21.15.16:0/616751854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a019de40 con 0x7fd1a01086b0 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19dd9b700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1a0103080 0x7fd1a0199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd19ed9d700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a01086b0 0x7fd1a019d700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd19800b070 tx=0x7fd198004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd18f7fe700 1 -- 172.21.15.16:0/616751854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198021070 con 0x7fd1a01086b0 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd18f7fe700 1 -- 172.21.15.16:0/616751854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd198007b80 con 0x7fd1a01086b0 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd18f7fe700 1 -- 172.21.15.16:0/616751854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198005370 con 0x7fd1a01086b0 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.233+0000 7fd1a48c3700 1 -- 172.21.15.16:0/616751854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a019e0d0 con 0x7fd1a01086b0 2024-03-27T06:40:43.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.234+0000 7fd1a48c3700 1 -- 172.21.15.16:0/616751854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a01a59b0 con 0x7fd1a01086b0 2024-03-27T06:40:43.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.234+0000 7fd1a48c3700 1 -- 172.21.15.16:0/616751854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a0069300 con 0x7fd1a01086b0 2024-03-27T06:40:43.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.236+0000 7fd18f7fe700 1 -- 172.21.15.16:0/616751854 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fd198024070 con 0x7fd1a01086b0 2024-03-27T06:40:43.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.236+0000 7fd18f7fe700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1880a3720 0x7fd1880a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:43.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.236+0000 7fd19e59c700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1880a3720 0x7fd1880a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:43.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.236+0000 7fd18f7fe700 1 -- 172.21.15.16:0/616751854 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd19800f020 con 0x7fd1a01086b0 2024-03-27T06:40:43.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.237+0000 7fd19e59c700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1880a3720 0x7fd1880a5be0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd190009be0 tx=0x7fd190009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:43.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.240+0000 7fd18f7fe700 1 -- 172.21.15.16:0/616751854 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd19802b020 con 0x7fd1a01086b0 2024-03-27T06:40:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.449+0000 7fd1a48c3700 1 -- 172.21.15.16:0/616751854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1a002dd30 con 0x7fd1a01086b0 2024-03-27T06:40:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.451+0000 7fd18f7fe700 1 -- 172.21.15.16:0/616751854 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18410 (secure 0 0 0) 0x7fd19809e1a0 con 0x7fd1a01086b0 2024-03-27T06:40:43.446 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:40:43.447 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:40:41.273117+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:40:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 -- 172.21.15.16:0/616751854 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1880a3720 msgr2=0x7fd1880a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1880a3720 0x7fd1880a5be0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd190009be0 tx=0x7fd190009260 comp rx=0 tx=0).stop 2024-03-27T06:40:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 -- 172.21.15.16:0/616751854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a01086b0 msgr2=0x7fd1a019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a01086b0 0x7fd1a019d700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd19800b070 tx=0x7fd198004d00 comp rx=0 tx=0).stop 2024-03-27T06:40:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 -- 172.21.15.16:0/616751854 shutdown_connections 2024-03-27T06:40:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a01086b0 0x7fd1a019d700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd1880a3720 0x7fd1880a5be0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1a01026c0 0x7fd1a0198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 --2- 172.21.15.16:0/616751854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1a0103080 0x7fd1a0199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 -- 172.21.15.16:0/616751854 >> 172.21.15.16:0/616751854 conn(0x7fd1a00796d0 msgr2=0x7fd1a0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 -- 172.21.15.16:0/616751854 shutdown_connections 2024-03-27T06:40:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:43.453+0000 7fd1a48c3700 1 -- 172.21.15.16:0/616751854 wait complete. 2024-03-27T06:40:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:43 smithi156 ceph-mon[36261]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:43 smithi156 ceph-mon[36261]: Detected new or changed devices on smithi170 2024-03-27T06:40:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:43 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi170 to 4270M 2024-03-27T06:40:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:43 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:43 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:43 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:43.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:43 smithi170 ceph-mon[36438]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:43 smithi170 ceph-mon[36438]: Detected new or changed devices on smithi170 2024-03-27T06:40:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:43 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi170 to 4270M 2024-03-27T06:40:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:43 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:43 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:43 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:43.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:43 smithi016 ceph-mon[31908]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:43.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:43 smithi016 ceph-mon[31908]: Detected new or changed devices on smithi170 2024-03-27T06:40:43.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:43 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi170 to 4270M 2024-03-27T06:40:43.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:43.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:43.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:43 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:44.161 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:40:44.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-27T06:40:44.408 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:40:44.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:44 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/616751854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:40:44.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:44 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/616751854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:40:44.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:44 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/616751854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:40:45.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:45 smithi170 ceph-mon[36438]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:45 smithi016 ceph-mon[31908]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:45.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:45 smithi156 ceph-mon[36261]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:46.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.261+0000 7f052fca3700 1 -- 172.21.15.16:0/1732438740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05281015a0 msgr2=0x7f0528101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.261+0000 7f052fca3700 1 --2- 172.21.15.16:0/1732438740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05281015a0 0x7f0528101a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f05180099c0 tx=0x7f0518009cd0 comp rx=0 tx=0).stop 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.261+0000 7f052fca3700 1 -- 172.21.15.16:0/1732438740 shutdown_connections 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.261+0000 7f052fca3700 1 --2- 172.21.15.16:0/1732438740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0528101f60 0x7f052810e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.261+0000 7f052fca3700 1 --2- 172.21.15.16:0/1732438740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05281075e0 0x7f05281079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.261+0000 7f052fca3700 1 --2- 172.21.15.16:0/1732438740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05281015a0 0x7f0528101a20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.261+0000 7f052fca3700 1 -- 172.21.15.16:0/1732438740 >> 172.21.15.16:0/1732438740 conn(0x7f0528077590 msgr2=0x7f05280779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.262+0000 7f052fca3700 1 -- 172.21.15.16:0/1732438740 shutdown_connections 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.262+0000 7f052fca3700 1 -- 172.21.15.16:0/1732438740 wait complete. 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.262+0000 7f052fca3700 1 Processor -- start 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.262+0000 7f052fca3700 1 -- start start 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.262+0000 7f052fca3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05281015a0 0x7f0528198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.262+0000 7f052fca3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0528101f60 0x7f0528199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.262+0000 7f052fca3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05281075e0 0x7f052819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.262+0000 7f052fca3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0528111950 con 0x7f0528101f60 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.262+0000 7f052fca3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f05281117d0 con 0x7f05281075e0 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.262+0000 7f052fca3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0528111ad0 con 0x7f05281015a0 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052d23e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0528101f60 0x7f0528199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052d23e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0528101f60 0x7f0528199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46408/0 (socket says 172.21.15.16:46408) 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052d23e700 1 -- 172.21.15.16:0/923600143 learned_addr learned my addr 172.21.15.16:0/923600143 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052da3f700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05281015a0 0x7f0528198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052e240700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05281075e0 0x7f052819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052d23e700 1 -- 172.21.15.16:0/923600143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05281015a0 msgr2=0x7f0528198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052d23e700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05281015a0 0x7f0528198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052d23e700 1 -- 172.21.15.16:0/923600143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05281075e0 msgr2=0x7f052819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052d23e700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05281075e0 0x7f052819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052d23e700 1 -- 172.21.15.16:0/923600143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052819ddc0 con 0x7f0528101f60 2024-03-27T06:40:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052d23e700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0528101f60 0x7f0528199340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0518000c00 tx=0x7f0518005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f051effd700 1 -- 172.21.15.16:0/923600143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518027070 con 0x7f0528101f60 2024-03-27T06:40:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f051effd700 1 -- 172.21.15.16:0/923600143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0518004040 con 0x7f0528101f60 2024-03-27T06:40:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052fca3700 1 -- 172.21.15.16:0/923600143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052819e050 con 0x7f0528101f60 2024-03-27T06:40:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f051effd700 1 -- 172.21.15.16:0/923600143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051800f460 con 0x7f0528101f60 2024-03-27T06:40:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.263+0000 7f052fca3700 1 -- 172.21.15.16:0/923600143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05281a5930 con 0x7f0528101f60 2024-03-27T06:40:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.264+0000 7f052fca3700 1 -- 172.21.15.16:0/923600143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0528102c50 con 0x7f0528101f60 2024-03-27T06:40:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.265+0000 7f051effd700 1 -- 172.21.15.16:0/923600143 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f051802a070 con 0x7f0528101f60 2024-03-27T06:40:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.266+0000 7f051effd700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f05140a36a0 0x7f05140a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.266+0000 7f052da3f700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f05140a36a0 0x7f05140a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.266+0000 7f051effd700 1 -- 172.21.15.16:0/923600143 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f05180d0c30 con 0x7f0528101f60 2024-03-27T06:40:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.267+0000 7f052da3f700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f05140a36a0 0x7f05140a5b60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0524005950 tx=0x7f052400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.270+0000 7f051effd700 1 -- 172.21.15.16:0/923600143 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f0518031020 con 0x7f0528101f60 2024-03-27T06:40:46.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.474+0000 7f052fca3700 1 -- 172.21.15.16:0/923600143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05281a5b00 con 0x7f0528101f60 2024-03-27T06:40:46.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.474+0000 7f051effd700 1 -- 172.21.15.16:0/923600143 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f0518013070 con 0x7f0528101f60 2024-03-27T06:40:46.470 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:40:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.476+0000 7f052fca3700 1 -- 172.21.15.16:0/923600143 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f05140a36a0 msgr2=0x7f05140a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.476+0000 7f052fca3700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f05140a36a0 0x7f05140a5b60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0524005950 tx=0x7f052400b410 comp rx=0 tx=0).stop 2024-03-27T06:40:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.476+0000 7f052fca3700 1 -- 172.21.15.16:0/923600143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0528101f60 msgr2=0x7f0528199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.476+0000 7f052fca3700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0528101f60 0x7f0528199340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0518000c00 tx=0x7f0518005590 comp rx=0 tx=0).stop 2024-03-27T06:40:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.477+0000 7f052fca3700 1 -- 172.21.15.16:0/923600143 shutdown_connections 2024-03-27T06:40:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.477+0000 7f052fca3700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05281075e0 0x7f052819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:46.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.477+0000 7f052fca3700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f05140a36a0 0x7f05140a5b60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.477+0000 7f052fca3700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05281015a0 0x7f0528198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.477+0000 7f052fca3700 1 --2- 172.21.15.16:0/923600143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0528101f60 0x7f0528199340 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.477+0000 7f052fca3700 1 -- 172.21.15.16:0/923600143 >> 172.21.15.16:0/923600143 conn(0x7f0528077590 msgr2=0x7f05281007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.477+0000 7f052fca3700 1 -- 172.21.15.16:0/923600143 shutdown_connections 2024-03-27T06:40:46.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:46.477+0000 7f052fca3700 1 -- 172.21.15.16:0/923600143 wait complete. 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.a on smithi016 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.d on smithi016 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.g on smithi016 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.j on smithi016 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.b on smithi156 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.e on smithi156 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.h on smithi156 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.k on smithi156 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.c on smithi170 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.f on smithi170 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.i on smithi170 2024-03-27T06:40:47.388 INFO:tasks.cephadm:Adding mds.l on smithi170 2024-03-27T06:40:47.389 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph orch apply mds all '12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l' 2024-03-27T06:40:47.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:47 smithi170 ceph-mon[36438]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:47.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:47 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/923600143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:40:47.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:47 smithi016 ceph-mon[31908]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:47.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:47 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/923600143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:40:47.639 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:40:47.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:47 smithi156 ceph-mon[36261]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:47 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/923600143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:40:49.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.385+0000 7ff54b924700 1 -- 172.21.15.170:0/2649407632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff544103950 msgr2=0x7ff544103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:49.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.385+0000 7ff54b924700 1 --2- 172.21.15.170:0/2649407632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff544103950 0x7ff544103db0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff534009990 tx=0x7ff534009ca0 comp rx=0 tx=0).stop 2024-03-27T06:40:49.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.385+0000 7ff54b924700 1 -- 172.21.15.170:0/2649407632 shutdown_connections 2024-03-27T06:40:49.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.385+0000 7ff54b924700 1 --2- 172.21.15.170:0/2649407632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5441042f0 0x7ff54410aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:49.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.385+0000 7ff54b924700 1 --2- 172.21.15.170:0/2649407632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff544100970 0x7ff544100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.385+0000 7ff54b924700 1 --2- 172.21.15.170:0/2649407632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff544103950 0x7ff544103db0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.385+0000 7ff54b924700 1 -- 172.21.15.170:0/2649407632 >> 172.21.15.170:0/2649407632 conn(0x7ff5440fc2d0 msgr2=0x7ff5440fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.385+0000 7ff54b924700 1 -- 172.21.15.170:0/2649407632 shutdown_connections 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.386+0000 7ff54b924700 1 -- 172.21.15.170:0/2649407632 wait complete. 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.386+0000 7ff54b924700 1 Processor -- start 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.386+0000 7ff54b924700 1 -- start start 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.386+0000 7ff54b924700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff544100970 0x7ff544198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.386+0000 7ff54b924700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544103950 0x7ff544198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.386+0000 7ff54b924700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5441042f0 0x7ff54419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff54b924700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff54410e3a0 con 0x7ff544103950 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff54b924700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ff54410e220 con 0x7ff544100970 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff54b924700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff54410e520 con 0x7ff5441042f0 2024-03-27T06:40:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff549ec1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5441042f0 0x7ff54419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff549ec1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5441042f0 0x7ff54419fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58528/0 (socket says 172.21.15.170:58528) 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff549ec1700 1 -- 172.21.15.170:0/4015167554 learned_addr learned my addr 172.21.15.170:0/4015167554 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff548ebf700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544103950 0x7ff544198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff5496c0700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff544100970 0x7ff544198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff549ec1700 1 -- 172.21.15.170:0/4015167554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff544100970 msgr2=0x7ff544198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff549ec1700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff544100970 0x7ff544198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff549ec1700 1 -- 172.21.15.170:0/4015167554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544103950 msgr2=0x7ff544198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff549ec1700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544103950 0x7ff544198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff549ec1700 1 -- 172.21.15.170:0/4015167554 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff544101740 con 0x7ff5441042f0 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff5496c0700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff544100970 0x7ff544198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff549ec1700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5441042f0 0x7ff54419fe20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff53c007f10 tx=0x7ff53c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff53a7fc700 1 -- 172.21.15.170:0/4015167554 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c004bb0 con 0x7ff5441042f0 2024-03-27T06:40:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff53a7fc700 1 -- 172.21.15.170:0/4015167554 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff53c004d50 con 0x7ff5441042f0 2024-03-27T06:40:49.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff54b924700 1 -- 172.21.15.170:0/4015167554 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff544101a30 con 0x7ff5441042f0 2024-03-27T06:40:49.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff53a7fc700 1 -- 172.21.15.170:0/4015167554 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0133f0 con 0x7ff5441042f0 2024-03-27T06:40:49.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.387+0000 7ff54b924700 1 -- 172.21.15.170:0/4015167554 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff544101f70 con 0x7ff5441042f0 2024-03-27T06:40:49.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.388+0000 7ff54b924700 1 -- 172.21.15.170:0/4015167554 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff544069300 con 0x7ff5441042f0 2024-03-27T06:40:49.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.389+0000 7ff53a7fc700 1 -- 172.21.15.170:0/4015167554 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7ff53c01a020 con 0x7ff5441042f0 2024-03-27T06:40:49.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.390+0000 7ff53a7fc700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff5300a3720 0x7ff5300a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:49.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.390+0000 7ff53a7fc700 1 -- 172.21.15.170:0/4015167554 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff53c0cf6c0 con 0x7ff5441042f0 2024-03-27T06:40:49.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.390+0000 7ff5496c0700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff5300a3720 0x7ff5300a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:49.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.391+0000 7ff5496c0700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff5300a3720 0x7ff5300a5be0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff54406ab80 tx=0x7ff54000a680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:49.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.394+0000 7ff53a7fc700 1 -- 172.21.15.170:0/4015167554 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff53c010070 con 0x7ff5441042f0 2024-03-27T06:40:49.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.599+0000 7ff54b924700 1 -- 172.21.15.170:0/4015167554 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}) -- 0x7ff544109850 con 0x7ff5300a3720 2024-03-27T06:40:49.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:49 smithi170 ceph-mon[36438]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:49.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.607+0000 7ff53a7fc700 1 -- 172.21.15.170:0/4015167554 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff544109850 con 0x7ff5300a3720 2024-03-27T06:40:49.602 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mds.all update... 2024-03-27T06:40:49.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 -- 172.21.15.170:0/4015167554 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff5300a3720 msgr2=0x7ff5300a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:49.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff5300a3720 0x7ff5300a5be0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff54406ab80 tx=0x7ff54000a680 comp rx=0 tx=0).stop 2024-03-27T06:40:49.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 -- 172.21.15.170:0/4015167554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5441042f0 msgr2=0x7ff54419fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:49.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5441042f0 0x7ff54419fe20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff53c007f10 tx=0x7ff53c00d3b0 comp rx=0 tx=0).stop 2024-03-27T06:40:49.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 -- 172.21.15.170:0/4015167554 shutdown_connections 2024-03-27T06:40:49.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff5300a3720 0x7ff5300a5be0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:49.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5441042f0 0x7ff54419fe20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:49.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff544100970 0x7ff544198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:49.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 --2- 172.21.15.170:0/4015167554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544103950 0x7ff544198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:49.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 -- 172.21.15.170:0/4015167554 >> 172.21.15.170:0/4015167554 conn(0x7ff5440fc2d0 msgr2=0x7ff544106b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:49.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 -- 172.21.15.170:0/4015167554 shutdown_connections 2024-03-27T06:40:49.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-27T06:40:49.609+0000 7ff54b924700 1 -- 172.21.15.170:0/4015167554 wait complete. 2024-03-27T06:40:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:49 smithi016 ceph-mon[31908]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:49.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:49 smithi156 ceph-mon[36261]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:50.364 DEBUG:teuthology.orchestra.run.smithi016:mds.a> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.a.service 2024-03-27T06:40:50.367 DEBUG:teuthology.orchestra.run.smithi016:mds.d> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.d.service 2024-03-27T06:40:50.409 DEBUG:teuthology.orchestra.run.smithi016:mds.g> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.g.service 2024-03-27T06:40:50.452 DEBUG:teuthology.orchestra.run.smithi016:mds.j> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.j.service 2024-03-27T06:40:50.495 DEBUG:teuthology.orchestra.run.smithi156:mds.b> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.b.service 2024-03-27T06:40:50.498 DEBUG:teuthology.orchestra.run.smithi156:mds.e> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.e.service 2024-03-27T06:40:50.543 DEBUG:teuthology.orchestra.run.smithi156:mds.h> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.h.service 2024-03-27T06:40:50.586 DEBUG:teuthology.orchestra.run.smithi156:mds.k> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.k.service 2024-03-27T06:40:50.629 DEBUG:teuthology.orchestra.run.smithi170:mds.c> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.c.service 2024-03-27T06:40:50.631 DEBUG:teuthology.orchestra.run.smithi170:mds.f> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.f.service 2024-03-27T06:40:50.674 DEBUG:teuthology.orchestra.run.smithi170:mds.i> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.i.service 2024-03-27T06:40:50.717 DEBUG:teuthology.orchestra.run.smithi170:mds.l> sudo journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.l.service 2024-03-27T06:40:50.760 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-27T06:40:50.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:40:50.787 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:50.787 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:40:50.787 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-03-27T06:40:50.787 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:50.787 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:50.787 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:50.787 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:50.787 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:50.788 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:40:50.788 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:40:50.788 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:50.788 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:50 smithi016 ceph-mon[31908]: Deploying daemon mds.l on smithi170 2024-03-27T06:40:50.874 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:50.875 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:40:50.875 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-03-27T06:40:50.875 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:50.875 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:50.876 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:50.876 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:50.876 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:50.876 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:40:50.876 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:40:50.876 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:50.876 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:50 smithi170 ceph-mon[36438]: Deploying daemon mds.l on smithi170 2024-03-27T06:40:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.883+0000 7f4a8631e640 1 -- 172.21.15.16:0/1445612391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a80100a70 msgr2=0x7f4a80100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.883+0000 7f4a8631e640 1 --2- 172.21.15.16:0/1445612391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a80100a70 0x7f4a80100e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009900 tx=0x7f4a7002bc30 comp rx=0 tx=0).stop 2024-03-27T06:40:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.883+0000 7f4a8631e640 1 -- 172.21.15.16:0/1445612391 shutdown_connections 2024-03-27T06:40:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.883+0000 7f4a8631e640 1 --2- 172.21.15.16:0/1445612391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a80102620 0x7f4a80108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.883+0000 7f4a8631e640 1 --2- 172.21.15.16:0/1445612391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a80101c60 0x7f4a801020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.883+0000 7f4a8631e640 1 --2- 172.21.15.16:0/1445612391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a80100a70 0x7f4a80100e90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.883+0000 7f4a8631e640 1 -- 172.21.15.16:0/1445612391 >> 172.21.15.16:0/1445612391 conn(0x7f4a800fbfe0 msgr2=0x7f4a800fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.883+0000 7f4a8631e640 1 -- 172.21.15.16:0/1445612391 shutdown_connections 2024-03-27T06:40:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.883+0000 7f4a8631e640 1 -- 172.21.15.16:0/1445612391 wait complete. 2024-03-27T06:40:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.883+0000 7f4a8631e640 1 Processor -- start 2024-03-27T06:40:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.883+0000 7f4a8631e640 1 -- start start 2024-03-27T06:40:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a8631e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a80100a70 0x7f4a80069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a8631e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a80101c60 0x7f4a8019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a8631e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a80102620 0x7f4a8019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a8631e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8010c7c0 con 0x7f4a80100a70 2024-03-27T06:40:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a8631e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8010c640 con 0x7f4a80102620 2024-03-27T06:40:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a8631e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8010c940 con 0x7f4a80101c60 2024-03-27T06:40:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a80100a70 0x7f4a80069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a80100a70 0x7f4a80069840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46422/0 (socket says 172.21.15.16:46422) 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a7ffff640 1 -- 172.21.15.16:0/4159794839 learned_addr learned my addr 172.21.15.16:0/4159794839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a7f7fe640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a80101c60 0x7f4a8019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a848bb640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a80102620 0x7f4a8019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a7ffff640 1 -- 172.21.15.16:0/4159794839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a80101c60 msgr2=0x7f4a8019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a7ffff640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a80101c60 0x7f4a8019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a7ffff640 1 -- 172.21.15.16:0/4159794839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a80102620 msgr2=0x7f4a8019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a7ffff640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a80102620 0x7f4a8019f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a7ffff640 1 -- 172.21.15.16:0/4159794839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8019fb30 con 0x7f4a80100a70 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a848bb640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a80102620 0x7f4a8019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a7ffff640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a80100a70 0x7f4a80069840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4a70005ab0 tx=0x7f4a7002c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.884+0000 7f4a7d7fa640 1 -- 172.21.15.16:0/4159794839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70043020 con 0x7f4a80100a70 2024-03-27T06:40:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.885+0000 7f4a8631e640 1 -- 172.21.15.16:0/4159794839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8019fdc0 con 0x7f4a80100a70 2024-03-27T06:40:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.885+0000 7f4a7d7fa640 1 -- 172.21.15.16:0/4159794839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4a70004350 con 0x7f4a80100a70 2024-03-27T06:40:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.885+0000 7f4a7d7fa640 1 -- 172.21.15.16:0/4159794839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70003d70 con 0x7f4a80100a70 2024-03-27T06:40:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.885+0000 7f4a8631e640 1 -- 172.21.15.16:0/4159794839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a801a0290 con 0x7f4a80100a70 2024-03-27T06:40:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.886+0000 7f4a8631e640 1 -- 172.21.15.16:0/4159794839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a80107d90 con 0x7f4a80100a70 2024-03-27T06:40:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.887+0000 7f4a7d7fa640 1 -- 172.21.15.16:0/4159794839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f4a70046020 con 0x7f4a80100a70 2024-03-27T06:40:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.887+0000 7f4a7d7fa640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a680a3610 0x7f4a680a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.887+0000 7f4a7d7fa640 1 -- 172.21.15.16:0/4159794839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4a700f1b60 con 0x7f4a80100a70 2024-03-27T06:40:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.887+0000 7f4a7f7fe640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a680a3610 0x7f4a680a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.888+0000 7f4a7f7fe640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a680a3610 0x7f4a680a5ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4a800695b0 tx=0x7f4a7400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:50.892+0000 7f4a7d7fa640 1 -- 172.21.15.16:0/4159794839 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4a7004d020 con 0x7f4a80100a70 2024-03-27T06:40:50.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:50.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:40:50.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-03-27T06:40:50.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:50.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:40:50.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:50.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:40:50.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:50.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:40:50.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:40:50.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:50.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:50 smithi156 ceph-mon[36261]: Deploying daemon mds.l on smithi170 2024-03-27T06:40:51.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.078+0000 7f4a8631e640 1 -- 172.21.15.16:0/4159794839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a8002d520 con 0x7f4a80100a70 2024-03-27T06:40:51.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.079+0000 7f4a7d7fa640 1 -- 172.21.15.16:0/4159794839 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18410 (secure 0 0 0) 0x7f4a700bebd0 con 0x7f4a80100a70 2024-03-27T06:40:51.075 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:40:51.075 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:40:41.273117+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:40:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 -- 172.21.15.16:0/4159794839 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a680a3610 msgr2=0x7f4a680a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a680a3610 0x7f4a680a5ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4a800695b0 tx=0x7f4a7400b410 comp rx=0 tx=0).stop 2024-03-27T06:40:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 -- 172.21.15.16:0/4159794839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a80100a70 msgr2=0x7f4a80069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a80100a70 0x7f4a80069840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4a70005ab0 tx=0x7f4a7002c7a0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 -- 172.21.15.16:0/4159794839 shutdown_connections 2024-03-27T06:40:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a680a3610 0x7f4a680a5ad0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a80102620 0x7f4a8019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a80101c60 0x7f4a8019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 --2- 172.21.15.16:0/4159794839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a80100a70 0x7f4a80069840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 -- 172.21.15.16:0/4159794839 >> 172.21.15.16:0/4159794839 conn(0x7f4a800fbfe0 msgr2=0x7f4a80104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 -- 172.21.15.16:0/4159794839 shutdown_connections 2024-03-27T06:40:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.081+0000 7f4a8631e640 1 -- 172.21.15.16:0/4159794839 wait complete. 2024-03-27T06:40:51.088 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:40:51.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T06:40:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.238+0000 7f81f26df640 1 -- 172.21.15.16:0/1525606927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ec102d60 msgr2=0x7f81ec1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.238+0000 7f81f26df640 1 --2- 172.21.15.16:0/1525606927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ec102d60 0x7f81ec1031e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009900 tx=0x7f81d402bc30 comp rx=0 tx=0).stop 2024-03-27T06:40:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.238+0000 7f81f26df640 1 -- 172.21.15.16:0/1525606927 shutdown_connections 2024-03-27T06:40:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.238+0000 7f81f26df640 1 --2- 172.21.15.16:0/1525606927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec103720 0x7f81ec109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.238+0000 7f81f26df640 1 --2- 172.21.15.16:0/1525606927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ec102d60 0x7f81ec1031e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.238+0000 7f81f26df640 1 --2- 172.21.15.16:0/1525606927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec101b70 0x7f81ec101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.238+0000 7f81f26df640 1 -- 172.21.15.16:0/1525606927 >> 172.21.15.16:0/1525606927 conn(0x7f81ec0fd0c0 msgr2=0x7f81ec0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.238+0000 7f81f26df640 1 -- 172.21.15.16:0/1525606927 shutdown_connections 2024-03-27T06:40:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.238+0000 7f81f26df640 1 -- 172.21.15.16:0/1525606927 wait complete. 2024-03-27T06:40:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81f26df640 1 Processor -- start 2024-03-27T06:40:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81f26df640 1 -- start start 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81f26df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec101b70 0x7f81ec075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81f26df640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec102d60 0x7f81ec073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81f26df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ec103720 0x7f81ec074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81f26df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec10d8d0 con 0x7f81ec103720 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81f26df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec10d750 con 0x7f81ec101b70 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81f26df640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec10da50 con 0x7f81ec102d60 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81f0c7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ec103720 0x7f81ec074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81f0c7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ec103720 0x7f81ec074480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46428/0 (socket says 172.21.15.16:46428) 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81f0c7c640 1 -- 172.21.15.16:0/611452387 learned_addr learned my addr 172.21.15.16:0/611452387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81eb7fe640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec102d60 0x7f81ec073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.239+0000 7f81ebfff640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec101b70 0x7f81ec075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.240+0000 7f81f0c7c640 1 -- 172.21.15.16:0/611452387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec102d60 msgr2=0x7f81ec073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.240+0000 7f81f0c7c640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec102d60 0x7f81ec073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.240+0000 7f81f0c7c640 1 -- 172.21.15.16:0/611452387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec101b70 msgr2=0x7f81ec075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.240+0000 7f81f0c7c640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec101b70 0x7f81ec075900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.240+0000 7f81f0c7c640 1 -- 172.21.15.16:0/611452387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ec074d10 con 0x7f81ec103720 2024-03-27T06:40:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.240+0000 7f81f0c7c640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ec103720 0x7f81ec074480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f81e400eaa0 tx=0x7f81e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.240+0000 7f81e97fa640 1 -- 172.21.15.16:0/611452387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e400cdb0 con 0x7f81ec103720 2024-03-27T06:40:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.240+0000 7f81e97fa640 1 -- 172.21.15.16:0/611452387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f81e4004b20 con 0x7f81ec103720 2024-03-27T06:40:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.240+0000 7f81f26df640 1 -- 172.21.15.16:0/611452387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ec1a1fe0 con 0x7f81ec103720 2024-03-27T06:40:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.240+0000 7f81e97fa640 1 -- 172.21.15.16:0/611452387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011760 con 0x7f81ec103720 2024-03-27T06:40:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.240+0000 7f81f26df640 1 -- 172.21.15.16:0/611452387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ec1a2450 con 0x7f81ec103720 2024-03-27T06:40:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.241+0000 7f81f26df640 1 -- 172.21.15.16:0/611452387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ec108e90 con 0x7f81ec103720 2024-03-27T06:40:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.242+0000 7f81e97fa640 1 -- 172.21.15.16:0/611452387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f81e4004cd0 con 0x7f81ec103720 2024-03-27T06:40:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.243+0000 7f81e97fa640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81d80a3610 0x7f81d80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.243+0000 7f81ebfff640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81d80a3610 0x7f81d80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.243+0000 7f81e97fa640 1 -- 172.21.15.16:0/611452387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f81e40cfae0 con 0x7f81ec103720 2024-03-27T06:40:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.243+0000 7f81ebfff640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81d80a3610 0x7f81d80a5ad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f81dc005d00 tx=0x7f81dc005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.248+0000 7f81e97fa640 1 -- 172.21.15.16:0/611452387 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f81e4016030 con 0x7f81ec103720 2024-03-27T06:40:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.431+0000 7f81f26df640 1 -- 172.21.15.16:0/611452387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81ec02d590 con 0x7f81ec103720 2024-03-27T06:40:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.431+0000 7f81e97fa640 1 -- 172.21.15.16:0/611452387 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f81e409cb50 con 0x7f81ec103720 2024-03-27T06:40:51.427 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 -- 172.21.15.16:0/611452387 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81d80a3610 msgr2=0x7f81d80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81d80a3610 0x7f81d80a5ad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f81dc005d00 tx=0x7f81dc005c50 comp rx=0 tx=0).stop 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 -- 172.21.15.16:0/611452387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ec103720 msgr2=0x7f81ec074480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ec103720 0x7f81ec074480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f81e400eaa0 tx=0x7f81e400ef60 comp rx=0 tx=0).stop 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 -- 172.21.15.16:0/611452387 shutdown_connections 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81d80a3610 0x7f81d80a5ad0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ec103720 0x7f81ec074480 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec102d60 0x7f81ec073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 --2- 172.21.15.16:0/611452387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81ec101b70 0x7f81ec075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 -- 172.21.15.16:0/611452387 >> 172.21.15.16:0/611452387 conn(0x7f81ec0fd0c0 msgr2=0x7f81ec106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 -- 172.21.15.16:0/611452387 shutdown_connections 2024-03-27T06:40:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.433+0000 7f81f26df640 1 -- 172.21.15.16:0/611452387 wait complete. 2024-03-27T06:40:51.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:40:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.547+0000 7f72771e2640 1 -- 172.21.15.16:0/3380024638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270100af0 msgr2=0x7f7270100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.547+0000 7f72771e2640 1 --2- 172.21.15.16:0/3380024638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270100af0 0x7f7270100f10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7264009900 tx=0x7f726402bc30 comp rx=0 tx=0).stop 2024-03-27T06:40:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.547+0000 7f72771e2640 1 -- 172.21.15.16:0/3380024638 shutdown_connections 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.547+0000 7f72771e2640 1 --2- 172.21.15.16:0/3380024638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72701026a0 0x7f7270108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.547+0000 7f72771e2640 1 --2- 172.21.15.16:0/3380024638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7270101ce0 0x7f7270102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.547+0000 7f72771e2640 1 --2- 172.21.15.16:0/3380024638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270100af0 0x7f7270100f10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.547+0000 7f72771e2640 1 -- 172.21.15.16:0/3380024638 >> 172.21.15.16:0/3380024638 conn(0x7f72700fc080 msgr2=0x7f72700fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.547+0000 7f72771e2640 1 -- 172.21.15.16:0/3380024638 shutdown_connections 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.547+0000 7f72771e2640 1 -- 172.21.15.16:0/3380024638 wait complete. 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f72771e2640 1 Processor -- start 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f72771e2640 1 -- start start 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f72771e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270100af0 0x7f727006ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f72771e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7270101ce0 0x7f727019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f72771e2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72701026a0 0x7f727019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f72771e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f727010c7d0 con 0x7f7270100af0 2024-03-27T06:40:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f72771e2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f727010c650 con 0x7f72701026a0 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f72771e2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f727010c950 con 0x7f7270101ce0 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f7274f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270100af0 0x7f727006ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f7274f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270100af0 0x7f727006ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46450/0 (socket says 172.21.15.16:46450) 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f7274f7e640 1 -- 172.21.15.16:0/2912684877 learned_addr learned my addr 172.21.15.16:0/2912684877 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f726ffff640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7270101ce0 0x7f727019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f727577f640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72701026a0 0x7f727019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f7274f7e640 1 -- 172.21.15.16:0/2912684877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7270101ce0 msgr2=0x7f727019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f7274f7e640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7270101ce0 0x7f727019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f7274f7e640 1 -- 172.21.15.16:0/2912684877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72701026a0 msgr2=0x7f727019f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f7274f7e640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72701026a0 0x7f727019f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f7274f7e640 1 -- 172.21.15.16:0/2912684877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727019fba0 con 0x7f7270100af0 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f726ffff640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7270101ce0 0x7f727019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:40:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f727577f640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72701026a0 0x7f727019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:40:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f7274f7e640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270100af0 0x7f727006ad10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7264005ab0 tx=0x7f726402c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f726dffb640 1 -- 172.21.15.16:0/2912684877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264043020 con 0x7f7270100af0 2024-03-27T06:40:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f726dffb640 1 -- 172.21.15.16:0/2912684877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7264004350 con 0x7f7270100af0 2024-03-27T06:40:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.548+0000 7f72771e2640 1 -- 172.21.15.16:0/2912684877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727019fe30 con 0x7f7270100af0 2024-03-27T06:40:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.549+0000 7f726dffb640 1 -- 172.21.15.16:0/2912684877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264003db0 con 0x7f7270100af0 2024-03-27T06:40:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.549+0000 7f72771e2640 1 -- 172.21.15.16:0/2912684877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72701a02f0 con 0x7f7270100af0 2024-03-27T06:40:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.550+0000 7f72771e2640 1 -- 172.21.15.16:0/2912684877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7270107e30 con 0x7f7270100af0 2024-03-27T06:40:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.551+0000 7f726dffb640 1 -- 172.21.15.16:0/2912684877 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f7264046020 con 0x7f7270100af0 2024-03-27T06:40:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.552+0000 7f726dffb640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f72600a3610 0x7f72600a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.552+0000 7f726ffff640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f72600a3610 0x7f72600a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.552+0000 7f726dffb640 1 -- 172.21.15.16:0/2912684877 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f72640f1fa0 con 0x7f7270100af0 2024-03-27T06:40:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.552+0000 7f726ffff640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f72600a3610 0x7f72600a5ad0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f725c0058c0 tx=0x7f725c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:51.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.556+0000 7f726dffb640 1 -- 172.21.15.16:0/2912684877 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f726404d020 con 0x7f7270100af0 2024-03-27T06:40:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.782+0000 7f72771e2640 1 -- 172.21.15.16:0/2912684877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f727002d370 con 0x7f7270100af0 2024-03-27T06:40:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.783+0000 7f726dffb640 1 -- 172.21.15.16:0/2912684877 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f72640bef70 con 0x7f7270100af0 2024-03-27T06:40:51.779 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:40:51.779 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-27T06:40:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 1 2024-03-27T06:40:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 -- 172.21.15.16:0/2912684877 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f72600a3610 msgr2=0x7f72600a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f72600a3610 0x7f72600a5ad0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f725c0058c0 tx=0x7f725c005850 comp rx=0 tx=0).stop 2024-03-27T06:40:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 -- 172.21.15.16:0/2912684877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270100af0 msgr2=0x7f727006ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270100af0 0x7f727006ad10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7264005ab0 tx=0x7f726402c7a0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 -- 172.21.15.16:0/2912684877 shutdown_connections 2024-03-27T06:40:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f72600a3610 0x7f72600a5ad0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72701026a0 0x7f727019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7270101ce0 0x7f727019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 --2- 172.21.15.16:0/2912684877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7270100af0 0x7f727006ad10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 -- 172.21.15.16:0/2912684877 >> 172.21.15.16:0/2912684877 conn(0x7f72700fc080 msgr2=0x7f7270105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 -- 172.21.15.16:0/2912684877 shutdown_connections 2024-03-27T06:40:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:51.785+0000 7f72771e2640 1 -- 172.21.15.16:0/2912684877 wait complete. 2024-03-27T06:40:51.790 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:51 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/4159794839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:40:51.790 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:51 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/611452387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:40:51.917 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:51 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/4159794839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:40:51.917 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:51 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/611452387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:40:51.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:51 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/4159794839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:40:51.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:51 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/611452387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:40:52.313 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:40:52 smithi170 systemd[1]: Starting Ceph mds.l for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:40:52.594 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:40:52 smithi170 podman[66919]: 2024-03-27 06:40:52.222178687 +0000 UTC m=+0.018175937 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:40:52.594 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:40:52 smithi170 podman[66919]: 2024-03-27 06:40:52.363687723 +0000 UTC m=+0.159684964 container create 03bcfff9364f673b293a30f98ec1571637e19d4b3047bc809b3d4eaa8eb7a047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, ceph=True) 2024-03-27T06:40:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:52 smithi016 ceph-mon[31908]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:52.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:52 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2912684877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:40:52.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:52 smithi156 ceph-mon[36261]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:52.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:52 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2912684877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:40:53.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:52 smithi170 ceph-mon[36438]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:53.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:52 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2912684877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:40:53.094 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:40:52 smithi170 podman[66919]: 2024-03-27 06:40:52.620421412 +0000 UTC m=+0.416418670 container init 03bcfff9364f673b293a30f98ec1571637e19d4b3047bc809b3d4eaa8eb7a047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-27T06:40:53.094 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:40:52 smithi170 podman[66919]: 2024-03-27 06:40:52.623115552 +0000 UTC m=+0.419112809 container start 03bcfff9364f673b293a30f98ec1571637e19d4b3047bc809b3d4eaa8eb7a047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:40:53.094 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:40:52 smithi170 ceph-mds[66938]: -- 172.21.15.170:0/875254082 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5629cbc3da40 con 0x5629cbc10c00 2024-03-27T06:40:53.094 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:40:52 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l[66934]: starting mds.l at 2024-03-27T06:40:53.094 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:40:52 smithi170 bash[66919]: 03bcfff9364f673b293a30f98ec1571637e19d4b3047bc809b3d4eaa8eb7a047 2024-03-27T06:40:53.094 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:40:52 smithi170 systemd[1]: Started Ceph mds.l for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:40:53.714 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:53 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6832/1368007304,v1:172.21.15.170:6833/1368007304] up:boot 2024-03-27T06:40:53.715 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:53 smithi156 ceph-mon[36261]: fsmap 1 up:standby 2024-03-27T06:40:53.715 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:53 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T06:40:53.715 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:53 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:53.715 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:53 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:53.715 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:53 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:53.715 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:53 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:40:53.715 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:53 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:40:53.715 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:53 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:53.715 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:53 smithi156 ceph-mon[36261]: Deploying daemon mds.k on smithi156 2024-03-27T06:40:53.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:40:53.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.912+0000 7f32fce9a640 1 -- 172.21.15.16:0/3699885867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f8101b60 msgr2=0x7f32f8101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:53.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.912+0000 7f32fce9a640 1 --2- 172.21.15.16:0/3699885867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f8101b60 0x7f32f8101f80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009900 tx=0x7f32e802bc30 comp rx=0 tx=0).stop 2024-03-27T06:40:53.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.912+0000 7f32fce9a640 1 -- 172.21.15.16:0/3699885867 shutdown_connections 2024-03-27T06:40:53.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.912+0000 7f32fce9a640 1 --2- 172.21.15.16:0/3699885867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32f8103710 0x7f32f8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.912+0000 7f32fce9a640 1 --2- 172.21.15.16:0/3699885867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f8102d50 0x7f32f81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.912+0000 7f32fce9a640 1 --2- 172.21.15.16:0/3699885867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f8101b60 0x7f32f8101f80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.912+0000 7f32fce9a640 1 -- 172.21.15.16:0/3699885867 >> 172.21.15.16:0/3699885867 conn(0x7f32f80fd130 msgr2=0x7f32f80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.913+0000 7f32fce9a640 1 -- 172.21.15.16:0/3699885867 shutdown_connections 2024-03-27T06:40:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.913+0000 7f32fce9a640 1 -- 172.21.15.16:0/3699885867 wait complete. 2024-03-27T06:40:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.913+0000 7f32fce9a640 1 Processor -- start 2024-03-27T06:40:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.913+0000 7f32fce9a640 1 -- start start 2024-03-27T06:40:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.913+0000 7f32fce9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f8101b60 0x7f32f8075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.913+0000 7f32fce9a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f8102d50 0x7f32f8073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.913+0000 7f32fce9a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32f8103710 0x7f32f8074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.913+0000 7f32fce9a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810d8c0 con 0x7f32f8101b60 2024-03-27T06:40:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32fce9a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810d740 con 0x7f32f8102d50 2024-03-27T06:40:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32fce9a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810da40 con 0x7f32f8103710 2024-03-27T06:40:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32f659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f8101b60 0x7f32f8075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32f659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f8101b60 0x7f32f8075900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46470/0 (socket says 172.21.15.16:46470) 2024-03-27T06:40:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32f659c640 1 -- 172.21.15.16:0/3429453155 learned_addr learned my addr 172.21.15.16:0/3429453155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:40:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32effff640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f8102d50 0x7f32f8073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32f6d9d640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32f8103710 0x7f32f8074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32f659c640 1 -- 172.21.15.16:0/3429453155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32f8103710 msgr2=0x7f32f8074480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32f659c640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32f8103710 0x7f32f8074480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32f659c640 1 -- 172.21.15.16:0/3429453155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f8102d50 msgr2=0x7f32f8073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32f659c640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f8102d50 0x7f32f8073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32f659c640 1 -- 172.21.15.16:0/3429453155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f8074d10 con 0x7f32f8101b60 2024-03-27T06:40:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32f659c640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f8101b60 0x7f32f8075900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000c00 tx=0x7f32e8004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32ef7fe640 1 -- 172.21.15.16:0/3429453155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8043020 con 0x7f32f8101b60 2024-03-27T06:40:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32ef7fe640 1 -- 172.21.15.16:0/3429453155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32e8004250 con 0x7f32f8101b60 2024-03-27T06:40:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32fce9a640 1 -- 172.21.15.16:0/3429453155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f81a2080 con 0x7f32f8101b60 2024-03-27T06:40:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32ef7fe640 1 -- 172.21.15.16:0/3429453155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8003b90 con 0x7f32f8101b60 2024-03-27T06:40:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.914+0000 7f32fce9a640 1 -- 172.21.15.16:0/3429453155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f81a2620 con 0x7f32f8101b60 2024-03-27T06:40:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.916+0000 7f32fce9a640 1 -- 172.21.15.16:0/3429453155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f8108e80 con 0x7f32f8101b60 2024-03-27T06:40:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.916+0000 7f32ef7fe640 1 -- 172.21.15.16:0/3429453155 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f32e8046020 con 0x7f32f8101b60 2024-03-27T06:40:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.917+0000 7f32ef7fe640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32e40a3690 0x7f32e40a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.917+0000 7f32effff640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32e40a3690 0x7f32e40a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.917+0000 7f32ef7fe640 1 -- 172.21.15.16:0/3429453155 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f32e80f1940 con 0x7f32f8101b60 2024-03-27T06:40:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.918+0000 7f32effff640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32e40a3690 0x7f32e40a5b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32e00095f0 tx=0x7f32e0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:53.922+0000 7f32ef7fe640 1 -- 172.21.15.16:0/3429453155 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f32e804d020 con 0x7f32f8101b60 2024-03-27T06:40:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:53 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6832/1368007304,v1:172.21.15.170:6833/1368007304] up:boot 2024-03-27T06:40:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:53 smithi170 ceph-mon[36438]: fsmap 1 up:standby 2024-03-27T06:40:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:53 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T06:40:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:53 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:53 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:53 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:53 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:40:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:53 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:40:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:53 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:53 smithi170 ceph-mon[36438]: Deploying daemon mds.k on smithi156 2024-03-27T06:40:54.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:53 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6832/1368007304,v1:172.21.15.170:6833/1368007304] up:boot 2024-03-27T06:40:54.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:53 smithi016 ceph-mon[31908]: fsmap 1 up:standby 2024-03-27T06:40:54.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:53 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T06:40:54.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:53 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:54.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:53 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:54.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:53 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:54.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:53 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:40:54.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:53 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:40:54.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:53 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:54.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:53 smithi016 ceph-mon[31908]: Deploying daemon mds.k on smithi156 2024-03-27T06:40:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.147+0000 7f32fce9a640 1 -- 172.21.15.16:0/3429453155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32f8062750 con 0x7f32f8101b60 2024-03-27T06:40:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.147+0000 7f32ef7fe640 1 -- 172.21.15.16:0/3429453155 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f32e80b8e20 con 0x7f32f8101b60 2024-03-27T06:40:54.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:40:54.142 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-27T06:40:54.142 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-03-27T06:40:54.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 -- 172.21.15.16:0/3429453155 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32e40a3690 msgr2=0x7f32e40a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32e40a3690 0x7f32e40a5b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32e00095f0 tx=0x7f32e0005f90 comp rx=0 tx=0).stop 2024-03-27T06:40:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 -- 172.21.15.16:0/3429453155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f8101b60 msgr2=0x7f32f8075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f8101b60 0x7f32f8075900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000c00 tx=0x7f32e8004a30 comp rx=0 tx=0).stop 2024-03-27T06:40:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 -- 172.21.15.16:0/3429453155 shutdown_connections 2024-03-27T06:40:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32e40a3690 0x7f32e40a5b50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32f8103710 0x7f32f8074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32f8102d50 0x7f32f8073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 --2- 172.21.15.16:0/3429453155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32f8101b60 0x7f32f8075900 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 -- 172.21.15.16:0/3429453155 >> 172.21.15.16:0/3429453155 conn(0x7f32f80fd130 msgr2=0x7f32f8106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 -- 172.21.15.16:0/3429453155 shutdown_connections 2024-03-27T06:40:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:54.149+0000 7f32fce9a640 1 -- 172.21.15.16:0/3429453155 wait complete. 2024-03-27T06:40:55.079 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:54 smithi156 ceph-mon[36261]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:55.080 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:54 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3429453155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:40:55.080 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:40:55 smithi156 systemd[1]: Starting Ceph mds.k for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:40:55.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:54 smithi170 ceph-mon[36438]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:55.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:54 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3429453155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:40:55.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:54 smithi016 ceph-mon[31908]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:55.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:54 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3429453155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:40:55.575 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:40:55 smithi156 podman[65819]: 2024-03-27 06:40:55.202128992 +0000 UTC m=+0.017761524 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:40:55.575 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:40:55 smithi156 podman[65819]: 2024-03-27 06:40:55.328428164 +0000 UTC m=+0.144060722 container create c3b70b9c62bfca3cc25d99b3ea3a2ac694d7ea5c6eb3407306c065a0a680bafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-03-27T06:40:55.575 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:40:55 smithi156 podman[65819]: 2024-03-27 06:40:55.581490789 +0000 UTC m=+0.397123308 container init c3b70b9c62bfca3cc25d99b3ea3a2ac694d7ea5c6eb3407306c065a0a680bafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:40:55.972 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:40:55 smithi156 podman[65819]: 2024-03-27 06:40:55.584337873 +0000 UTC m=+0.399970393 container start c3b70b9c62bfca3cc25d99b3ea3a2ac694d7ea5c6eb3407306c065a0a680bafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-27T06:40:55.972 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:40:55 smithi156 ceph-mds[65838]: -- 172.21.15.156:0/155458864 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56519f821a40 con 0x56519f7f5000 2024-03-27T06:40:55.972 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:40:55 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k[65834]: starting mds.k at 2024-03-27T06:40:55.972 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:40:55 smithi156 bash[65819]: c3b70b9c62bfca3cc25d99b3ea3a2ac694d7ea5c6eb3407306c065a0a680bafe 2024-03-27T06:40:55.972 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:40:55 smithi156 systemd[1]: Started Ceph mds.k for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:40:56.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:40:56.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.268+0000 7f9079a71640 1 -- 172.21.15.16:0/3959228593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074102690 msgr2=0x7f9074104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:56.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.268+0000 7f9079a71640 1 --2- 172.21.15.16:0/3959228593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074102690 0x7f9074104a80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9068009900 tx=0x7f906802bc30 comp rx=0 tx=0).stop 2024-03-27T06:40:56.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.268+0000 7f9079a71640 1 -- 172.21.15.16:0/3959228593 shutdown_connections 2024-03-27T06:40:56.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.268+0000 7f9079a71640 1 --2- 172.21.15.16:0/3959228593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90741078f0 0x7f9074109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.268+0000 7f9079a71640 1 --2- 172.21.15.16:0/3959228593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9074104fc0 0x7f90741073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.268+0000 7f9079a71640 1 --2- 172.21.15.16:0/3959228593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074102690 0x7f9074104a80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.268+0000 7f9079a71640 1 -- 172.21.15.16:0/3959228593 >> 172.21.15.16:0/3959228593 conn(0x7f90740fbfe0 msgr2=0x7f90740fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.268+0000 7f9079a71640 1 -- 172.21.15.16:0/3959228593 shutdown_connections 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.268+0000 7f9079a71640 1 -- 172.21.15.16:0/3959228593 wait complete. 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.269+0000 7f9079a71640 1 Processor -- start 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.269+0000 7f9079a71640 1 -- start start 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.269+0000 7f9079a71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074102690 0x7f9074197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.269+0000 7f9079a71640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9074104fc0 0x7f90741984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.269+0000 7f9079a71640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90741078f0 0x7f907419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.269+0000 7f9079a71640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f907410d8f0 con 0x7f9074102690 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.269+0000 7f9079a71640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f907410d770 con 0x7f90741078f0 2024-03-27T06:40:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.269+0000 7f9079a71640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f907410da70 con 0x7f9074104fc0 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.269+0000 7f90737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074102690 0x7f9074197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.269+0000 7f90737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074102690 0x7f9074197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53102/0 (socket says 172.21.15.16:53102) 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.269+0000 7f90737fe640 1 -- 172.21.15.16:0/694653505 learned_addr learned my addr 172.21.15.16:0/694653505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f9073fff640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90741078f0 0x7f907419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f9072ffd640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9074104fc0 0x7f90741984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f90737fe640 1 -- 172.21.15.16:0/694653505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9074104fc0 msgr2=0x7f90741984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f90737fe640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9074104fc0 0x7f90741984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f90737fe640 1 -- 172.21.15.16:0/694653505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90741078f0 msgr2=0x7f907419f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f90737fe640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90741078f0 0x7f907419f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f90737fe640 1 -- 172.21.15.16:0/694653505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907419fbc0 con 0x7f9074102690 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f90737fe640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074102690 0x7f9074197fa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9068009230 tx=0x7f906802c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f9070ff9640 1 -- 172.21.15.16:0/694653505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068043020 con 0x7f9074102690 2024-03-27T06:40:56.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f9070ff9640 1 -- 172.21.15.16:0/694653505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f90680043d0 con 0x7f9074102690 2024-03-27T06:40:56.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f9070ff9640 1 -- 172.21.15.16:0/694653505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068003c30 con 0x7f9074102690 2024-03-27T06:40:56.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f9079a71640 1 -- 172.21.15.16:0/694653505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907419fe50 con 0x7f9074102690 2024-03-27T06:40:56.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.270+0000 7f9079a71640 1 -- 172.21.15.16:0/694653505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90741a02b0 con 0x7f9074102690 2024-03-27T06:40:56.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.271+0000 7f9079a71640 1 -- 172.21.15.16:0/694653505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9074068940 con 0x7f9074102690 2024-03-27T06:40:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.273+0000 7f9070ff9640 1 -- 172.21.15.16:0/694653505 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f9068046020 con 0x7f9074102690 2024-03-27T06:40:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.274+0000 7f9070ff9640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f905c0a3610 0x7f905c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.274+0000 7f9072ffd640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f905c0a3610 0x7f905c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.274+0000 7f9070ff9640 1 -- 172.21.15.16:0/694653505 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f90680f1fb0 con 0x7f9074102690 2024-03-27T06:40:56.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.274+0000 7f9072ffd640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f905c0a3610 0x7f905c0a5ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9064005d00 tx=0x7f9064005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:56.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.278+0000 7f9070ff9640 1 -- 172.21.15.16:0/694653505 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f906804d020 con 0x7f9074102690 2024-03-27T06:40:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.517+0000 7f9079a71640 1 -- 172.21.15.16:0/694653505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90740fdad0 con 0x7f9074102690 2024-03-27T06:40:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.517+0000 7f9070ff9640 1 -- 172.21.15.16:0/694653505 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f90680bf020 con 0x7f9074102690 2024-03-27T06:40:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:40:56.512 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-27T06:40:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-03-27T06:40:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.519+0000 7f9079a71640 1 -- 172.21.15.16:0/694653505 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f905c0a3610 msgr2=0x7f905c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.519+0000 7f9079a71640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f905c0a3610 0x7f905c0a5ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9064005d00 tx=0x7f9064005c50 comp rx=0 tx=0).stop 2024-03-27T06:40:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.519+0000 7f9079a71640 1 -- 172.21.15.16:0/694653505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074102690 msgr2=0x7f9074197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.519+0000 7f9079a71640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074102690 0x7f9074197fa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9068009230 tx=0x7f906802c820 comp rx=0 tx=0).stop 2024-03-27T06:40:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.519+0000 7f9079a71640 1 -- 172.21.15.16:0/694653505 shutdown_connections 2024-03-27T06:40:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.519+0000 7f9079a71640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f905c0a3610 0x7f905c0a5ad0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.519+0000 7f9079a71640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90741078f0 0x7f907419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.519+0000 7f9079a71640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9074104fc0 0x7f90741984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.519+0000 7f9079a71640 1 --2- 172.21.15.16:0/694653505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9074102690 0x7f9074197fa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.519+0000 7f9079a71640 1 -- 172.21.15.16:0/694653505 >> 172.21.15.16:0/694653505 conn(0x7f90740fbfe0 msgr2=0x7f9074102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.520+0000 7f9079a71640 1 -- 172.21.15.16:0/694653505 shutdown_connections 2024-03-27T06:40:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:56.520+0000 7f9079a71640 1 -- 172.21.15.16:0/694653505 wait complete. 2024-03-27T06:40:57.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:57.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:57.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:57.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:57.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:40:57.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:40:57.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:57.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: Deploying daemon mds.j on smithi016 2024-03-27T06:40:57.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6832/521338694,v1:172.21.15.156:6833/521338694] up:boot 2024-03-27T06:40:57.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: fsmap 2 up:standby 2024-03-27T06:40:57.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T06:40:57.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:56 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/694653505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:40:57.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:57.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:57.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:57.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:57.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:40:57.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:40:57.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:57.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: Deploying daemon mds.j on smithi016 2024-03-27T06:40:57.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6832/521338694,v1:172.21.15.156:6833/521338694] up:boot 2024-03-27T06:40:57.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: fsmap 2 up:standby 2024-03-27T06:40:57.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T06:40:57.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:56 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/694653505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:40:57.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:57.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:57.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:57.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:40:57.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:40:57.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:40:57.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:40:57.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: Deploying daemon mds.j on smithi016 2024-03-27T06:40:57.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6832/521338694,v1:172.21.15.156:6833/521338694] up:boot 2024-03-27T06:40:57.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: fsmap 2 up:standby 2024-03-27T06:40:57.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T06:40:57.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:56 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/694653505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:40:58.088 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:40:58 smithi016 systemd[1]: Starting Ceph mds.j for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:40:58.361 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:40:58 smithi016 podman[72670]: 2024-03-27 06:40:58.211430238 +0000 UTC m=+0.017916447 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:40:58.361 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:40:58 smithi016 podman[72670]: 2024-03-27 06:40:58.311736919 +0000 UTC m=+0.118223126 container create 79533e0080189b483e8347d8e008934ad919373d66900afcd287076a51ee0d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-j, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-27T06:40:58.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:40:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.648+0000 7f279ffac640 1 -- 172.21.15.16:0/655075981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2798100a70 msgr2=0x7f2798100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.648+0000 7f279ffac640 1 --2- 172.21.15.16:0/655075981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2798100a70 0x7f2798100e90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2788009900 tx=0x7f278802bc30 comp rx=0 tx=0).stop 2024-03-27T06:40:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 -- 172.21.15.16:0/655075981 shutdown_connections 2024-03-27T06:40:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 --2- 172.21.15.16:0/655075981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2798102620 0x7f2798108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 --2- 172.21.15.16:0/655075981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2798101c60 0x7f27981020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 --2- 172.21.15.16:0/655075981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2798100a70 0x7f2798100e90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 -- 172.21.15.16:0/655075981 >> 172.21.15.16:0/655075981 conn(0x7f27980fbfe0 msgr2=0x7f27980fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 -- 172.21.15.16:0/655075981 shutdown_connections 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 -- 172.21.15.16:0/655075981 wait complete. 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 Processor -- start 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 -- start start 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2798100a70 0x7f2798104990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2798101c60 0x7f2798103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2798102620 0x7f2798103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f279810c830 con 0x7f2798100a70 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f279810c6b0 con 0x7f2798101c60 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279ffac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f279810c9b0 con 0x7f2798102620 2024-03-27T06:40:58.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.649+0000 7f279dd48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2798100a70 0x7f2798104990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f279dd48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2798100a70 0x7f2798104990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53136/0 (socket says 172.21.15.16:53136) 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f279dd48640 1 -- 172.21.15.16:0/326633410 learned_addr learned my addr 172.21.15.16:0/326633410 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f279d547640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2798101c60 0x7f2798103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f279dd48640 1 -- 172.21.15.16:0/326633410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2798102620 msgr2=0x7f2798103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f279dd48640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2798102620 0x7f2798103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f279dd48640 1 -- 172.21.15.16:0/326633410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2798101c60 msgr2=0x7f2798103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f279dd48640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2798101c60 0x7f2798103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f279dd48640 1 -- 172.21.15.16:0/326633410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2798103da0 con 0x7f2798100a70 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f279dd48640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2798100a70 0x7f2798104990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2788000c00 tx=0x7f278802c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f278effd640 1 -- 172.21.15.16:0/326633410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788043020 con 0x7f2798100a70 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f279ffac640 1 -- 172.21.15.16:0/326633410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27981a20e0 con 0x7f2798100a70 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f278effd640 1 -- 172.21.15.16:0/326633410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f27880042d0 con 0x7f2798100a70 2024-03-27T06:40:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f278effd640 1 -- 172.21.15.16:0/326633410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788003c90 con 0x7f2798100a70 2024-03-27T06:40:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.650+0000 7f279ffac640 1 -- 172.21.15.16:0/326633410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27981a2700 con 0x7f2798100a70 2024-03-27T06:40:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.651+0000 7f279ffac640 1 -- 172.21.15.16:0/326633410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2798068940 con 0x7f2798100a70 2024-03-27T06:40:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.652+0000 7f278effd640 1 -- 172.21.15.16:0/326633410 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f2788046020 con 0x7f2798100a70 2024-03-27T06:40:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.653+0000 7f278effd640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f27840a3610 0x7f27840a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:40:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.653+0000 7f279d547640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f27840a3610 0x7f27840a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:40:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.653+0000 7f278effd640 1 -- 172.21.15.16:0/326633410 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f27880f19d0 con 0x7f2798100a70 2024-03-27T06:40:58.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.653+0000 7f279d547640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f27840a3610 0x7f27840a5ad0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2798104700 tx=0x7f2794005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:40:58.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.657+0000 7f278effd640 1 -- 172.21.15.16:0/326633410 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f278804d020 con 0x7f2798100a70 2024-03-27T06:40:58.727 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:40:58 smithi016 podman[72670]: 2024-03-27 06:40:58.581435011 +0000 UTC m=+0.387921218 container init 79533e0080189b483e8347d8e008934ad919373d66900afcd287076a51ee0d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-j, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-27T06:40:58.727 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:40:58 smithi016 podman[72670]: 2024-03-27 06:40:58.584351931 +0000 UTC m=+0.390838145 container start 79533e0080189b483e8347d8e008934ad919373d66900afcd287076a51ee0d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-j, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-27T06:40:58.729 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:40:58 smithi016 ceph-mds[72709]: -- 172.21.15.16:0/3972360069 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586a4aaba40 con 0x5586a4a7e800 2024-03-27T06:40:58.730 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:40:58 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-j[72685]: starting mds.j at 2024-03-27T06:40:58.730 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:40:58 smithi016 bash[72670]: 79533e0080189b483e8347d8e008934ad919373d66900afcd287076a51ee0d3f 2024-03-27T06:40:58.730 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:40:58 smithi016 systemd[1]: Started Ceph mds.j for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:40:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.910+0000 7f279ffac640 1 -- 172.21.15.16:0/326633410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27981a2a90 con 0x7f2798100a70 2024-03-27T06:40:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.910+0000 7f278effd640 1 -- 172.21.15.16:0/326633410 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f27880bead0 con 0x7f2798100a70 2024-03-27T06:40:58.905 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:40:58.905 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-27T06:40:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 4 2024-03-27T06:40:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.912+0000 7f279ffac640 1 -- 172.21.15.16:0/326633410 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f27840a3610 msgr2=0x7f27840a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.912+0000 7f279ffac640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f27840a3610 0x7f27840a5ad0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2798104700 tx=0x7f2794005f90 comp rx=0 tx=0).stop 2024-03-27T06:40:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.912+0000 7f279ffac640 1 -- 172.21.15.16:0/326633410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2798100a70 msgr2=0x7f2798104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:40:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.912+0000 7f279ffac640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2798100a70 0x7f2798104990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2788000c00 tx=0x7f278802c710 comp rx=0 tx=0).stop 2024-03-27T06:40:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.912+0000 7f279ffac640 1 -- 172.21.15.16:0/326633410 shutdown_connections 2024-03-27T06:40:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.912+0000 7f279ffac640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f27840a3610 0x7f27840a5ad0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.912+0000 7f279ffac640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2798102620 0x7f2798103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.912+0000 7f279ffac640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2798101c60 0x7f2798103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.912+0000 7f279ffac640 1 --2- 172.21.15.16:0/326633410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2798100a70 0x7f2798104990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:40:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.912+0000 7f279ffac640 1 -- 172.21.15.16:0/326633410 >> 172.21.15.16:0/326633410 conn(0x7f27980fbfe0 msgr2=0x7f27980fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:40:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.913+0000 7f279ffac640 1 -- 172.21.15.16:0/326633410 shutdown_connections 2024-03-27T06:40:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:40:58.913+0000 7f279ffac640 1 -- 172.21.15.16:0/326633410 wait complete. 2024-03-27T06:40:59.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:58 smithi170 ceph-mon[36438]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:59.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:58 smithi016 ceph-mon[31908]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:40:59.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:58 smithi156 ceph-mon[36261]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:00.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:59 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.16:6834/1554082728,v1:172.21.15.16:6835/1554082728] up:boot 2024-03-27T06:41:00.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:59 smithi170 ceph-mon[36438]: fsmap 3 up:standby 2024-03-27T06:41:00.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-27T06:41:00.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:00.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:00.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:00.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:00.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:00.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:59 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:00.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:59 smithi170 ceph-mon[36438]: Deploying daemon mds.d on smithi016 2024-03-27T06:41:00.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:40:59 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/326633410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:00.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:59 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.16:6834/1554082728,v1:172.21.15.16:6835/1554082728] up:boot 2024-03-27T06:41:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:59 smithi016 ceph-mon[31908]: fsmap 3 up:standby 2024-03-27T06:41:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-27T06:41:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:59 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:59 smithi016 ceph-mon[31908]: Deploying daemon mds.d on smithi016 2024-03-27T06:41:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:40:59 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/326633410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:00.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:59 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.16:6834/1554082728,v1:172.21.15.16:6835/1554082728] up:boot 2024-03-27T06:41:00.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:59 smithi156 ceph-mon[36261]: fsmap 3 up:standby 2024-03-27T06:41:00.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-27T06:41:00.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:00.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:00.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:00.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:00.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:00.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:59 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:00.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:59 smithi156 ceph-mon[36261]: Deploying daemon mds.d on smithi016 2024-03-27T06:41:00.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:40:59 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/326633410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:00.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:01.063 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:00 smithi016 ceph-mon[31908]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:01.092 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:00 smithi170 ceph-mon[36438]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:01.123 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:00 smithi156 ceph-mon[36261]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.130+0000 7f34f5313640 1 -- 172.21.15.16:0/4293960005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f01078f0 msgr2=0x7f34f0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.130+0000 7f34f5313640 1 --2- 172.21.15.16:0/4293960005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f01078f0 0x7f34f0109ce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009ca0 tx=0x7f34e802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:41:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.130+0000 7f34f5313640 1 -- 172.21.15.16:0/4293960005 shutdown_connections 2024-03-27T06:41:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.130+0000 7f34f5313640 1 --2- 172.21.15.16:0/4293960005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f01078f0 0x7f34f0109ce0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.130+0000 7f34f5313640 1 --2- 172.21.15.16:0/4293960005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f0104fc0 0x7f34f01073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:01.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.130+0000 7f34f5313640 1 --2- 172.21.15.16:0/4293960005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f0102690 0x7f34f0104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:01.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.130+0000 7f34f5313640 1 -- 172.21.15.16:0/4293960005 >> 172.21.15.16:0/4293960005 conn(0x7f34f00fbfe0 msgr2=0x7f34f00fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:01.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.131+0000 7f34f5313640 1 -- 172.21.15.16:0/4293960005 shutdown_connections 2024-03-27T06:41:01.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.131+0000 7f34f5313640 1 -- 172.21.15.16:0/4293960005 wait complete. 2024-03-27T06:41:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.131+0000 7f34f5313640 1 Processor -- start 2024-03-27T06:41:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.131+0000 7f34f5313640 1 -- start start 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.131+0000 7f34f5313640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f0102690 0x7f34f0197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.131+0000 7f34f5313640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f0104fc0 0x7f34f0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.131+0000 7f34f5313640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f01078f0 0x7f34f019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.131+0000 7f34f5313640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34f010d940 con 0x7f34f0102690 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.131+0000 7f34f5313640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f34f010d7c0 con 0x7f34f01078f0 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.131+0000 7f34f5313640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f34f010dac0 con 0x7f34f0104fc0 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f0102690 0x7f34f0197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f0102690 0x7f34f0197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53142/0 (socket says 172.21.15.16:53142) 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34eeffd640 1 -- 172.21.15.16:0/55338326 learned_addr learned my addr 172.21.15.16:0/55338326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34ef7fe640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f01078f0 0x7f34f019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34ee7fc640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f0104fc0 0x7f34f0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34eeffd640 1 -- 172.21.15.16:0/55338326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f0104fc0 msgr2=0x7f34f0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:01.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34eeffd640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f0104fc0 0x7f34f0198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34eeffd640 1 -- 172.21.15.16:0/55338326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f01078f0 msgr2=0x7f34f019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34eeffd640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f01078f0 0x7f34f019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34eeffd640 1 -- 172.21.15.16:0/55338326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f019fb40 con 0x7f34f0102690 2024-03-27T06:41:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34eeffd640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f0102690 0x7f34f0197f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f34e000d950 tx=0x7f34e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34d7fff640 1 -- 172.21.15.16:0/55338326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0017070 con 0x7f34f0102690 2024-03-27T06:41:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34f5313640 1 -- 172.21.15.16:0/55338326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f019fe30 con 0x7f34f0102690 2024-03-27T06:41:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34d7fff640 1 -- 172.21.15.16:0/55338326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f34e0009c30 con 0x7f34f0102690 2024-03-27T06:41:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.132+0000 7f34d7fff640 1 -- 172.21.15.16:0/55338326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e000f740 con 0x7f34f0102690 2024-03-27T06:41:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.133+0000 7f34f5313640 1 -- 172.21.15.16:0/55338326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f01a02f0 con 0x7f34f0102690 2024-03-27T06:41:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.133+0000 7f34f5313640 1 -- 172.21.15.16:0/55338326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f00fdad0 con 0x7f34f0102690 2024-03-27T06:41:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.135+0000 7f34d7fff640 1 -- 172.21.15.16:0/55338326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f34e0010a80 con 0x7f34f0102690 2024-03-27T06:41:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.135+0000 7f34d7fff640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f34d80a3610 0x7f34d80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.135+0000 7f34d7fff640 1 -- 172.21.15.16:0/55338326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f34e00cf3f0 con 0x7f34f0102690 2024-03-27T06:41:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.135+0000 7f34ee7fc640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f34d80a3610 0x7f34d80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.136+0000 7f34ee7fc640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f34d80a3610 0x7f34d80a5ad0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f34f0199430 tx=0x7f34e400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.143+0000 7f34d7fff640 1 -- 172.21.15.16:0/55338326 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f34e0009de0 con 0x7f34f0102690 2024-03-27T06:41:01.319 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:41:01 smithi016 podman[73064]: 2024-03-27 06:41:01.325589521 +0000 UTC m=+0.146892224 container create 0b99dee229916d3d4bc6dcb97114cc650b924318032339a8e2ac18c567cd8704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5) 2024-03-27T06:41:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.375+0000 7f34f5313640 1 -- 172.21.15.16:0/55338326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34f002d570 con 0x7f34f0102690 2024-03-27T06:41:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.376+0000 7f34d7fff640 1 -- 172.21.15.16:0/55338326 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f34e009be10 con 0x7f34f0102690 2024-03-27T06:41:01.370 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:01.371 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-27T06:41:01.371 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 4 2024-03-27T06:41:01.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 -- 172.21.15.16:0/55338326 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f34d80a3610 msgr2=0x7f34d80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:01.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f34d80a3610 0x7f34d80a5ad0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f34f0199430 tx=0x7f34e400b390 comp rx=0 tx=0).stop 2024-03-27T06:41:01.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 -- 172.21.15.16:0/55338326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f0102690 msgr2=0x7f34f0197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:01.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f0102690 0x7f34f0197f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f34e000d950 tx=0x7f34e000de10 comp rx=0 tx=0).stop 2024-03-27T06:41:01.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 -- 172.21.15.16:0/55338326 shutdown_connections 2024-03-27T06:41:01.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f34d80a3610 0x7f34d80a5ad0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:01.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34f01078f0 0x7f34f019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:01.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f0104fc0 0x7f34f0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:01.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 --2- 172.21.15.16:0/55338326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f0102690 0x7f34f0197f20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:01.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 -- 172.21.15.16:0/55338326 >> 172.21.15.16:0/55338326 conn(0x7f34f00fbfe0 msgr2=0x7f34f0102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:01.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 -- 172.21.15.16:0/55338326 shutdown_connections 2024-03-27T06:41:01.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:01.378+0000 7f34f5313640 1 -- 172.21.15.16:0/55338326 wait complete. 2024-03-27T06:41:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:01 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/55338326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:01.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:01.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:01.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:01.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:01 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:01.862 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:41:01 smithi016 podman[73064]: 2024-03-27 06:41:01.578191869 +0000 UTC m=+0.399494574 container init 0b99dee229916d3d4bc6dcb97114cc650b924318032339a8e2ac18c567cd8704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-27T06:41:01.862 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:41:01 smithi016 podman[73064]: 2024-03-27 06:41:01.580837544 +0000 UTC m=+0.402140249 container start 0b99dee229916d3d4bc6dcb97114cc650b924318032339a8e2ac18c567cd8704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:41:01.862 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:41:01 smithi016 ceph-mds[73085]: -- 172.21.15.16:0/1521392669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a2e85c7a40 con 0x55a2e859ac00 2024-03-27T06:41:01.862 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:41:01 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-d[73081]: starting mds.d at 2024-03-27T06:41:01.862 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:41:01 smithi016 bash[73064]: 0b99dee229916d3d4bc6dcb97114cc650b924318032339a8e2ac18c567cd8704 2024-03-27T06:41:01.863 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:41:01 smithi016 systemd[1]: Started Ceph mds.d for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:41:02.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:01 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/55338326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:02.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:02.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:02.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:02.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:02.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:02.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:01 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:02.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:01 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/55338326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:02.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:02.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:02.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:02.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:02.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:02.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:01 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:02.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:02 smithi156 ceph-mon[36261]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:02.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:02 smithi156 ceph-mon[36261]: Deploying daemon mds.e on smithi156 2024-03-27T06:41:02.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:02 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.16:6836/1241440055,v1:172.21.15.16:6837/1241440055] up:boot 2024-03-27T06:41:02.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:02 smithi156 ceph-mon[36261]: fsmap 4 up:standby 2024-03-27T06:41:02.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:02 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-27T06:41:03.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:02 smithi170 ceph-mon[36438]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:03.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:02 smithi170 ceph-mon[36438]: Deploying daemon mds.e on smithi156 2024-03-27T06:41:03.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:02 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.16:6836/1241440055,v1:172.21.15.16:6837/1241440055] up:boot 2024-03-27T06:41:03.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:02 smithi170 ceph-mon[36438]: fsmap 4 up:standby 2024-03-27T06:41:03.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:02 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-27T06:41:03.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:02 smithi016 ceph-mon[31908]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:03.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:02 smithi016 ceph-mon[31908]: Deploying daemon mds.e on smithi156 2024-03-27T06:41:03.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:02 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.16:6836/1241440055,v1:172.21.15.16:6837/1241440055] up:boot 2024-03-27T06:41:03.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:02 smithi016 ceph-mon[31908]: fsmap 4 up:standby 2024-03-27T06:41:03.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:02 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-27T06:41:03.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.503+0000 7f472f2a3640 1 -- 172.21.15.16:0/723088624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4728100a90 msgr2=0x7f4728100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.503+0000 7f472f2a3640 1 --2- 172.21.15.16:0/723088624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4728100a90 0x7f4728100eb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4724009900 tx=0x7f472402bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.503+0000 7f472f2a3640 1 -- 172.21.15.16:0/723088624 shutdown_connections 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.503+0000 7f472f2a3640 1 --2- 172.21.15.16:0/723088624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4728102640 0x7f4728108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.503+0000 7f472f2a3640 1 --2- 172.21.15.16:0/723088624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4728101c80 0x7f4728102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.503+0000 7f472f2a3640 1 --2- 172.21.15.16:0/723088624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4728100a90 0x7f4728100eb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.503+0000 7f472f2a3640 1 -- 172.21.15.16:0/723088624 >> 172.21.15.16:0/723088624 conn(0x7f47280fc000 msgr2=0x7f47280fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.503+0000 7f472f2a3640 1 -- 172.21.15.16:0/723088624 shutdown_connections 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.503+0000 7f472f2a3640 1 -- 172.21.15.16:0/723088624 wait complete. 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472f2a3640 1 Processor -- start 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472f2a3640 1 -- start start 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472f2a3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4728100a90 0x7f472806a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472f2a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4728101c80 0x7f472819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472f2a3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4728102640 0x7f472819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:03.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472f2a3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f472810c7f0 con 0x7f4728101c80 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472f2a3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f472810c670 con 0x7f4728102640 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472f2a3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f472810c970 con 0x7f4728100a90 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472c83e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4728101c80 0x7f472819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472c83e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4728101c80 0x7f472819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53178/0 (socket says 172.21.15.16:53178) 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472c83e640 1 -- 172.21.15.16:0/2532852395 learned_addr learned my addr 172.21.15.16:0/2532852395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472d03f640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4728100a90 0x7f472806a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472d840640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4728102640 0x7f472819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472c83e640 1 -- 172.21.15.16:0/2532852395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4728100a90 msgr2=0x7f472806a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472c83e640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4728100a90 0x7f472806a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472c83e640 1 -- 172.21.15.16:0/2532852395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4728102640 msgr2=0x7f472819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472c83e640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4728102640 0x7f472819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472c83e640 1 -- 172.21.15.16:0/2532852395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472819fbc0 con 0x7f4728101c80 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.504+0000 7f472c83e640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4728101c80 0x7f472819cd30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f471800d770 tx=0x7f471800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.505+0000 7f471e7fc640 1 -- 172.21.15.16:0/2532852395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718017070 con 0x7f4728101c80 2024-03-27T06:41:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.505+0000 7f471e7fc640 1 -- 172.21.15.16:0/2532852395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4718005520 con 0x7f4728101c80 2024-03-27T06:41:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.505+0000 7f472f2a3640 1 -- 172.21.15.16:0/2532852395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472819feb0 con 0x7f4728101c80 2024-03-27T06:41:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.505+0000 7f471e7fc640 1 -- 172.21.15.16:0/2532852395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718010460 con 0x7f4728101c80 2024-03-27T06:41:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.505+0000 7f472f2a3640 1 -- 172.21.15.16:0/2532852395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47281a0320 con 0x7f4728101c80 2024-03-27T06:41:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.506+0000 7f472f2a3640 1 -- 172.21.15.16:0/2532852395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4728107ed0 con 0x7f4728101c80 2024-03-27T06:41:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.507+0000 7f471e7fc640 1 -- 172.21.15.16:0/2532852395 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f4718004ea0 con 0x7f4728101c80 2024-03-27T06:41:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.507+0000 7f471e7fc640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f47140a3610 0x7f47140a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.508+0000 7f472d03f640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f47140a3610 0x7f47140a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.508+0000 7f471e7fc640 1 -- 172.21.15.16:0/2532852395 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f47180cf290 con 0x7f4728101c80 2024-03-27T06:41:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.508+0000 7f472d03f640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f47140a3610 0x7f47140a5ad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4724005ab0 tx=0x7f4724005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:03.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.512+0000 7f471e7fc640 1 -- 172.21.15.16:0/2532852395 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f471809bcb0 con 0x7f4728101c80 2024-03-27T06:41:03.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.735+0000 7f472f2a3640 1 -- 172.21.15.16:0/2532852395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f472802d5c0 con 0x7f4728101c80 2024-03-27T06:41:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.736+0000 7f471e7fc640 1 -- 172.21.15.16:0/2532852395 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f471809ffb0 con 0x7f4728101c80 2024-03-27T06:41:03.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:03.730 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-27T06:41:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-03-27T06:41:03.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.737+0000 7f472f2a3640 1 -- 172.21.15.16:0/2532852395 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f47140a3610 msgr2=0x7f47140a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:03.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.737+0000 7f472f2a3640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f47140a3610 0x7f47140a5ad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4724005ab0 tx=0x7f4724005fd0 comp rx=0 tx=0).stop 2024-03-27T06:41:03.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.737+0000 7f472f2a3640 1 -- 172.21.15.16:0/2532852395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4728101c80 msgr2=0x7f472819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:03.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.737+0000 7f472f2a3640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4728101c80 0x7f472819cd30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f471800d770 tx=0x7f471800dc30 comp rx=0 tx=0).stop 2024-03-27T06:41:03.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.738+0000 7f472f2a3640 1 -- 172.21.15.16:0/2532852395 shutdown_connections 2024-03-27T06:41:03.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.738+0000 7f472f2a3640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f47140a3610 0x7f47140a5ad0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.738+0000 7f472f2a3640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4728102640 0x7f472819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.738+0000 7f472f2a3640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4728101c80 0x7f472819cd30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.738+0000 7f472f2a3640 1 --2- 172.21.15.16:0/2532852395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4728100a90 0x7f472806a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.738+0000 7f472f2a3640 1 -- 172.21.15.16:0/2532852395 >> 172.21.15.16:0/2532852395 conn(0x7f47280fc000 msgr2=0x7f47281050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.738+0000 7f472f2a3640 1 -- 172.21.15.16:0/2532852395 shutdown_connections 2024-03-27T06:41:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:03.738+0000 7f472f2a3640 1 -- 172.21.15.16:0/2532852395 wait complete. 2024-03-27T06:41:04.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:03 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2532852395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:04.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:03 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2532852395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:04.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:03 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2532852395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:04.223 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:41:04 smithi156 systemd[1]: Starting Ceph mds.e for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:41:04.539 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:41:04 smithi156 podman[66142]: 2024-03-27 06:41:04.170436514 +0000 UTC m=+0.017761745 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:41:04.539 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:41:04 smithi156 podman[66142]: 2024-03-27 06:41:04.284289814 +0000 UTC m=+0.131615040 container create 0979e0d68b54e5831aa9b6421a58e83bd0f8483d00eb017bf1847273b32765ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-27T06:41:04.540 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:41:04 smithi156 podman[66142]: 2024-03-27 06:41:04.545588182 +0000 UTC m=+0.392913408 container init 0979e0d68b54e5831aa9b6421a58e83bd0f8483d00eb017bf1847273b32765ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:41:04.816 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:41:04 smithi156 podman[66142]: 2024-03-27 06:41:04.548445745 +0000 UTC m=+0.395770969 container start 0979e0d68b54e5831aa9b6421a58e83bd0f8483d00eb017bf1847273b32765ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-27T06:41:04.816 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:41:04 smithi156 ceph-mds[66161]: -- 172.21.15.156:0/424390593 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e3af11a40 con 0x561e3aee5000 2024-03-27T06:41:04.816 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:41:04 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e[66157]: starting mds.e at 2024-03-27T06:41:04.816 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:41:04 smithi156 bash[66142]: 0979e0d68b54e5831aa9b6421a58e83bd0f8483d00eb017bf1847273b32765ca 2024-03-27T06:41:04.817 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:41:04 smithi156 systemd[1]: Started Ceph mds.e for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:41:05.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:04 smithi170 ceph-mon[36438]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:05.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:04 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:05.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:04 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:05.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:04 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:05.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:04 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:05.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:04 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:05.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:04 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:05.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:04 smithi016 ceph-mon[31908]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:05.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:04 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:05.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:04 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:05.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:04 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:05.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:04 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:05.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:04 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:05.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:04 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:05.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:04 smithi156 ceph-mon[36261]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:05.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:04 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:05.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:04 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:05.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:04 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:05.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:04 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:05.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:04 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:05.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:04 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:05.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:05.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:05 smithi170 ceph-mon[36438]: Deploying daemon mds.f on smithi170 2024-03-27T06:41:05.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:05 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] up:boot 2024-03-27T06:41:05.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:05 smithi170 ceph-mon[36438]: fsmap 5 up:standby 2024-03-27T06:41:05.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:05 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T06:41:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.865+0000 7f2ad5288640 1 -- 172.21.15.16:0/3574941201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad0102620 msgr2=0x7f2ad0108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.865+0000 7f2ad5288640 1 --2- 172.21.15.16:0/3574941201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad0102620 0x7f2ad0108eb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009ca0 tx=0x7f2ac002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:41:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.865+0000 7f2ad5288640 1 -- 172.21.15.16:0/3574941201 shutdown_connections 2024-03-27T06:41:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.865+0000 7f2ad5288640 1 --2- 172.21.15.16:0/3574941201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad0102620 0x7f2ad0108eb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.865+0000 7f2ad5288640 1 --2- 172.21.15.16:0/3574941201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0101c60 0x7f2ad01020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.865+0000 7f2ad5288640 1 --2- 172.21.15.16:0/3574941201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad0100a70 0x7f2ad0100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.865+0000 7f2ad5288640 1 -- 172.21.15.16:0/3574941201 >> 172.21.15.16:0/3574941201 conn(0x7f2ad00fbfe0 msgr2=0x7f2ad00fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.865+0000 7f2ad5288640 1 -- 172.21.15.16:0/3574941201 shutdown_connections 2024-03-27T06:41:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.866+0000 7f2ad5288640 1 -- 172.21.15.16:0/3574941201 wait complete. 2024-03-27T06:41:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.866+0000 7f2ad5288640 1 Processor -- start 2024-03-27T06:41:05.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.866+0000 7f2ad5288640 1 -- start start 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.866+0000 7f2ad5288640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad0100a70 0x7f2ad0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.866+0000 7f2ad5288640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad0101c60 0x7f2ad0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.866+0000 7f2ad5288640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0102620 0x7f2ad019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.866+0000 7f2ad5288640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010c820 con 0x7f2ad0101c60 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.866+0000 7f2ad5288640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010c6a0 con 0x7f2ad0100a70 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.866+0000 7f2ad5288640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad010c9a0 con 0x7f2ad0102620 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ace7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad0101c60 0x7f2ad0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ace7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad0101c60 0x7f2ad0198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55576/0 (socket says 172.21.15.16:55576) 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ace7fc640 1 -- 172.21.15.16:0/974637215 learned_addr learned my addr 172.21.15.16:0/974637215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2aceffd640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad0100a70 0x7f2ad0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2acf7fe640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0102620 0x7f2ad019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ace7fc640 1 -- 172.21.15.16:0/974637215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0102620 msgr2=0x7f2ad019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ace7fc640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0102620 0x7f2ad019f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ace7fc640 1 -- 172.21.15.16:0/974637215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad0100a70 msgr2=0x7f2ad0197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ace7fc640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad0100a70 0x7f2ad0197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ace7fc640 1 -- 172.21.15.16:0/974637215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad019fb20 con 0x7f2ad0101c60 2024-03-27T06:41:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2acf7fe640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0102620 0x7f2ad019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:41:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ace7fc640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad0101c60 0x7f2ad0198440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400d950 tx=0x7f2ac400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ab7fff640 1 -- 172.21.15.16:0/974637215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4017030 con 0x7f2ad0101c60 2024-03-27T06:41:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ad5288640 1 -- 172.21.15.16:0/974637215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad019fe10 con 0x7f2ad0101c60 2024-03-27T06:41:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ab7fff640 1 -- 172.21.15.16:0/974637215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2ac4004020 con 0x7f2ad0101c60 2024-03-27T06:41:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ab7fff640 1 -- 172.21.15.16:0/974637215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4004a00 con 0x7f2ad0101c60 2024-03-27T06:41:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.867+0000 7f2ad5288640 1 -- 172.21.15.16:0/974637215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad01a02d0 con 0x7f2ad0101c60 2024-03-27T06:41:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.868+0000 7f2ad5288640 1 -- 172.21.15.16:0/974637215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad01a0560 con 0x7f2ad0101c60 2024-03-27T06:41:05.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.869+0000 7f2ab7fff640 1 -- 172.21.15.16:0/974637215 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f2ac40041d0 con 0x7f2ad0101c60 2024-03-27T06:41:05.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.870+0000 7f2ab7fff640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2ab80a3610 0x7f2ab80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:05.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.870+0000 7f2aceffd640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2ab80a3610 0x7f2ab80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:05.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.870+0000 7f2ab7fff640 1 -- 172.21.15.16:0/974637215 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2ac40cf9b0 con 0x7f2ad0101c60 2024-03-27T06:41:05.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.871+0000 7f2aceffd640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2ab80a3610 0x7f2ab80a5ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8005fd0 tx=0x7f2ac8005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:05.874+0000 7f2ab7fff640 1 -- 172.21.15.16:0/974637215 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f2ac400e350 con 0x7f2ad0101c60 2024-03-27T06:41:06.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.099+0000 7f2ad5288640 1 -- 172.21.15.16:0/974637215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ad01a0730 con 0x7f2ad0101c60 2024-03-27T06:41:06.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.100+0000 7f2ab7fff640 1 -- 172.21.15.16:0/974637215 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f2ac409ca20 con 0x7f2ad0101c60 2024-03-27T06:41:06.095 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:06.095 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-27T06:41:06.095 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 -- 172.21.15.16:0/974637215 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2ab80a3610 msgr2=0x7f2ab80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2ab80a3610 0x7f2ab80a5ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8005fd0 tx=0x7f2ac8005850 comp rx=0 tx=0).stop 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 -- 172.21.15.16:0/974637215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad0101c60 msgr2=0x7f2ad0198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad0101c60 0x7f2ad0198440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400d950 tx=0x7f2ac400de10 comp rx=0 tx=0).stop 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 -- 172.21.15.16:0/974637215 shutdown_connections 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2ab80a3610 0x7f2ab80a5ad0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0102620 0x7f2ad019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad0101c60 0x7f2ad0198440 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 --2- 172.21.15.16:0/974637215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2ad0100a70 0x7f2ad0197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 -- 172.21.15.16:0/974637215 >> 172.21.15.16:0/974637215 conn(0x7f2ad00fbfe0 msgr2=0x7f2ad00fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 -- 172.21.15.16:0/974637215 shutdown_connections 2024-03-27T06:41:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:06.102+0000 7f2ad5288640 1 -- 172.21.15.16:0/974637215 wait complete. 2024-03-27T06:41:06.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:05 smithi016 ceph-mon[31908]: Deploying daemon mds.f on smithi170 2024-03-27T06:41:06.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:05 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] up:boot 2024-03-27T06:41:06.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:05 smithi016 ceph-mon[31908]: fsmap 5 up:standby 2024-03-27T06:41:06.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:05 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T06:41:06.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:05 smithi156 ceph-mon[36261]: Deploying daemon mds.f on smithi170 2024-03-27T06:41:06.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:05 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] up:boot 2024-03-27T06:41:06.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:05 smithi156 ceph-mon[36261]: fsmap 5 up:standby 2024-03-27T06:41:06.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:05 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T06:41:07.046 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:06 smithi170 ceph-mon[36438]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:07.047 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:06 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/974637215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:07.047 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:41:06 smithi170 systemd[1]: Starting Ceph mds.f for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:41:07.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:06 smithi016 ceph-mon[31908]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:07.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:06 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/974637215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:07.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:06 smithi156 ceph-mon[36261]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:07.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:06 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/974637215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:07.316 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:41:07 smithi170 podman[67240]: 2024-03-27 06:41:07.052371375 +0000 UTC m=+0.113110520 container create dac4d49d7a3889a2bc8e5b55eff1131cceb73633c22795a6fd1d19d73978f6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-27T06:41:07.316 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:41:07 smithi170 podman[67240]: 2024-03-27 06:41:06.95709986 +0000 UTC m=+0.017839024 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:41:07.316 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:41:07 smithi170 podman[67240]: 2024-03-27 06:41:07.296786591 +0000 UTC m=+0.357525750 container init dac4d49d7a3889a2bc8e5b55eff1131cceb73633c22795a6fd1d19d73978f6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T06:41:07.317 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:41:07 smithi170 podman[67240]: 2024-03-27 06:41:07.299407944 +0000 UTC m=+0.360147093 container start dac4d49d7a3889a2bc8e5b55eff1131cceb73633c22795a6fd1d19d73978f6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-27T06:41:07.317 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:41:07 smithi170 ceph-mds[67259]: -- 172.21.15.170:0/2047904782 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562554323a40 con 0x5625542f6800 2024-03-27T06:41:07.594 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:41:07 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: starting mds.f at 2024-03-27T06:41:07.594 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:41:07 smithi170 bash[67240]: dac4d49d7a3889a2bc8e5b55eff1131cceb73633c22795a6fd1d19d73978f6e1 2024-03-27T06:41:07.594 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:41:07 smithi170 systemd[1]: Started Ceph mds.f for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:41:08.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:08.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.229+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2786138552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a70 msgr2=0x7fb798100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:08.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.229+0000 7fb79eb3a640 1 --2- 172.21.15.16:0/2786138552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a70 0x7fb798100e90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb790009ca0 tx=0x7fb79002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:41:08.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.229+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2786138552 shutdown_connections 2024-03-27T06:41:08.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.229+0000 7fb79eb3a640 1 --2- 172.21.15.16:0/2786138552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb798102620 0x7fb798108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:08.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.229+0000 7fb79eb3a640 1 --2- 172.21.15.16:0/2786138552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb798101c60 0x7fb7981020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:08.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.229+0000 7fb79eb3a640 1 --2- 172.21.15.16:0/2786138552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a70 0x7fb798100e90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:08.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.229+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2786138552 >> 172.21.15.16:0/2786138552 conn(0x7fb7980fbfe0 msgr2=0x7fb7980fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:08.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.229+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2786138552 shutdown_connections 2024-03-27T06:41:08.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.229+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2786138552 wait complete. 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79eb3a640 1 Processor -- start 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79eb3a640 1 -- start start 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79eb3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a70 0x7fb798104a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79eb3a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb798101c60 0x7fb798103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79eb3a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb798102620 0x7fb798103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79eb3a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb79810c810 con 0x7fb798100a70 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79eb3a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fb79810c690 con 0x7fb798101c60 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79eb3a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb79810c990 con 0x7fb798102620 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79c8d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a70 0x7fb798104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79c8d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a70 0x7fb798104a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55592/0 (socket says 172.21.15.16:55592) 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79c8d6640 1 -- 172.21.15.16:0/2858401464 learned_addr learned my addr 172.21.15.16:0/2858401464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79d0d7640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb798102620 0x7fb798103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb797fff640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb798101c60 0x7fb798103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:08.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79c8d6640 1 -- 172.21.15.16:0/2858401464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb798102620 msgr2=0x7fb798103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:08.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79c8d6640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb798102620 0x7fb798103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:08.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79c8d6640 1 -- 172.21.15.16:0/2858401464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb798101c60 msgr2=0x7fb798103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:08.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79c8d6640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb798101c60 0x7fb798103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:08.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79c8d6640 1 -- 172.21.15.16:0/2858401464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb798103e10 con 0x7fb798100a70 2024-03-27T06:41:08.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79d0d7640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb798102620 0x7fb798103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:41:08.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.230+0000 7fb79c8d6640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a70 0x7fb798104a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb790037040 tx=0x7fb7900346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:08.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.231+0000 7fb795ffb640 1 -- 172.21.15.16:0/2858401464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790039040 con 0x7fb798100a70 2024-03-27T06:41:08.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.231+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2858401464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7981a2030 con 0x7fb798100a70 2024-03-27T06:41:08.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.231+0000 7fb795ffb640 1 -- 172.21.15.16:0/2858401464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb790044070 con 0x7fb798100a70 2024-03-27T06:41:08.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.231+0000 7fb795ffb640 1 -- 172.21.15.16:0/2858401464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7900044d0 con 0x7fb798100a70 2024-03-27T06:41:08.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.231+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2858401464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7981a2580 con 0x7fb798100a70 2024-03-27T06:41:08.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.232+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2858401464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7981a28a0 con 0x7fb798100a70 2024-03-27T06:41:08.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.233+0000 7fb795ffb640 1 -- 172.21.15.16:0/2858401464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fb790004670 con 0x7fb798100a70 2024-03-27T06:41:08.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.233+0000 7fb795ffb640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb7800a3610 0x7fb7800a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:08.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.234+0000 7fb795ffb640 1 -- 172.21.15.16:0/2858401464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb7900f1fa0 con 0x7fb798100a70 2024-03-27T06:41:08.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.234+0000 7fb797fff640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb7800a3610 0x7fb7800a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:08.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.234+0000 7fb797fff640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb7800a3610 0x7fb7800a5ad0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb798104770 tx=0x7fb78c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:08.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.238+0000 7fb795ffb640 1 -- 172.21.15.16:0/2858401464 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb7900bf010 con 0x7fb798100a70 2024-03-27T06:41:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.465+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2858401464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7981a2a70 con 0x7fb798100a70 2024-03-27T06:41:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.465+0000 7fb795ffb640 1 -- 172.21.15.16:0/2858401464 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7fb7900be910 con 0x7fb798100a70 2024-03-27T06:41:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:08.460 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-27T06:41:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2024-03-27T06:41:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2858401464 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb7800a3610 msgr2=0x7fb7800a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb7800a3610 0x7fb7800a5ad0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb798104770 tx=0x7fb78c005f90 comp rx=0 tx=0).stop 2024-03-27T06:41:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2858401464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a70 msgr2=0x7fb798104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a70 0x7fb798104a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb790037040 tx=0x7fb7900346f0 comp rx=0 tx=0).stop 2024-03-27T06:41:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2858401464 shutdown_connections 2024-03-27T06:41:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb7800a3610 0x7fb7800a5ad0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb798102620 0x7fb798103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb798101c60 0x7fb798103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 --2- 172.21.15.16:0/2858401464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a70 0x7fb798104a00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2858401464 >> 172.21.15.16:0/2858401464 conn(0x7fb7980fbfe0 msgr2=0x7fb7980fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2858401464 shutdown_connections 2024-03-27T06:41:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:08.467+0000 7fb79eb3a640 1 -- 172.21.15.16:0/2858401464 wait complete. 2024-03-27T06:41:08.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:08 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:08.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:08 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:08.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:08 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:08.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:08 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:08.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:08 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:08.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:08 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:08.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:08 smithi156 ceph-mon[36261]: Deploying daemon mds.c on smithi170 2024-03-27T06:41:08.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:08 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6834/4279306199,v1:172.21.15.170:6835/4279306199] up:boot 2024-03-27T06:41:08.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:08 smithi156 ceph-mon[36261]: fsmap 6 up:standby 2024-03-27T06:41:08.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:08 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:41:08.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:08 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:08.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:08 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:08.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:08 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:08.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:08 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:08.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:08 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:08.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:08 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:08.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:08 smithi170 ceph-mon[36438]: Deploying daemon mds.c on smithi170 2024-03-27T06:41:08.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:08 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6834/4279306199,v1:172.21.15.170:6835/4279306199] up:boot 2024-03-27T06:41:08.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:08 smithi170 ceph-mon[36438]: fsmap 6 up:standby 2024-03-27T06:41:08.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:08 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:41:08.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:08 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:08.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:08 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:08.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:08 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:08.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:08 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:08.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:08 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:08.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:08 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:08.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:08 smithi016 ceph-mon[31908]: Deploying daemon mds.c on smithi170 2024-03-27T06:41:08.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:08 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6834/4279306199,v1:172.21.15.170:6835/4279306199] up:boot 2024-03-27T06:41:08.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:08 smithi016 ceph-mon[31908]: fsmap 6 up:standby 2024-03-27T06:41:08.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:08 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:41:09.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:09 smithi156 ceph-mon[36261]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:09.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:09 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2858401464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:09.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:09 smithi170 ceph-mon[36438]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:09.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:09 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2858401464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:09.751 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:41:09 smithi170 systemd[1]: Starting Ceph mds.c for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:41:09.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:09 smithi016 ceph-mon[31908]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:09.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:09 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2858401464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:10.039 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:41:09 smithi170 podman[67552]: 2024-03-27 06:41:09.659728513 +0000 UTC m=+0.017863100 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:41:10.039 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:41:09 smithi170 podman[67552]: 2024-03-27 06:41:09.783566746 +0000 UTC m=+0.141701316 container create 45944325da2566cbf92429140bd28a5689db9c73287f254a8f9c73f73f617eba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-03-27T06:41:10.039 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:41:10 smithi170 podman[67552]: 2024-03-27 06:41:10.04497549 +0000 UTC m=+0.403110071 container init 45944325da2566cbf92429140bd28a5689db9c73287f254a8f9c73f73f617eba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-03-27T06:41:10.344 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:41:10 smithi170 podman[67552]: 2024-03-27 06:41:10.047554623 +0000 UTC m=+0.405689189 container start 45944325da2566cbf92429140bd28a5689db9c73287f254a8f9c73f73f617eba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:41:10.344 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:41:10 smithi170 ceph-mds[67572]: -- 172.21.15.170:0/3954022023 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a259bda40 con 0x559a25990c00 2024-03-27T06:41:10.344 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:41:10 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c[67568]: starting mds.c at 2024-03-27T06:41:10.344 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:41:10 smithi170 bash[67552]: 45944325da2566cbf92429140bd28a5689db9c73287f254a8f9c73f73f617eba 2024-03-27T06:41:10.344 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:41:10 smithi170 systemd[1]: Started Ceph mds.c for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:41:10.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:10.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 -- 172.21.15.16:0/2995960931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6484100dc0 msgr2=0x7f6484101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 --2- 172.21.15.16:0/2995960931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6484100dc0 0x7f6484101220 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6478009900 tx=0x7f647802bcd0 comp rx=0 tx=0).stop 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 -- 172.21.15.16:0/2995960931 shutdown_connections 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 --2- 172.21.15.16:0/2995960931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6484101760 0x7f6484101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 --2- 172.21.15.16:0/2995960931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6484100dc0 0x7f6484101220 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 --2- 172.21.15.16:0/2995960931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6484100460 0x7f6484100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 -- 172.21.15.16:0/2995960931 >> 172.21.15.16:0/2995960931 conn(0x7f64840fbfe0 msgr2=0x7f64840fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 -- 172.21.15.16:0/2995960931 shutdown_connections 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 -- 172.21.15.16:0/2995960931 wait complete. 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 Processor -- start 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 -- start start 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6484100460 0x7f648419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6484100dc0 0x7f648419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:10.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6484101760 0x7f64841a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f648410fb70 con 0x7f6484100dc0 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f648410f9f0 con 0x7f6484101760 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.595+0000 7f648c3c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f648410fcf0 con 0x7f6484100460 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f6489964640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6484100dc0 0x7f648419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f6489964640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6484100dc0 0x7f648419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55602/0 (socket says 172.21.15.16:55602) 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f6489964640 1 -- 172.21.15.16:0/2090601540 learned_addr learned my addr 172.21.15.16:0/2090601540 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f648a966640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6484101760 0x7f64841a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f648a165640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6484100460 0x7f648419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f6489964640 1 -- 172.21.15.16:0/2090601540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6484100460 msgr2=0x7f648419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f6489964640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6484100460 0x7f648419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f6489964640 1 -- 172.21.15.16:0/2090601540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6484101760 msgr2=0x7f64841a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f6489964640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6484101760 0x7f64841a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f6489964640 1 -- 172.21.15.16:0/2090601540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64841a1d50 con 0x7f6484100dc0 2024-03-27T06:41:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f648a966640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6484101760 0x7f64841a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:41:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f6489964640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6484100dc0 0x7f648419a670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6478000c00 tx=0x7f647802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f64837fe640 1 -- 172.21.15.16:0/2090601540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478043020 con 0x7f6484100dc0 2024-03-27T06:41:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f64837fe640 1 -- 172.21.15.16:0/2090601540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f64780043d0 con 0x7f6484100dc0 2024-03-27T06:41:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f648c3c9640 1 -- 172.21.15.16:0/2090601540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64841a1fe0 con 0x7f6484100dc0 2024-03-27T06:41:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f64837fe640 1 -- 172.21.15.16:0/2090601540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478003ca0 con 0x7f6484100dc0 2024-03-27T06:41:10.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.596+0000 7f648c3c9640 1 -- 172.21.15.16:0/2090601540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64841a24f0 con 0x7f6484100dc0 2024-03-27T06:41:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.597+0000 7f648c3c9640 1 -- 172.21.15.16:0/2090601540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648410b2c0 con 0x7f6484100dc0 2024-03-27T06:41:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.598+0000 7f64837fe640 1 -- 172.21.15.16:0/2090601540 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f6478046020 con 0x7f6484100dc0 2024-03-27T06:41:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.599+0000 7f64837fe640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f646c0a3690 0x7f646c0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.599+0000 7f648a165640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f646c0a3690 0x7f646c0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.599+0000 7f64837fe640 1 -- 172.21.15.16:0/2090601540 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f64780f1910 con 0x7f6484100dc0 2024-03-27T06:41:10.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.600+0000 7f648a165640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f646c0a3690 0x7f646c0a5b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f647c009bd0 tx=0x7f647c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.604+0000 7f64837fe640 1 -- 172.21.15.16:0/2090601540 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f647804d020 con 0x7f6484100dc0 2024-03-27T06:41:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.829+0000 7f648c3c9640 1 -- 172.21.15.16:0/2090601540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f648402d520 con 0x7f6484100dc0 2024-03-27T06:41:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.830+0000 7f64837fe640 1 -- 172.21.15.16:0/2090601540 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f64780be980 con 0x7f6484100dc0 2024-03-27T06:41:10.825 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:10.825 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-27T06:41:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-03-27T06:41:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 -- 172.21.15.16:0/2090601540 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f646c0a3690 msgr2=0x7f646c0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f646c0a3690 0x7f646c0a5b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f647c009bd0 tx=0x7f647c005f90 comp rx=0 tx=0).stop 2024-03-27T06:41:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 -- 172.21.15.16:0/2090601540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6484100dc0 msgr2=0x7f648419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6484100dc0 0x7f648419a670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6478000c00 tx=0x7f647802c850 comp rx=0 tx=0).stop 2024-03-27T06:41:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 -- 172.21.15.16:0/2090601540 shutdown_connections 2024-03-27T06:41:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f646c0a3690 0x7f646c0a5b50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6484101760 0x7f64841a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6484100dc0 0x7f648419a670 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 --2- 172.21.15.16:0/2090601540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6484100460 0x7f648419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 -- 172.21.15.16:0/2090601540 >> 172.21.15.16:0/2090601540 conn(0x7f64840fbfe0 msgr2=0x7f64840fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 -- 172.21.15.16:0/2090601540 shutdown_connections 2024-03-27T06:41:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:10.832+0000 7f648c3c9640 1 -- 172.21.15.16:0/2090601540 wait complete. 2024-03-27T06:41:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: Deploying daemon mds.h on smithi156 2024-03-27T06:41:11.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] up:boot 2024-03-27T06:41:11.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: fsmap 7 up:standby 2024-03-27T06:41:11.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T06:41:11.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:11 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2090601540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:11.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:11.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:11.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:11.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:11.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:11.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:11.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:11.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: Deploying daemon mds.h on smithi156 2024-03-27T06:41:11.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] up:boot 2024-03-27T06:41:11.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: fsmap 7 up:standby 2024-03-27T06:41:11.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T06:41:11.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:11 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2090601540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:11.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:11.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:11.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:11.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:11.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:11.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:11.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:11.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: Deploying daemon mds.h on smithi156 2024-03-27T06:41:11.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] up:boot 2024-03-27T06:41:11.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: fsmap 7 up:standby 2024-03-27T06:41:11.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T06:41:11.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:11 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2090601540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:12.680 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:41:12 smithi156 systemd[1]: Starting Ceph mds.h for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:41:12.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:12.954 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:41:12 smithi156 podman[66449]: 2024-03-27 06:41:12.589150003 +0000 UTC m=+0.018027118 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:41:12.954 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:41:12 smithi156 podman[66449]: 2024-03-27 06:41:12.707117831 +0000 UTC m=+0.135994937 container create a728dab8dfcd064aa5f983e988b5aa013a26925075466bbbaf2f8f3cde73c49f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-27T06:41:12.954 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:41:12 smithi156 podman[66449]: 2024-03-27 06:41:12.959969929 +0000 UTC m=+0.388847035 container init a728dab8dfcd064aa5f983e988b5aa013a26925075466bbbaf2f8f3cde73c49f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-27T06:41:12.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.959+0000 7f9f73288640 1 -- 172.21.15.16:0/3244962489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f6c101b80 msgr2=0x7f9f6c101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.959+0000 7f9f73288640 1 --2- 172.21.15.16:0/3244962489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f6c101b80 0x7f9f6c101fa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009900 tx=0x7f9f6402bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.959+0000 7f9f73288640 1 -- 172.21.15.16:0/3244962489 shutdown_connections 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.959+0000 7f9f73288640 1 --2- 172.21.15.16:0/3244962489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c103730 0x7f9f6c109fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.959+0000 7f9f73288640 1 --2- 172.21.15.16:0/3244962489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f6c102d70 0x7f9f6c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.959+0000 7f9f73288640 1 --2- 172.21.15.16:0/3244962489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f6c101b80 0x7f9f6c101fa0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.959+0000 7f9f73288640 1 -- 172.21.15.16:0/3244962489 >> 172.21.15.16:0/3244962489 conn(0x7f9f6c0fd0f0 msgr2=0x7f9f6c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.959+0000 7f9f73288640 1 -- 172.21.15.16:0/3244962489 shutdown_connections 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.959+0000 7f9f73288640 1 -- 172.21.15.16:0/3244962489 wait complete. 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f73288640 1 Processor -- start 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f73288640 1 -- start start 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f73288640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f6c101b80 0x7f9f6c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f73288640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c102d70 0x7f9f6c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f73288640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f6c103730 0x7f9f6c19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:12.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f73288640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c10d8e0 con 0x7f9f6c103730 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f73288640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c10d760 con 0x7f9f6c101b80 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f73288640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c10da60 con 0x7f9f6c102d70 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f71825640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f6c103730 0x7f9f6c19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f71024640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f6c101b80 0x7f9f6c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f71825640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f6c103730 0x7f9f6c19f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55608/0 (socket says 172.21.15.16:55608) 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f71024640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f6c101b80 0x7f9f6c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:50128/0 (socket says 172.21.15.16:50128) 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f71825640 1 -- 172.21.15.16:0/978369376 learned_addr learned my addr 172.21.15.16:0/978369376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f70823640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c102d70 0x7f9f6c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f71825640 1 -- 172.21.15.16:0/978369376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c102d70 msgr2=0x7f9f6c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f71825640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c102d70 0x7f9f6c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f71825640 1 -- 172.21.15.16:0/978369376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f6c101b80 msgr2=0x7f9f6c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f71825640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f6c101b80 0x7f9f6c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.960+0000 7f9f71825640 1 -- 172.21.15.16:0/978369376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c19fb80 con 0x7f9f6c103730 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.961+0000 7f9f71825640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f6c103730 0x7f9f6c19f510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00cad0 tx=0x7f9f5c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.961+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/978369376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00ae20 con 0x7f9f6c103730 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.961+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/978369376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9f5c004b20 con 0x7f9f6c103730 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.961+0000 7f9f73288640 1 -- 172.21.15.16:0/978369376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6c19fe70 con 0x7f9f6c103730 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.961+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/978369376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00f780 con 0x7f9f6c103730 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.961+0000 7f9f73288640 1 -- 172.21.15.16:0/978369376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6c1a0330 con 0x7f9f6c103730 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.962+0000 7f9f73288640 1 -- 172.21.15.16:0/978369376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f6c108ea0 con 0x7f9f6c103730 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.963+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/978369376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f9f5c004cd0 con 0x7f9f6c103730 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.964+0000 7f9f6a7fc640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9f540a3610 0x7f9f540a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.964+0000 7f9f71024640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9f540a3610 0x7f9f540a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.964+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/978369376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9f5c0cf8b0 con 0x7f9f6c103730 2024-03-27T06:41:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.964+0000 7f9f71024640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9f540a3610 0x7f9f540a5ad0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9f64000c00 tx=0x7f9f64005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:12.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:12.968+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/978369376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f9f5c014030 con 0x7f9f6c103730 2024-03-27T06:41:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.202+0000 7f9f73288640 1 -- 172.21.15.16:0/978369376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f6c02d370 con 0x7f9f6c103730 2024-03-27T06:41:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.202+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/978369376 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7f9f5c09c230 con 0x7f9f6c103730 2024-03-27T06:41:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:13.197 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-27T06:41:13.197 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-03-27T06:41:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.204+0000 7f9f73288640 1 -- 172.21.15.16:0/978369376 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9f540a3610 msgr2=0x7f9f540a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.204+0000 7f9f73288640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9f540a3610 0x7f9f540a5ad0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9f64000c00 tx=0x7f9f64005fd0 comp rx=0 tx=0).stop 2024-03-27T06:41:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.204+0000 7f9f73288640 1 -- 172.21.15.16:0/978369376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f6c103730 msgr2=0x7f9f6c19f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.204+0000 7f9f73288640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f6c103730 0x7f9f6c19f510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00cad0 tx=0x7f9f5c00cf90 comp rx=0 tx=0).stop 2024-03-27T06:41:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.204+0000 7f9f73288640 1 -- 172.21.15.16:0/978369376 shutdown_connections 2024-03-27T06:41:13.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.204+0000 7f9f73288640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9f540a3610 0x7f9f540a5ad0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:13.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.204+0000 7f9f73288640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f6c103730 0x7f9f6c19f510 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:13.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.204+0000 7f9f73288640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c102d70 0x7f9f6c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:13.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.204+0000 7f9f73288640 1 --2- 172.21.15.16:0/978369376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f6c101b80 0x7f9f6c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:13.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.205+0000 7f9f73288640 1 -- 172.21.15.16:0/978369376 >> 172.21.15.16:0/978369376 conn(0x7f9f6c0fd0f0 msgr2=0x7f9f6c106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:13.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.205+0000 7f9f73288640 1 -- 172.21.15.16:0/978369376 shutdown_connections 2024-03-27T06:41:13.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:13.205+0000 7f9f73288640 1 -- 172.21.15.16:0/978369376 wait complete. 2024-03-27T06:41:13.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:13 smithi156 ceph-mon[36261]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:13.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:13.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:13.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:13.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:13.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:13.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:13 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:13.222 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:41:12 smithi156 podman[66449]: 2024-03-27 06:41:12.962744255 +0000 UTC m=+0.391621362 container start a728dab8dfcd064aa5f983e988b5aa013a26925075466bbbaf2f8f3cde73c49f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-03-27T06:41:13.222 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:41:12 smithi156 ceph-mds[66469]: -- 172.21.15.156:0/2785386762 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3e48e9a40 con 0x55b3e48bc800 2024-03-27T06:41:13.223 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:41:12 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: starting mds.h at 2024-03-27T06:41:13.223 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:41:13 smithi156 bash[66449]: a728dab8dfcd064aa5f983e988b5aa013a26925075466bbbaf2f8f3cde73c49f 2024-03-27T06:41:13.223 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:41:13 smithi156 systemd[1]: Started Ceph mds.h for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:41:13.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:13 smithi170 ceph-mon[36438]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:13.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:13.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:13.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:13.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:13.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:13.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:13 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:13.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:13 smithi016 ceph-mon[31908]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:13.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:13.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:13.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:13.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:13.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:13.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:13 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:14.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:14 smithi170 ceph-mon[36438]: Deploying daemon mds.i on smithi170 2024-03-27T06:41:14.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:14 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] up:boot 2024-03-27T06:41:14.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:14 smithi170 ceph-mon[36438]: fsmap 8 up:standby 2024-03-27T06:41:14.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:14 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T06:41:14.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:14 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/978369376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:14.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:14 smithi156 ceph-mon[36261]: Deploying daemon mds.i on smithi170 2024-03-27T06:41:14.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:14 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] up:boot 2024-03-27T06:41:14.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:14 smithi156 ceph-mon[36261]: fsmap 8 up:standby 2024-03-27T06:41:14.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:14 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T06:41:14.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:14 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/978369376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:14.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:14 smithi016 ceph-mon[31908]: Deploying daemon mds.i on smithi170 2024-03-27T06:41:14.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:14 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] up:boot 2024-03-27T06:41:14.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:14 smithi016 ceph-mon[31908]: fsmap 8 up:standby 2024-03-27T06:41:14.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:14 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T06:41:14.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:14 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/978369376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:15.211 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.331+0000 7fd24bb63640 1 -- 172.21.15.16:0/4060620703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd244100a60 msgr2=0x7fd244100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.331+0000 7fd24bb63640 1 --2- 172.21.15.16:0/4060620703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd244100a60 0x7fd244100e80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd240009900 tx=0x7fd24002bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.331+0000 7fd24bb63640 1 -- 172.21.15.16:0/4060620703 shutdown_connections 2024-03-27T06:41:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.331+0000 7fd24bb63640 1 --2- 172.21.15.16:0/4060620703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd244102610 0x7fd244108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.331+0000 7fd24bb63640 1 --2- 172.21.15.16:0/4060620703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd244101c50 0x7fd2441020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.331+0000 7fd24bb63640 1 --2- 172.21.15.16:0/4060620703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd244100a60 0x7fd244100e80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.331+0000 7fd24bb63640 1 -- 172.21.15.16:0/4060620703 >> 172.21.15.16:0/4060620703 conn(0x7fd2440fbff0 msgr2=0x7fd2440fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd24bb63640 1 -- 172.21.15.16:0/4060620703 shutdown_connections 2024-03-27T06:41:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd24bb63640 1 -- 172.21.15.16:0/4060620703 wait complete. 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd24bb63640 1 Processor -- start 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd24bb63640 1 -- start start 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd24bb63640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd244100a60 0x7fd244197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd24bb63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd244101c50 0x7fd244198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd24bb63640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd244102610 0x7fd24419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd24bb63640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410c7c0 con 0x7fd244101c50 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd24bb63640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410c640 con 0x7fd244100a60 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd24bb63640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410c940 con 0x7fd244102610 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd2490fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd244101c50 0x7fd244198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd2490fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd244101c50 0x7fd244198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55630/0 (socket says 172.21.15.16:55630) 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.332+0000 7fd2490fe640 1 -- 172.21.15.16:0/830216796 learned_addr learned my addr 172.21.15.16:0/830216796 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd24a100640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd244102610 0x7fd24419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd2498ff640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd244100a60 0x7fd244197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd2490fe640 1 -- 172.21.15.16:0/830216796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd244102610 msgr2=0x7fd24419f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd2490fe640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd244102610 0x7fd24419f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd2490fe640 1 -- 172.21.15.16:0/830216796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd244100a60 msgr2=0x7fd244197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd2490fe640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd244100a60 0x7fd244197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd2490fe640 1 -- 172.21.15.16:0/830216796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24419fb90 con 0x7fd244101c50 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd2490fe640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd244101c50 0x7fd244198500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c890 tx=0x7fd23400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd23affd640 1 -- 172.21.15.16:0/830216796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234004220 con 0x7fd244101c50 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd24bb63640 1 -- 172.21.15.16:0/830216796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24419fe80 con 0x7fd244101c50 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd23affd640 1 -- 172.21.15.16:0/830216796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd2340043c0 con 0x7fd244101c50 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd23affd640 1 -- 172.21.15.16:0/830216796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234004b10 con 0x7fd244101c50 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.333+0000 7fd24bb63640 1 -- 172.21.15.16:0/830216796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2441a0340 con 0x7fd244101c50 2024-03-27T06:41:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.334+0000 7fd24bb63640 1 -- 172.21.15.16:0/830216796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2441a05d0 con 0x7fd244101c50 2024-03-27T06:41:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.335+0000 7fd23affd640 1 -- 172.21.15.16:0/830216796 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fd2340051b0 con 0x7fd244101c50 2024-03-27T06:41:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.336+0000 7fd23affd640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd2300a3610 0x7fd2300a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.336+0000 7fd2498ff640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd2300a3610 0x7fd2300a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.336+0000 7fd23affd640 1 -- 172.21.15.16:0/830216796 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd2340cf8c0 con 0x7fd244101c50 2024-03-27T06:41:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.336+0000 7fd2498ff640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd2300a3610 0x7fd2300a5ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd240000c00 tx=0x7fd240005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.340+0000 7fd23affd640 1 -- 172.21.15.16:0/830216796 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd234014030 con 0x7fd244101c50 2024-03-27T06:41:15.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:15 smithi170 ceph-mon[36438]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:15.344 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:41:15 smithi170 systemd[1]: Starting Ceph mds.i for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:41:15.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:15 smithi156 ceph-mon[36261]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:15.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.566+0000 7fd24bb63640 1 -- 172.21.15.16:0/830216796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2441a07a0 con 0x7fd244101c50 2024-03-27T06:41:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.566+0000 7fd23affd640 1 -- 172.21.15.16:0/830216796 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7fd23409c240 con 0x7fd244101c50 2024-03-27T06:41:15.561 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:15.561 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-27T06:41:15.561 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-03-27T06:41:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 -- 172.21.15.16:0/830216796 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd2300a3610 msgr2=0x7fd2300a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd2300a3610 0x7fd2300a5ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd240000c00 tx=0x7fd240005fd0 comp rx=0 tx=0).stop 2024-03-27T06:41:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 -- 172.21.15.16:0/830216796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd244101c50 msgr2=0x7fd244198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd244101c50 0x7fd244198500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c890 tx=0x7fd23400cd50 comp rx=0 tx=0).stop 2024-03-27T06:41:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 -- 172.21.15.16:0/830216796 shutdown_connections 2024-03-27T06:41:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd2300a3610 0x7fd2300a5ad0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd244102610 0x7fd24419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd244101c50 0x7fd244198500 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:15.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 --2- 172.21.15.16:0/830216796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd244100a60 0x7fd244197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 -- 172.21.15.16:0/830216796 >> 172.21.15.16:0/830216796 conn(0x7fd2440fbff0 msgr2=0x7fd2440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 -- 172.21.15.16:0/830216796 shutdown_connections 2024-03-27T06:41:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:15.568+0000 7fd24bb63640 1 -- 172.21.15.16:0/830216796 wait complete. 2024-03-27T06:41:15.572 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:15 smithi016 ceph-mon[31908]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:15.719 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:41:15 smithi170 podman[67863]: 2024-03-27 06:41:15.355573622 +0000 UTC m=+0.018181112 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:41:15.719 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:41:15 smithi170 podman[67863]: 2024-03-27 06:41:15.455411244 +0000 UTC m=+0.118018726 container create 2ae822f5ef10d9de5ea999c4c1f0895c97b813834a2b6f3471f400b21fe21494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:41:15.719 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:41:15 smithi170 podman[67863]: 2024-03-27 06:41:15.700162119 +0000 UTC m=+0.362769613 container init 2ae822f5ef10d9de5ea999c4c1f0895c97b813834a2b6f3471f400b21fe21494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:41:15.720 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:41:15 smithi170 podman[67863]: 2024-03-27 06:41:15.702779171 +0000 UTC m=+0.365386654 container start 2ae822f5ef10d9de5ea999c4c1f0895c97b813834a2b6f3471f400b21fe21494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-27T06:41:16.093 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:41:15 smithi170 ceph-mds[67882]: -- 172.21.15.170:0/3121092204 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f983f55a40 con 0x55f983f29000 2024-03-27T06:41:16.093 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:41:15 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: starting mds.i at 2024-03-27T06:41:16.094 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:41:15 smithi170 bash[67863]: 2ae822f5ef10d9de5ea999c4c1f0895c97b813834a2b6f3471f400b21fe21494 2024-03-27T06:41:16.094 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:41:15 smithi170 systemd[1]: Started Ceph mds.i for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:41:16.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:16 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/830216796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:16.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:16 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:16.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:16 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:16.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:16 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:16.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:16 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:16.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:16 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:16.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:16 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:16.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:16 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/830216796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:16.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:16 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:16.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:16 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:16.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:16 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:16.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:16 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:16.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:16 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:16.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:16 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:16.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:16 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/830216796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:16.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:16 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:16.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:16 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:16.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:16 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:16.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:16 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:16.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:16 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:16.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:16 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:17.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:17 smithi156 ceph-mon[36261]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:17.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:17 smithi156 ceph-mon[36261]: Deploying daemon mds.g on smithi016 2024-03-27T06:41:17.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:17 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] up:boot 2024-03-27T06:41:17.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:17 smithi156 ceph-mon[36261]: fsmap 9 up:standby 2024-03-27T06:41:17.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:17 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T06:41:17.536 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:17 smithi016 ceph-mon[31908]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:17.536 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:17 smithi016 ceph-mon[31908]: Deploying daemon mds.g on smithi016 2024-03-27T06:41:17.537 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:17 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] up:boot 2024-03-27T06:41:17.537 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:17 smithi016 ceph-mon[31908]: fsmap 9 up:standby 2024-03-27T06:41:17.537 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:17 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T06:41:17.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:17.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:17 smithi170 ceph-mon[36438]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:17.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:17 smithi170 ceph-mon[36438]: Deploying daemon mds.g on smithi016 2024-03-27T06:41:17.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:17 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] up:boot 2024-03-27T06:41:17.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:17 smithi170 ceph-mon[36438]: fsmap 9 up:standby 2024-03-27T06:41:17.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:17 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T06:41:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.707+0000 7f446b640640 1 -- 172.21.15.16:0/311249265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4464101760 msgr2=0x7f4464101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.707+0000 7f446b640640 1 --2- 172.21.15.16:0/311249265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4464101760 0x7f4464101be0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f445c009ca0 tx=0x7f445c02c180 comp rx=0 tx=0).stop 2024-03-27T06:41:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.707+0000 7f446b640640 1 -- 172.21.15.16:0/311249265 shutdown_connections 2024-03-27T06:41:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.707+0000 7f446b640640 1 --2- 172.21.15.16:0/311249265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4464101760 0x7f4464101be0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.707+0000 7f446b640640 1 --2- 172.21.15.16:0/311249265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4464100dc0 0x7f4464101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.707+0000 7f446b640640 1 --2- 172.21.15.16:0/311249265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4464100460 0x7f4464100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.707+0000 7f446b640640 1 -- 172.21.15.16:0/311249265 >> 172.21.15.16:0/311249265 conn(0x7f44640fbfe0 msgr2=0x7f44640fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.707+0000 7f446b640640 1 -- 172.21.15.16:0/311249265 shutdown_connections 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.707+0000 7f446b640640 1 -- 172.21.15.16:0/311249265 wait complete. 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f446b640640 1 Processor -- start 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f446b640640 1 -- start start 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f446b640640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4464100460 0x7f446419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f446b640640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4464100dc0 0x7f446419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f446b640640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4464101760 0x7f44641a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f446b640640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f446410fb20 con 0x7f4464100460 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f446b640640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f446410f9a0 con 0x7f4464101760 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f446b640640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f446410fca0 con 0x7f4464100dc0 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f44693dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4464100460 0x7f446419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f44693dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4464100460 0x7f446419a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51324/0 (socket says 172.21.15.16:51324) 2024-03-27T06:41:17.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f44693dc640 1 -- 172.21.15.16:0/2878536674 learned_addr learned my addr 172.21.15.16:0/2878536674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f4468bdb640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4464100dc0 0x7f446419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f4469bdd640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4464101760 0x7f44641a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f44693dc640 1 -- 172.21.15.16:0/2878536674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4464100dc0 msgr2=0x7f446419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f44693dc640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4464100dc0 0x7f446419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f44693dc640 1 -- 172.21.15.16:0/2878536674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4464101760 msgr2=0x7f44641a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f44693dc640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4464101760 0x7f44641a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f44693dc640 1 -- 172.21.15.16:0/2878536674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44641a1d00 con 0x7f4464100460 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.708+0000 7f4469bdd640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4464101760 0x7f44641a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.709+0000 7f44693dc640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4464100460 0x7f446419a0e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f445400e9f0 tx=0x7f445400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.709+0000 7f445a7fc640 1 -- 172.21.15.16:0/2878536674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445400cd80 con 0x7f4464100460 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.709+0000 7f445a7fc640 1 -- 172.21.15.16:0/2878536674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4454010930 con 0x7f4464100460 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.709+0000 7f446b640640 1 -- 172.21.15.16:0/2878536674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44641a1ff0 con 0x7f4464100460 2024-03-27T06:41:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.709+0000 7f445a7fc640 1 -- 172.21.15.16:0/2878536674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454018b00 con 0x7f4464100460 2024-03-27T06:41:17.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.709+0000 7f446b640640 1 -- 172.21.15.16:0/2878536674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44641a2450 con 0x7f4464100460 2024-03-27T06:41:17.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.710+0000 7f446b640640 1 -- 172.21.15.16:0/2878536674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f446410b2c0 con 0x7f4464100460 2024-03-27T06:41:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.711+0000 7f445a7fc640 1 -- 172.21.15.16:0/2878536674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f4454018ca0 con 0x7f4464100460 2024-03-27T06:41:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.711+0000 7f445a7fc640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f44500a3610 0x7f44500a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.712+0000 7f4468bdb640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f44500a3610 0x7f44500a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.712+0000 7f445a7fc640 1 -- 172.21.15.16:0/2878536674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f44540cfde0 con 0x7f4464100460 2024-03-27T06:41:17.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.712+0000 7f4468bdb640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f44500a3610 0x7f44500a5ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f446419b5f0 tx=0x7f4460005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:17.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.716+0000 7f445a7fc640 1 -- 172.21.15.16:0/2878536674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f445409cdb0 con 0x7f4464100460 2024-03-27T06:41:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.954+0000 7f446b640640 1 -- 172.21.15.16:0/2878536674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f446402d370 con 0x7f4464100460 2024-03-27T06:41:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.955+0000 7f445a7fc640 1 -- 172.21.15.16:0/2878536674 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f446402d370 con 0x7f4464100460 2024-03-27T06:41:17.950 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:17.950 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-27T06:41:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 10 2024-03-27T06:41:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 -- 172.21.15.16:0/2878536674 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f44500a3610 msgr2=0x7f44500a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f44500a3610 0x7f44500a5ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f446419b5f0 tx=0x7f4460005f90 comp rx=0 tx=0).stop 2024-03-27T06:41:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 -- 172.21.15.16:0/2878536674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4464100460 msgr2=0x7f446419a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4464100460 0x7f446419a0e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f445400e9f0 tx=0x7f445400eeb0 comp rx=0 tx=0).stop 2024-03-27T06:41:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 -- 172.21.15.16:0/2878536674 shutdown_connections 2024-03-27T06:41:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f44500a3610 0x7f44500a5ad0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4464101760 0x7f44641a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4464100dc0 0x7f446419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 --2- 172.21.15.16:0/2878536674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4464100460 0x7f446419a0e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 -- 172.21.15.16:0/2878536674 >> 172.21.15.16:0/2878536674 conn(0x7f44640fbfe0 msgr2=0x7f44640fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 -- 172.21.15.16:0/2878536674 shutdown_connections 2024-03-27T06:41:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:17.957+0000 7f446b640640 1 -- 172.21.15.16:0/2878536674 wait complete. 2024-03-27T06:41:18.111 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:41:18 smithi016 systemd[1]: Starting Ceph mds.g for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:41:18.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:18 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2878536674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:18.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:18 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2878536674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:18.510 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:41:18 smithi016 podman[73661]: 2024-03-27 06:41:18.128740324 +0000 UTC m=+0.017935942 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:41:18.510 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:41:18 smithi016 podman[73661]: 2024-03-27 06:41:18.254390811 +0000 UTC m=+0.143586420 container create c254894f86e3a3e404337de3a86282075f0312e8cbe83d9affdc08f3cb8994ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS) 2024-03-27T06:41:18.510 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:41:18 smithi016 podman[73661]: 2024-03-27 06:41:18.515805496 +0000 UTC m=+0.405001103 container init c254894f86e3a3e404337de3a86282075f0312e8cbe83d9affdc08f3cb8994ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:41:18.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:18 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2878536674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:18.861 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:41:18 smithi016 podman[73661]: 2024-03-27 06:41:18.518778343 +0000 UTC m=+0.407973959 container start c254894f86e3a3e404337de3a86282075f0312e8cbe83d9affdc08f3cb8994ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-27T06:41:18.861 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:41:18 smithi016 ceph-mds[73680]: -- 172.21.15.16:0/645243860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bbcef49a40 con 0x55bbcef1c800 2024-03-27T06:41:18.861 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:41:18 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g[73676]: starting mds.g at 2024-03-27T06:41:18.862 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:41:18 smithi016 bash[73661]: c254894f86e3a3e404337de3a86282075f0312e8cbe83d9affdc08f3cb8994ce 2024-03-27T06:41:18.862 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:41:18 smithi016 systemd[1]: Started Ceph mds.g for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:41:19.463 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:19 smithi016 ceph-mon[31908]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:19.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:19.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:19.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:19.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:19.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:19.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:19 smithi156 ceph-mon[36261]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:19.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:19.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:19 smithi170 ceph-mon[36438]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:19.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:19.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:19.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:19.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:19.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:19.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:19.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.077+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/521830098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e14101c80 msgr2=0x7f5e14102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.077+0000 7f5e1aeb5640 1 --2- 172.21.15.16:0/521830098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e14101c80 0x7f5e14102100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009900 tx=0x7f5e0002bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.077+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/521830098 shutdown_connections 2024-03-27T06:41:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.077+0000 7f5e1aeb5640 1 --2- 172.21.15.16:0/521830098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e14102640 0x7f5e14108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.077+0000 7f5e1aeb5640 1 --2- 172.21.15.16:0/521830098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e14101c80 0x7f5e14102100 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.077+0000 7f5e1aeb5640 1 --2- 172.21.15.16:0/521830098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e14100a90 0x7f5e14100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.077+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/521830098 >> 172.21.15.16:0/521830098 conn(0x7f5e140fc000 msgr2=0x7f5e140fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.077+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/521830098 shutdown_connections 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.077+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/521830098 wait complete. 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e1aeb5640 1 Processor -- start 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e1aeb5640 1 -- start start 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e1aeb5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e14100a90 0x7f5e14197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e1aeb5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e14101c80 0x7f5e14198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e1aeb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e14102640 0x7f5e1419f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e1aeb5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1410c7f0 con 0x7f5e14102640 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e1aeb5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1410c670 con 0x7f5e14101c80 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e1aeb5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1410c970 con 0x7f5e14100a90 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e19452640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e14102640 0x7f5e1419f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e19452640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e14102640 0x7f5e1419f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51362/0 (socket says 172.21.15.16:51362) 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e19452640 1 -- 172.21.15.16:0/1614652697 learned_addr learned my addr 172.21.15.16:0/1614652697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e18c51640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e14100a90 0x7f5e14197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e13fff640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e14101c80 0x7f5e14198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e19452640 1 -- 172.21.15.16:0/1614652697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e14100a90 msgr2=0x7f5e14197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e19452640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e14100a90 0x7f5e14197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e19452640 1 -- 172.21.15.16:0/1614652697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e14101c80 msgr2=0x7f5e14198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e19452640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e14101c80 0x7f5e14198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e19452640 1 -- 172.21.15.16:0/1614652697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1419fba0 con 0x7f5e14102640 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.078+0000 7f5e19452640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e14102640 0x7f5e1419f530 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00cad0 tx=0x7f5e0c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.079+0000 7f5e11ffb640 1 -- 172.21.15.16:0/1614652697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c00ae20 con 0x7f5e14102640 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.079+0000 7f5e11ffb640 1 -- 172.21.15.16:0/1614652697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5e0c004b20 con 0x7f5e14102640 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.079+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/1614652697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1419fe90 con 0x7f5e14102640 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.079+0000 7f5e11ffb640 1 -- 172.21.15.16:0/1614652697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c00f780 con 0x7f5e14102640 2024-03-27T06:41:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.079+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/1614652697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e141a0300 con 0x7f5e14102640 2024-03-27T06:41:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.080+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/1614652697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e141a0590 con 0x7f5e14102640 2024-03-27T06:41:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.082+0000 7f5e11ffb640 1 -- 172.21.15.16:0/1614652697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f5e0c004cd0 con 0x7f5e14102640 2024-03-27T06:41:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.082+0000 7f5e11ffb640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5e040a3610 0x7f5e040a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.083+0000 7f5e18c51640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5e040a3610 0x7f5e040a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.083+0000 7f5e11ffb640 1 -- 172.21.15.16:0/1614652697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5e0c0cf910 con 0x7f5e14102640 2024-03-27T06:41:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.083+0000 7f5e18c51640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5e040a3610 0x7f5e040a5ad0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e08005fd0 tx=0x7f5e0800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.087+0000 7f5e11ffb640 1 -- 172.21.15.16:0/1614652697 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5e0c014030 con 0x7f5e14102640 2024-03-27T06:41:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.317+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/1614652697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e141a0760 con 0x7f5e14102640 2024-03-27T06:41:20.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.318+0000 7f5e11ffb640 1 -- 172.21.15.16:0/1614652697 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f5e0c09c980 con 0x7f5e14102640 2024-03-27T06:41:20.312 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:20.313 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-27T06:41:20.313 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 11 2024-03-27T06:41:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/1614652697 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5e040a3610 msgr2=0x7f5e040a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5e040a3610 0x7f5e040a5ad0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e08005fd0 tx=0x7f5e0800b390 comp rx=0 tx=0).stop 2024-03-27T06:41:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/1614652697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e14102640 msgr2=0x7f5e1419f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e14102640 0x7f5e1419f530 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00cad0 tx=0x7f5e0c00cf90 comp rx=0 tx=0).stop 2024-03-27T06:41:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/1614652697 shutdown_connections 2024-03-27T06:41:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5e040a3610 0x7f5e040a5ad0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e14102640 0x7f5e1419f530 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e14101c80 0x7f5e14198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 --2- 172.21.15.16:0/1614652697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e14100a90 0x7f5e14197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/1614652697 >> 172.21.15.16:0/1614652697 conn(0x7f5e140fc000 msgr2=0x7f5e1406a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/1614652697 shutdown_connections 2024-03-27T06:41:20.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:20.320+0000 7f5e1aeb5640 1 -- 172.21.15.16:0/1614652697 wait complete. 2024-03-27T06:41:20.324 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:20 smithi016 ceph-mon[31908]: Deploying daemon mds.a on smithi016 2024-03-27T06:41:20.324 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:20 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.16:6838/420981926,v1:172.21.15.16:6839/420981926] up:boot 2024-03-27T06:41:20.324 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:20 smithi016 ceph-mon[31908]: fsmap 10 up:standby 2024-03-27T06:41:20.324 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T06:41:20.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:20 smithi156 ceph-mon[36261]: Deploying daemon mds.a on smithi016 2024-03-27T06:41:20.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:20 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.16:6838/420981926,v1:172.21.15.16:6839/420981926] up:boot 2024-03-27T06:41:20.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:20 smithi156 ceph-mon[36261]: fsmap 10 up:standby 2024-03-27T06:41:20.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T06:41:20.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:20 smithi170 ceph-mon[36438]: Deploying daemon mds.a on smithi016 2024-03-27T06:41:20.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:20 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.16:6838/420981926,v1:172.21.15.16:6839/420981926] up:boot 2024-03-27T06:41:20.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:20 smithi170 ceph-mon[36438]: fsmap 10 up:standby 2024-03-27T06:41:20.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T06:41:21.029 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:41:20 smithi016 systemd[1]: Starting Ceph mds.a for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:41:21.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:21 smithi016 ceph-mon[31908]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:21.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:21 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1614652697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:21.282 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:41:21 smithi016 podman[74022]: 2024-03-27 06:41:20.937258001 +0000 UTC m=+0.017435141 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:41:21.282 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:41:21 smithi016 podman[74022]: 2024-03-27 06:41:21.061676402 +0000 UTC m=+0.141853564 container create e4e0d4d26dd0ac212c08730962f7aaa5ed0c54bda9424ab07d82ac836c7e6c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:41:21.282 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:41:21 smithi016 podman[74022]: 2024-03-27 06:41:21.287781321 +0000 UTC m=+0.367958441 container init e4e0d4d26dd0ac212c08730962f7aaa5ed0c54bda9424ab07d82ac836c7e6c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-03-27T06:41:21.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:21 smithi170 ceph-mon[36438]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:21.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:21 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1614652697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:21.611 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:41:21 smithi016 podman[74022]: 2024-03-27 06:41:21.290463289 +0000 UTC m=+0.370640409 container start e4e0d4d26dd0ac212c08730962f7aaa5ed0c54bda9424ab07d82ac836c7e6c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326) 2024-03-27T06:41:21.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:41:21 smithi016 ceph-mds[74041]: -- 172.21.15.16:0/4244617890 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fbb4233a40 con 0x55fbb4207000 2024-03-27T06:41:21.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:41:21 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a[74037]: starting mds.a at 2024-03-27T06:41:21.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:41:21 smithi016 bash[74022]: e4e0d4d26dd0ac212c08730962f7aaa5ed0c54bda9424ab07d82ac836c7e6c63 2024-03-27T06:41:21.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:41:21 smithi016 systemd[1]: Started Ceph mds.a for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:41:21.721 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:21 smithi156 ceph-mon[36261]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:21.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:21 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1614652697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:22.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.452+0000 7f41cffac640 1 -- 172.21.15.16:0/2532365810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8102640 msgr2=0x7f41c8108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.452+0000 7f41cffac640 1 --2- 172.21.15.16:0/2532365810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8102640 0x7f41c8108ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009ca0 tx=0x7f41b802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:41:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.452+0000 7f41cffac640 1 -- 172.21.15.16:0/2532365810 shutdown_connections 2024-03-27T06:41:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.452+0000 7f41cffac640 1 --2- 172.21.15.16:0/2532365810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8102640 0x7f41c8108ed0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.452+0000 7f41cffac640 1 --2- 172.21.15.16:0/2532365810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c8101c80 0x7f41c8102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.452+0000 7f41cffac640 1 --2- 172.21.15.16:0/2532365810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c8100a90 0x7f41c8100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.452+0000 7f41cffac640 1 -- 172.21.15.16:0/2532365810 >> 172.21.15.16:0/2532365810 conn(0x7f41c80fc000 msgr2=0x7f41c80fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.452+0000 7f41cffac640 1 -- 172.21.15.16:0/2532365810 shutdown_connections 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.452+0000 7f41cffac640 1 -- 172.21.15.16:0/2532365810 wait complete. 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41cffac640 1 Processor -- start 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41cffac640 1 -- start start 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41cffac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c8100a90 0x7f41c8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41cffac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c8101c80 0x7f41c8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41cffac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8102640 0x7f41c819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41cffac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41c810c840 con 0x7f41c8102640 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41cffac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f41c810c6c0 con 0x7f41c8100a90 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41cffac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f41c810c9c0 con 0x7f41c8101c80 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41ce549640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8102640 0x7f41c819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41ce549640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8102640 0x7f41c819f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51406/0 (socket says 172.21.15.16:51406) 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41ce549640 1 -- 172.21.15.16:0/578801931 learned_addr learned my addr 172.21.15.16:0/578801931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:22.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41cdd48640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c8100a90 0x7f41c8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.453+0000 7f41cd547640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c8101c80 0x7f41c8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41ce549640 1 -- 172.21.15.16:0/578801931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c8101c80 msgr2=0x7f41c8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41ce549640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c8101c80 0x7f41c8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41ce549640 1 -- 172.21.15.16:0/578801931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c8100a90 msgr2=0x7f41c8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41ce549640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c8100a90 0x7f41c8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41ce549640 1 -- 172.21.15.16:0/578801931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c819fb10 con 0x7f41c8102640 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41cd547640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c8101c80 0x7f41c8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41ce549640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8102640 0x7f41c819f4a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f41b8037040 tx=0x7f41b80346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41c6ffd640 1 -- 172.21.15.16:0/578801931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8039040 con 0x7f41c8102640 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41c6ffd640 1 -- 172.21.15.16:0/578801931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f41b8044070 con 0x7f41c8102640 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41cffac640 1 -- 172.21.15.16:0/578801931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c819fda0 con 0x7f41c8102640 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41c6ffd640 1 -- 172.21.15.16:0/578801931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b80045a0 con 0x7f41c8102640 2024-03-27T06:41:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.454+0000 7f41cffac640 1 -- 172.21.15.16:0/578801931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c81a0230 con 0x7f41c8102640 2024-03-27T06:41:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.455+0000 7f41cffac640 1 -- 172.21.15.16:0/578801931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c81a0460 con 0x7f41c8102640 2024-03-27T06:41:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.456+0000 7f41c6ffd640 1 -- 172.21.15.16:0/578801931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f41b8003970 con 0x7f41c8102640 2024-03-27T06:41:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.457+0000 7f41c6ffd640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f41b00a3610 0x7f41b00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.457+0000 7f41c6ffd640 1 -- 172.21.15.16:0/578801931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f41b80f2710 con 0x7f41c8102640 2024-03-27T06:41:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.457+0000 7f41cdd48640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f41b00a3610 0x7f41b00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.457+0000 7f41cdd48640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f41b00a3610 0x7f41b00a5ad0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41c0005fd0 tx=0x7f41c000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.460+0000 7f41c6ffd640 1 -- 172.21.15.16:0/578801931 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f41b80bf8a0 con 0x7f41c8102640 2024-03-27T06:41:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:22.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:22.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:22 smithi016 ceph-mon[31908]: Deploying daemon mds.b on smithi156 2024-03-27T06:41:22.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:22 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.16:6840/2365316099,v1:172.21.15.16:6841/2365316099] up:boot 2024-03-27T06:41:22.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:22 smithi016 ceph-mon[31908]: fsmap 11 up:standby 2024-03-27T06:41:22.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:22 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T06:41:22.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.686+0000 7f41cffac640 1 -- 172.21.15.16:0/578801931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41c81a0750 con 0x7f41c8102640 2024-03-27T06:41:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.686+0000 7f41c6ffd640 1 -- 172.21.15.16:0/578801931 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f41b80bf1a0 con 0x7f41c8102640 2024-03-27T06:41:22.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:22.681 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-27T06:41:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 12 2024-03-27T06:41:22.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 -- 172.21.15.16:0/578801931 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f41b00a3610 msgr2=0x7f41b00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:22.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f41b00a3610 0x7f41b00a5ad0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41c0005fd0 tx=0x7f41c000b390 comp rx=0 tx=0).stop 2024-03-27T06:41:22.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 -- 172.21.15.16:0/578801931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8102640 msgr2=0x7f41c819f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:22.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8102640 0x7f41c819f4a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f41b8037040 tx=0x7f41b80346d0 comp rx=0 tx=0).stop 2024-03-27T06:41:22.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 -- 172.21.15.16:0/578801931 shutdown_connections 2024-03-27T06:41:22.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f41b00a3610 0x7f41b00a5ad0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:22.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8102640 0x7f41c819f4a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:22.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c8101c80 0x7f41c8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:22.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 --2- 172.21.15.16:0/578801931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c8100a90 0x7f41c8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:22.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 -- 172.21.15.16:0/578801931 >> 172.21.15.16:0/578801931 conn(0x7f41c80fc000 msgr2=0x7f41c80fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:22.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 -- 172.21.15.16:0/578801931 shutdown_connections 2024-03-27T06:41:22.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:22.689+0000 7f41cffac640 1 -- 172.21.15.16:0/578801931 wait complete. 2024-03-27T06:41:22.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:22.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:22.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:22.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:22.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:22.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:22.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:22 smithi156 ceph-mon[36261]: Deploying daemon mds.b on smithi156 2024-03-27T06:41:22.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:22 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.16:6840/2365316099,v1:172.21.15.16:6841/2365316099] up:boot 2024-03-27T06:41:22.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:22 smithi156 ceph-mon[36261]: fsmap 11 up:standby 2024-03-27T06:41:22.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:22 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T06:41:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-27T06:41:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-27T06:41:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:22 smithi170 ceph-mon[36438]: Deploying daemon mds.b on smithi156 2024-03-27T06:41:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:22 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.16:6840/2365316099,v1:172.21.15.16:6841/2365316099] up:boot 2024-03-27T06:41:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:22 smithi170 ceph-mon[36438]: fsmap 11 up:standby 2024-03-27T06:41:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:22 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T06:41:23.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:23 smithi170 ceph-mon[36438]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:23 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/578801931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:23.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:23 smithi016 ceph-mon[31908]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:23.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:23 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/578801931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:23.930 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:23 smithi156 ceph-mon[36261]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:23.930 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:23 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/578801931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:23.930 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:23 smithi156 systemd[1]: Starting Ceph mds.b for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:41:24.218 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:23 smithi156 podman[66761]: 2024-03-27 06:41:23.839170914 +0000 UTC m=+0.018089267 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:41:24.218 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:23 smithi156 podman[66761]: 2024-03-27 06:41:23.962944929 +0000 UTC m=+0.141863261 container create c90d68b0db85fa4ec66f5ad51aa90ed05759053e97e0142914af734b59a4c987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:41:24.218 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:24 smithi156 podman[66761]: 2024-03-27 06:41:24.224138555 +0000 UTC m=+0.403056895 container init c90d68b0db85fa4ec66f5ad51aa90ed05759053e97e0142914af734b59a4c987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-27T06:41:24.472 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:24 smithi156 podman[66761]: 2024-03-27 06:41:24.226870023 +0000 UTC m=+0.405788367 container start c90d68b0db85fa4ec66f5ad51aa90ed05759053e97e0142914af734b59a4c987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-27T06:41:24.472 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:24 smithi156 ceph-mds[66780]: -- 172.21.15.156:0/421447014 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579b6ad3a40 con 0x5579b6aa6800 2024-03-27T06:41:24.472 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:24 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b[66776]: starting mds.b at 2024-03-27T06:41:24.472 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:24 smithi156 bash[66761]: c90d68b0db85fa4ec66f5ad51aa90ed05759053e97e0142914af734b59a4c987 2024-03-27T06:41:24.472 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:24 smithi156 systemd[1]: Started Ceph mds.b for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:41:24.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 -- 172.21.15.16:0/1178585071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54102450 msgr2=0x7fdf54108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 --2- 172.21.15.16:0/1178585071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54102450 0x7fdf54108ce0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009ca0 tx=0x7fdf4c02c0f0 comp rx=0 tx=0).stop 2024-03-27T06:41:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 -- 172.21.15.16:0/1178585071 shutdown_connections 2024-03-27T06:41:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 --2- 172.21.15.16:0/1178585071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54102450 0x7fdf54108ce0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 --2- 172.21.15.16:0/1178585071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf54101a90 0x7fdf54101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 --2- 172.21.15.16:0/1178585071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf541008a0 0x7fdf54100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 -- 172.21.15.16:0/1178585071 >> 172.21.15.16:0/1178585071 conn(0x7fdf540fbe30 msgr2=0x7fdf540fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 -- 172.21.15.16:0/1178585071 shutdown_connections 2024-03-27T06:41:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 -- 172.21.15.16:0/1178585071 wait complete. 2024-03-27T06:41:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 Processor -- start 2024-03-27T06:41:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 -- start start 2024-03-27T06:41:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf541008a0 0x7fdf54197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf54101a90 0x7fdf54198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54102450 0x7fdf5419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5410c650 con 0x7fdf54102450 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5410c4d0 con 0x7fdf54101a90 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.806+0000 7fdf599a0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5410c7d0 con 0x7fdf541008a0 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf5919f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54102450 0x7fdf5419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf5899e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf541008a0 0x7fdf54197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf5919f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54102450 0x7fdf5419f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51422/0 (socket says 172.21.15.16:51422) 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf5919f640 1 -- 172.21.15.16:0/730975002 learned_addr learned my addr 172.21.15.16:0/730975002 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf53fff640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf54101a90 0x7fdf54198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf5919f640 1 -- 172.21.15.16:0/730975002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf541008a0 msgr2=0x7fdf54197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf5919f640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf541008a0 0x7fdf54197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf5919f640 1 -- 172.21.15.16:0/730975002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf54101a90 msgr2=0x7fdf54198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf5919f640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf54101a90 0x7fdf54198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf5919f640 1 -- 172.21.15.16:0/730975002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5419f930 con 0x7fdf54102450 2024-03-27T06:41:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf5899e640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf541008a0 0x7fdf54197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:41:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf5919f640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54102450 0x7fdf5419f290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c02c700 tx=0x7fdf4c0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf51ffb640 1 -- 172.21.15.16:0/730975002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c039040 con 0x7fdf54102450 2024-03-27T06:41:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf51ffb640 1 -- 172.21.15.16:0/730975002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf4c044070 con 0x7fdf54102450 2024-03-27T06:41:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf599a0640 1 -- 172.21.15.16:0/730975002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5419fbc0 con 0x7fdf54102450 2024-03-27T06:41:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf51ffb640 1 -- 172.21.15.16:0/730975002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0043c0 con 0x7fdf54102450 2024-03-27T06:41:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.807+0000 7fdf599a0640 1 -- 172.21.15.16:0/730975002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf541a00b0 con 0x7fdf54102450 2024-03-27T06:41:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.809+0000 7fdf599a0640 1 -- 172.21.15.16:0/730975002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf541a0280 con 0x7fdf54102450 2024-03-27T06:41:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.809+0000 7fdf51ffb640 1 -- 172.21.15.16:0/730975002 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fdf4c004580 con 0x7fdf54102450 2024-03-27T06:41:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.810+0000 7fdf51ffb640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdf440a3610 0x7fdf440a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.810+0000 7fdf5899e640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdf440a3610 0x7fdf440a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.810+0000 7fdf51ffb640 1 -- 172.21.15.16:0/730975002 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdf4c0f1f30 con 0x7fdf54102450 2024-03-27T06:41:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.811+0000 7fdf5899e640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdf440a3610 0x7fdf440a5ad0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdf541991f0 tx=0x7fdf48005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:24.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:24.815+0000 7fdf51ffb640 1 -- 172.21.15.16:0/730975002 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fdf4c0befa0 con 0x7fdf54102450 2024-03-27T06:41:25.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.076+0000 7fdf599a0640 1 -- 172.21.15.16:0/730975002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf541a0610 con 0x7fdf54102450 2024-03-27T06:41:25.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.077+0000 7fdf51ffb640 1 -- 172.21.15.16:0/730975002 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7fdf4c0be8a0 con 0x7fdf54102450 2024-03-27T06:41:25.071 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:25.071 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-27T06:41:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 13 2024-03-27T06:41:25.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 -- 172.21.15.16:0/730975002 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdf440a3610 msgr2=0x7fdf440a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdf440a3610 0x7fdf440a5ad0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdf541991f0 tx=0x7fdf48005f90 comp rx=0 tx=0).stop 2024-03-27T06:41:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 -- 172.21.15.16:0/730975002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54102450 msgr2=0x7fdf5419f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54102450 0x7fdf5419f290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c02c700 tx=0x7fdf4c0346d0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 -- 172.21.15.16:0/730975002 shutdown_connections 2024-03-27T06:41:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdf440a3610 0x7fdf440a5ad0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf54102450 0x7fdf5419f290 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf54101a90 0x7fdf54198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 --2- 172.21.15.16:0/730975002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf541008a0 0x7fdf54197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 -- 172.21.15.16:0/730975002 >> 172.21.15.16:0/730975002 conn(0x7fdf540fbe30 msgr2=0x7fdf540fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 -- 172.21.15.16:0/730975002 shutdown_connections 2024-03-27T06:41:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.079+0000 7fdf599a0640 1 -- 172.21.15.16:0/730975002 wait complete. 2024-03-27T06:41:25.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:41:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.204+0000 7fdb1581c640 1 -- 172.21.15.16:0/1425121069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10102670 msgr2=0x7fdb10104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.204+0000 7fdb1581c640 1 --2- 172.21.15.16:0/1425121069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10102670 0x7fdb10104a60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009900 tx=0x7fdb0002bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.204+0000 7fdb1581c640 1 -- 172.21.15.16:0/1425121069 shutdown_connections 2024-03-27T06:41:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.204+0000 7fdb1581c640 1 --2- 172.21.15.16:0/1425121069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101078d0 0x7fdb10109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.204+0000 7fdb1581c640 1 --2- 172.21.15.16:0/1425121069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb10104fa0 0x7fdb10107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.204+0000 7fdb1581c640 1 --2- 172.21.15.16:0/1425121069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10102670 0x7fdb10104a60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.204+0000 7fdb1581c640 1 -- 172.21.15.16:0/1425121069 >> 172.21.15.16:0/1425121069 conn(0x7fdb100fc080 msgr2=0x7fdb100fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:25.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.204+0000 7fdb1581c640 1 -- 172.21.15.16:0/1425121069 shutdown_connections 2024-03-27T06:41:25.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.204+0000 7fdb1581c640 1 -- 172.21.15.16:0/1425121069 wait complete. 2024-03-27T06:41:25.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb1581c640 1 Processor -- start 2024-03-27T06:41:25.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb1581c640 1 -- start start 2024-03-27T06:41:25.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb1581c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10102670 0x7fdb10101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb1581c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb10104fa0 0x7fdb10100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb1581c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101078d0 0x7fdb10100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb1581c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1010d900 con 0x7fdb10102670 2024-03-27T06:41:25.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb1581c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1010d780 con 0x7fdb10104fa0 2024-03-27T06:41:25.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb1581c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1010da80 con 0x7fdb101078d0 2024-03-27T06:41:25.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10102670 0x7fdb10101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb0e7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb10104fa0 0x7fdb10100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb0effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10102670 0x7fdb10101fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51428/0 (socket says 172.21.15.16:51428) 2024-03-27T06:41:25.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb0effd640 1 -- 172.21.15.16:0/2666872365 learned_addr learned my addr 172.21.15.16:0/2666872365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:25.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.205+0000 7fdb0f7fe640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101078d0 0x7fdb10100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb0e7fc640 1 -- 172.21.15.16:0/2666872365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101078d0 msgr2=0x7fdb10100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb0e7fc640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101078d0 0x7fdb10100b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb0e7fc640 1 -- 172.21.15.16:0/2666872365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10102670 msgr2=0x7fdb10101fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb0e7fc640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10102670 0x7fdb10101fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb0e7fc640 1 -- 172.21.15.16:0/2666872365 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb101013f0 con 0x7fdb10104fa0 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb0effd640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10102670 0x7fdb10101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb0e7fc640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb10104fa0 0x7fdb10100620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdb04005fd0 tx=0x7fdb0400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb1481a640 1 -- 172.21.15.16:0/2666872365 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04009c00 con 0x7fdb10104fa0 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb1581c640 1 -- 172.21.15.16:0/2666872365 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb10197bf0 con 0x7fdb10104fa0 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb1481a640 1 -- 172.21.15.16:0/2666872365 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdb04009da0 con 0x7fdb10104fa0 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb1481a640 1 -- 172.21.15.16:0/2666872365 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04010460 con 0x7fdb10104fa0 2024-03-27T06:41:25.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.206+0000 7fdb1581c640 1 -- 172.21.15.16:0/2666872365 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb10198140 con 0x7fdb10104fa0 2024-03-27T06:41:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.207+0000 7fdb1581c640 1 -- 172.21.15.16:0/2666872365 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb100fdb70 con 0x7fdb10104fa0 2024-03-27T06:41:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.208+0000 7fdb1481a640 1 -- 172.21.15.16:0/2666872365 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fdb04003fb0 con 0x7fdb10104fa0 2024-03-27T06:41:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.209+0000 7fdb1481a640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdaf80a3610 0x7fdaf80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.209+0000 7fdb1481a640 1 -- 172.21.15.16:0/2666872365 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdb040cf780 con 0x7fdb10104fa0 2024-03-27T06:41:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.209+0000 7fdb0effd640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdaf80a3610 0x7fdaf80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.209+0000 7fdb0effd640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdaf80a3610 0x7fdaf80a5ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb00005ec0 tx=0x7fdb00005e30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:25.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.214+0000 7fdb1481a640 1 -- 172.21.15.16:0/2666872365 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fdb04017020 con 0x7fdb10104fa0 2024-03-27T06:41:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.402+0000 7fdb1581c640 1 -- 172.21.15.16:0/2666872365 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb10062750 con 0x7fdb10104fa0 2024-03-27T06:41:25.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.403+0000 7fdb1481a640 1 -- 172.21.15.16:0/2666872365 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18410 (secure 0 0 0) 0x7fdb0409c100 con 0x7fdb10104fa0 2024-03-27T06:41:25.399 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:25.399 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:40:41.273117+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:41:25.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 -- 172.21.15.16:0/2666872365 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdaf80a3610 msgr2=0x7fdaf80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdaf80a3610 0x7fdaf80a5ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb00005ec0 tx=0x7fdb00005e30 comp rx=0 tx=0).stop 2024-03-27T06:41:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 -- 172.21.15.16:0/2666872365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb10104fa0 msgr2=0x7fdb10100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb10104fa0 0x7fdb10100620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdb04005fd0 tx=0x7fdb0400dde0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 -- 172.21.15.16:0/2666872365 shutdown_connections 2024-03-27T06:41:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdaf80a3610 0x7fdaf80a5ad0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101078d0 0x7fdb10100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb10104fa0 0x7fdb10100620 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 --2- 172.21.15.16:0/2666872365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10102670 0x7fdb10101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 -- 172.21.15.16:0/2666872365 >> 172.21.15.16:0/2666872365 conn(0x7fdb100fc080 msgr2=0x7fdb10102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 -- 172.21.15.16:0/2666872365 shutdown_connections 2024-03-27T06:41:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.406+0000 7fdb1581c640 1 -- 172.21.15.16:0/2666872365 wait complete. 2024-03-27T06:41:25.411 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:41:25.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T06:41:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.524+0000 7f86da629640 1 -- 172.21.15.16:0/3422066709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41025d0 msgr2=0x7f86d41049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.524+0000 7f86da629640 1 --2- 172.21.15.16:0/3422066709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41025d0 0x7f86d41049c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009900 tx=0x7f86c402bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 -- 172.21.15.16:0/3422066709 shutdown_connections 2024-03-27T06:41:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 --2- 172.21.15.16:0/3422066709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4107830 0x7f86d4109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 --2- 172.21.15.16:0/3422066709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86d4104f00 0x7f86d41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 --2- 172.21.15.16:0/3422066709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d41025d0 0x7f86d41049c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 -- 172.21.15.16:0/3422066709 >> 172.21.15.16:0/3422066709 conn(0x7f86d40fbfe0 msgr2=0x7f86d40fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 -- 172.21.15.16:0/3422066709 shutdown_connections 2024-03-27T06:41:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 -- 172.21.15.16:0/3422066709 wait complete. 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 Processor -- start 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 -- start start 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d41025d0 0x7f86d4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4104f00 0x7f86d41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86d4107830 0x7f86d419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86d410d8b0 con 0x7f86d4104f00 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f86d410d730 con 0x7f86d4107830 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.525+0000 7f86da629640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86d410da30 con 0x7f86d41025d0 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4104f00 0x7f86d41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4104f00 0x7f86d41984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51446/0 (socket says 172.21.15.16:51446) 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d37fe640 1 -- 172.21.15.16:0/3684290310 learned_addr learned my addr 172.21.15.16:0/3684290310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d8bc6640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86d4107830 0x7f86d419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d3fff640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d41025d0 0x7f86d4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d37fe640 1 -- 172.21.15.16:0/3684290310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d41025d0 msgr2=0x7f86d4197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d37fe640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d41025d0 0x7f86d4197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d37fe640 1 -- 172.21.15.16:0/3684290310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86d4107830 msgr2=0x7f86d419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d37fe640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86d4107830 0x7f86d419f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d37fe640 1 -- 172.21.15.16:0/3684290310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d419fbd0 con 0x7f86d4104f00 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d37fe640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4104f00 0x7f86d41984f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e9c0 tx=0x7f86c800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d17fa640 1 -- 172.21.15.16:0/3684290310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c800ccc0 con 0x7f86d4104f00 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86da629640 1 -- 172.21.15.16:0/3684290310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d419fec0 con 0x7f86d4104f00 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d17fa640 1 -- 172.21.15.16:0/3684290310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f86c8005520 con 0x7f86d4104f00 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86d17fa640 1 -- 172.21.15.16:0/3684290310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8010690 con 0x7f86d4104f00 2024-03-27T06:41:25.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.526+0000 7f86da629640 1 -- 172.21.15.16:0/3684290310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d41a03d0 con 0x7f86d4104f00 2024-03-27T06:41:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.527+0000 7f86da629640 1 -- 172.21.15.16:0/3684290310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d40fdad0 con 0x7f86d4104f00 2024-03-27T06:41:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.529+0000 7f86d17fa640 1 -- 172.21.15.16:0/3684290310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f86c8004e40 con 0x7f86d4104f00 2024-03-27T06:41:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.530+0000 7f86d17fa640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86bc0a3610 0x7f86bc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.530+0000 7f86d3fff640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86bc0a3610 0x7f86bc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.530+0000 7f86d17fa640 1 -- 172.21.15.16:0/3684290310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f86c80d0100 con 0x7f86d4104f00 2024-03-27T06:41:25.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.530+0000 7f86d3fff640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86bc0a3610 0x7f86bc0a5ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86c4000c00 tx=0x7f86c4005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:25.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.534+0000 7f86d17fa640 1 -- 172.21.15.16:0/3684290310 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f86c809d170 con 0x7f86d4104f00 2024-03-27T06:41:25.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] up:boot 2024-03-27T06:41:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: fsmap 12 up:standby 2024-03-27T06:41:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T06:41:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:41:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:25 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/730975002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:25.689 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:25.689 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.689 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.689 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.689 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.689 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.689 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] up:boot 2024-03-27T06:41:25.689 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: fsmap 12 up:standby 2024-03-27T06:41:25.689 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T06:41:25.690 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.690 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:41:25.690 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:25 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/730975002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:25.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:25.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] up:boot 2024-03-27T06:41:25.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: fsmap 12 up:standby 2024-03-27T06:41:25.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T06:41:25.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:25.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:41:25.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:25 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/730975002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.722+0000 7f86da629640 1 -- 172.21.15.16:0/3684290310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f86d402d570 con 0x7f86d4104f00 2024-03-27T06:41:25.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.723+0000 7f86d17fa640 1 -- 172.21.15.16:0/3684290310 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f86c809ca70 con 0x7f86d4104f00 2024-03-27T06:41:25.717 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:41:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.724+0000 7f86da629640 1 -- 172.21.15.16:0/3684290310 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86bc0a3610 msgr2=0x7f86bc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.724+0000 7f86da629640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86bc0a3610 0x7f86bc0a5ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86c4000c00 tx=0x7f86c4005fd0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.724+0000 7f86da629640 1 -- 172.21.15.16:0/3684290310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4104f00 msgr2=0x7f86d41984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.724+0000 7f86da629640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4104f00 0x7f86d41984f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e9c0 tx=0x7f86c800edc0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.725+0000 7f86da629640 1 -- 172.21.15.16:0/3684290310 shutdown_connections 2024-03-27T06:41:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.725+0000 7f86da629640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86bc0a3610 0x7f86bc0a5ad0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.725+0000 7f86da629640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86d4107830 0x7f86d419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.725+0000 7f86da629640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d4104f00 0x7f86d41984f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.725+0000 7f86da629640 1 --2- 172.21.15.16:0/3684290310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d41025d0 0x7f86d4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.725+0000 7f86da629640 1 -- 172.21.15.16:0/3684290310 >> 172.21.15.16:0/3684290310 conn(0x7f86d40fbfe0 msgr2=0x7f86d4102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.725+0000 7f86da629640 1 -- 172.21.15.16:0/3684290310 shutdown_connections 2024-03-27T06:41:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.725+0000 7f86da629640 1 -- 172.21.15.16:0/3684290310 wait complete. 2024-03-27T06:41:25.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-27T06:41:25.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.839+0000 7f38dd2d4640 1 -- 172.21.15.16:0/782212380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8107910 msgr2=0x7f38d8109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.839+0000 7f38dd2d4640 1 --2- 172.21.15.16:0/782212380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8107910 0x7f38d8109d00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009ca0 tx=0x7f38d002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.839+0000 7f38dd2d4640 1 -- 172.21.15.16:0/782212380 shutdown_connections 2024-03-27T06:41:25.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.839+0000 7f38dd2d4640 1 --2- 172.21.15.16:0/782212380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8107910 0x7f38d8109d00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.839+0000 7f38dd2d4640 1 --2- 172.21.15.16:0/782212380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d8104fe0 0x7f38d81073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.839+0000 7f38dd2d4640 1 --2- 172.21.15.16:0/782212380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38d81026b0 0x7f38d8104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.839+0000 7f38dd2d4640 1 -- 172.21.15.16:0/782212380 >> 172.21.15.16:0/782212380 conn(0x7f38d80fc000 msgr2=0x7f38d80fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.839+0000 7f38dd2d4640 1 -- 172.21.15.16:0/782212380 shutdown_connections 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.839+0000 7f38dd2d4640 1 -- 172.21.15.16:0/782212380 wait complete. 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38dd2d4640 1 Processor -- start 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38dd2d4640 1 -- start start 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38dd2d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d81026b0 0x7f38d8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38dd2d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38d8104fe0 0x7f38d81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38dd2d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8107910 0x7f38d819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38dd2d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38d810d930 con 0x7f38d8107910 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38dd2d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f38d810d7b0 con 0x7f38d81026b0 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38dd2d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f38d810dab0 con 0x7f38d8104fe0 2024-03-27T06:41:25.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8107910 0x7f38d819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8107910 0x7f38d819f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37248/0 (socket says 172.21.15.16:37248) 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38d77fe640 1 -- 172.21.15.16:0/1795777030 learned_addr learned my addr 172.21.15.16:0/1795777030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38d6ffd640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d81026b0 0x7f38d8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38cffff640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38d8104fe0 0x7f38d81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38d77fe640 1 -- 172.21.15.16:0/1795777030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38d8104fe0 msgr2=0x7f38d81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38d77fe640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38d8104fe0 0x7f38d81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38d77fe640 1 -- 172.21.15.16:0/1795777030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d81026b0 msgr2=0x7f38d8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38d77fe640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d81026b0 0x7f38d8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38d77fe640 1 -- 172.21.15.16:0/1795777030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d819fac0 con 0x7f38d8107910 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38d6ffd640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d81026b0 0x7f38d8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38cffff640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38d8104fe0 0x7f38d81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:41:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.840+0000 7f38d77fe640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8107910 0x7f38d819f450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38d002c700 tx=0x7f38d00346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.841+0000 7f38d4ff9640 1 -- 172.21.15.16:0/1795777030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0039040 con 0x7f38d8107910 2024-03-27T06:41:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.841+0000 7f38dd2d4640 1 -- 172.21.15.16:0/1795777030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d819fd50 con 0x7f38d8107910 2024-03-27T06:41:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.841+0000 7f38d4ff9640 1 -- 172.21.15.16:0/1795777030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f38d0044070 con 0x7f38d8107910 2024-03-27T06:41:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.841+0000 7f38d4ff9640 1 -- 172.21.15.16:0/1795777030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d00044d0 con 0x7f38d8107910 2024-03-27T06:41:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.841+0000 7f38dd2d4640 1 -- 172.21.15.16:0/1795777030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d81a01e0 con 0x7f38d8107910 2024-03-27T06:41:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.842+0000 7f38dd2d4640 1 -- 172.21.15.16:0/1795777030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d80fdaf0 con 0x7f38d8107910 2024-03-27T06:41:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.843+0000 7f38d4ff9640 1 -- 172.21.15.16:0/1795777030 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f38d0004670 con 0x7f38d8107910 2024-03-27T06:41:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.843+0000 7f38d4ff9640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f38c40a3610 0x7f38c40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.843+0000 7f38d6ffd640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f38c40a3610 0x7f38c40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.843+0000 7f38d4ff9640 1 -- 172.21.15.16:0/1795777030 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f38d00f20b0 con 0x7f38d8107910 2024-03-27T06:41:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.844+0000 7f38d6ffd640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f38c40a3610 0x7f38c40a5ad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f38c800abe0 tx=0x7f38c800a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:25.848+0000 7f38d4ff9640 1 -- 172.21.15.16:0/1795777030 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f38d00bf120 con 0x7f38d8107910 2024-03-27T06:41:26.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.039+0000 7f38dd2d4640 1 -- 172.21.15.16:0/1795777030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f38d80627f0 con 0x7f38d8107910 2024-03-27T06:41:26.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.039+0000 7f38d4ff9640 1 -- 172.21.15.16:0/1795777030 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f38d00bea20 con 0x7f38d8107910 2024-03-27T06:41:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:26.034 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-03-27T06:41:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:41:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-03-27T06:41:26.034 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-03-27T06:41:26.035 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:41:26.035 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-03-27T06:41:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 -- 172.21.15.16:0/1795777030 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f38c40a3610 msgr2=0x7f38c40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f38c40a3610 0x7f38c40a5ad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f38c800abe0 tx=0x7f38c800a5c0 comp rx=0 tx=0).stop 2024-03-27T06:41:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 -- 172.21.15.16:0/1795777030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8107910 msgr2=0x7f38d819f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8107910 0x7f38d819f450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38d002c700 tx=0x7f38d00346d0 comp rx=0 tx=0).stop 2024-03-27T06:41:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 -- 172.21.15.16:0/1795777030 shutdown_connections 2024-03-27T06:41:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f38c40a3610 0x7f38c40a5ad0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d8107910 0x7f38d819f450 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38d8104fe0 0x7f38d81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 --2- 172.21.15.16:0/1795777030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d81026b0 0x7f38d8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 -- 172.21.15.16:0/1795777030 >> 172.21.15.16:0/1795777030 conn(0x7f38d80fc000 msgr2=0x7f38d8102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:26.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 -- 172.21.15.16:0/1795777030 shutdown_connections 2024-03-27T06:41:26.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.041+0000 7f38dd2d4640 1 -- 172.21.15.16:0/1795777030 wait complete. 2024-03-27T06:41:26.048 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-27T06:41:26.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-27T06:41:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.170+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2881019668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b8102670 msgr2=0x7fb1b8104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.170+0000 7fb1bf1e2640 1 --2- 172.21.15.16:0/2881019668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b8102670 0x7fb1b8104a60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009900 tx=0x7fb1ac02bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.170+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2881019668 shutdown_connections 2024-03-27T06:41:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.170+0000 7fb1bf1e2640 1 --2- 172.21.15.16:0/2881019668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1b81078d0 0x7fb1b8109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.170+0000 7fb1bf1e2640 1 --2- 172.21.15.16:0/2881019668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b8104fa0 0x7fb1b8107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.170+0000 7fb1bf1e2640 1 --2- 172.21.15.16:0/2881019668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b8102670 0x7fb1b8104a60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.170+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2881019668 >> 172.21.15.16:0/2881019668 conn(0x7fb1b80fc000 msgr2=0x7fb1b80fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.170+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2881019668 shutdown_connections 2024-03-27T06:41:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.170+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2881019668 wait complete. 2024-03-27T06:41:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.170+0000 7fb1bf1e2640 1 Processor -- start 2024-03-27T06:41:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.170+0000 7fb1bf1e2640 1 -- start start 2024-03-27T06:41:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1bf1e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1b8102670 0x7fb1b8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1bf1e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b8104fa0 0x7fb1b8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1bf1e2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b81078d0 0x7fb1b819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1bf1e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b810d8f0 con 0x7fb1b8104fa0 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1bf1e2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b810d770 con 0x7fb1b81078d0 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1bf1e2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b810da70 con 0x7fb1b8102670 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b8104fa0 0x7fb1b8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b8104fa0 0x7fb1b8198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37268/0 (socket says 172.21.15.16:37268) 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b7fff640 1 -- 172.21.15.16:0/2577260424 learned_addr learned my addr 172.21.15.16:0/2577260424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1bcf7e640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1b8102670 0x7fb1b8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1bd77f640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b81078d0 0x7fb1b819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b7fff640 1 -- 172.21.15.16:0/2577260424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1b8102670 msgr2=0x7fb1b8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b7fff640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1b8102670 0x7fb1b8198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b7fff640 1 -- 172.21.15.16:0/2577260424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b81078d0 msgr2=0x7fb1b819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b7fff640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b81078d0 0x7fb1b819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b7fff640 1 -- 172.21.15.16:0/2577260424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b819fc40 con 0x7fb1b8104fa0 2024-03-27T06:41:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b7fff640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b8104fa0 0x7fb1b8198560 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4005fd0 tx=0x7fb1a400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b5ffb640 1 -- 172.21.15.16:0/2577260424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4017070 con 0x7fb1b8104fa0 2024-03-27T06:41:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2577260424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b819ff30 con 0x7fb1b8104fa0 2024-03-27T06:41:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b5ffb640 1 -- 172.21.15.16:0/2577260424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb1a4009c30 con 0x7fb1b8104fa0 2024-03-27T06:41:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.171+0000 7fb1b5ffb640 1 -- 172.21.15.16:0/2577260424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4010460 con 0x7fb1b8104fa0 2024-03-27T06:41:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.172+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2577260424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b81a03f0 con 0x7fb1b8104fa0 2024-03-27T06:41:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.173+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2577260424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b80fdaf0 con 0x7fb1b8104fa0 2024-03-27T06:41:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.174+0000 7fb1b5ffb640 1 -- 172.21.15.16:0/2577260424 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fb1a4003eb0 con 0x7fb1b8104fa0 2024-03-27T06:41:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.175+0000 7fb1b5ffb640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb1a80a3610 0x7fb1a80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.175+0000 7fb1bcf7e640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb1a80a3610 0x7fb1a80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.175+0000 7fb1b5ffb640 1 -- 172.21.15.16:0/2577260424 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb1a40cf570 con 0x7fb1b8104fa0 2024-03-27T06:41:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.175+0000 7fb1bcf7e640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb1a80a3610 0x7fb1a80a5ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009230 tx=0x7fb1ac005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.179+0000 7fb1b5ffb640 1 -- 172.21.15.16:0/2577260424 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb1a4020030 con 0x7fb1b8104fa0 2024-03-27T06:41:26.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:26.389+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2577260424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb1b80627f0 con 0x7fb1b8104fa0 2024-03-27T06:41:26.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:26 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2666872365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:26.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:26 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3684290310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:41:26.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:26 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1795777030' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-27T06:41:26.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:26.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:26.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:26.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:26 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:26.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:26 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2666872365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:26 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3684290310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:41:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:26 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1795777030' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-27T06:41:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:26 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:26.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:26 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2666872365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:26.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:26 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3684290310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:41:26.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:26 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1795777030' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-27T06:41:26.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:26.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:26.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:26.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:26 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:27.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.375+0000 7fb1b5ffb640 1 -- 172.21.15.16:0/2577260424 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fb1a409c540 con 0x7fb1b8104fa0 2024-03-27T06:41:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-03-27T06:41:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2577260424 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb1a80a3610 msgr2=0x7fb1a80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb1a80a3610 0x7fb1a80a5ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009230 tx=0x7fb1ac005fd0 comp rx=0 tx=0).stop 2024-03-27T06:41:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2577260424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b8104fa0 msgr2=0x7fb1b8198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b8104fa0 0x7fb1b8198560 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4005fd0 tx=0x7fb1a400dde0 comp rx=0 tx=0).stop 2024-03-27T06:41:27.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2577260424 shutdown_connections 2024-03-27T06:41:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb1a80a3610 0x7fb1a80a5ad0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1b81078d0 0x7fb1b819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1b8104fa0 0x7fb1b8198560 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 --2- 172.21.15.16:0/2577260424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1b8102670 0x7fb1b8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2577260424 >> 172.21.15.16:0/2577260424 conn(0x7fb1b80fc000 msgr2=0x7fb1b81031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2577260424 shutdown_connections 2024-03-27T06:41:27.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.378+0000 7fb1bf1e2640 1 -- 172.21.15.16:0/2577260424 wait complete. 2024-03-27T06:41:27.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-27T06:41:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.506+0000 7f12d4716640 1 -- 172.21.15.16:0/928185725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc104f80 msgr2=0x7f12cc107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.506+0000 7f12d4716640 1 --2- 172.21.15.16:0/928185725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc104f80 0x7f12cc107370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009900 tx=0x7f12bc02bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.506+0000 7f12d4716640 1 -- 172.21.15.16:0/928185725 shutdown_connections 2024-03-27T06:41:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.506+0000 7f12d4716640 1 --2- 172.21.15.16:0/928185725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc1078b0 0x7f12cc109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.506+0000 7f12d4716640 1 --2- 172.21.15.16:0/928185725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc104f80 0x7f12cc107370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.506+0000 7f12d4716640 1 --2- 172.21.15.16:0/928185725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12cc102650 0x7f12cc104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.506+0000 7f12d4716640 1 -- 172.21.15.16:0/928185725 >> 172.21.15.16:0/928185725 conn(0x7f12cc0fc000 msgr2=0x7f12cc0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.506+0000 7f12d4716640 1 -- 172.21.15.16:0/928185725 shutdown_connections 2024-03-27T06:41:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.506+0000 7f12d4716640 1 -- 172.21.15.16:0/928185725 wait complete. 2024-03-27T06:41:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d4716640 1 Processor -- start 2024-03-27T06:41:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d4716640 1 -- start start 2024-03-27T06:41:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d4716640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc102650 0x7f12cc19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d4716640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc104f80 0x7f12cc19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d4716640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12cc1078b0 0x7f12cc1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d4716640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10d930 con 0x7f12cc102650 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d4716640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10d7b0 con 0x7f12cc104f80 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d4716640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10dab0 con 0x7f12cc1078b0 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d24b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc102650 0x7f12cc19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d24b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc102650 0x7f12cc19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37286/0 (socket says 172.21.15.16:37286) 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d24b2640 1 -- 172.21.15.16:0/514112504 learned_addr learned my addr 172.21.15.16:0/514112504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d2cb3640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12cc1078b0 0x7f12cc1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d1cb1640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc104f80 0x7f12cc19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d24b2640 1 -- 172.21.15.16:0/514112504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12cc1078b0 msgr2=0x7f12cc1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d24b2640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12cc1078b0 0x7f12cc1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d24b2640 1 -- 172.21.15.16:0/514112504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc104f80 msgr2=0x7f12cc19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d24b2640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc104f80 0x7f12cc19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d24b2640 1 -- 172.21.15.16:0/514112504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc1a1da0 con 0x7f12cc102650 2024-03-27T06:41:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d2cb3640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12cc1078b0 0x7f12cc1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:41:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.507+0000 7f12d1cb1640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc104f80 0x7f12cc19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:41:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.508+0000 7f12d24b2640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc102650 0x7f12cc19a180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12c800c920 tx=0x7f12c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.508+0000 7f12c37fe640 1 -- 172.21.15.16:0/514112504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8007c70 con 0x7f12cc102650 2024-03-27T06:41:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.508+0000 7f12d4716640 1 -- 172.21.15.16:0/514112504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc1a2090 con 0x7f12cc102650 2024-03-27T06:41:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.508+0000 7f12c37fe640 1 -- 172.21.15.16:0/514112504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f12c8007e10 con 0x7f12cc102650 2024-03-27T06:41:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.508+0000 7f12c37fe640 1 -- 172.21.15.16:0/514112504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8018940 con 0x7f12cc102650 2024-03-27T06:41:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.508+0000 7f12d4716640 1 -- 172.21.15.16:0/514112504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc1a2660 con 0x7f12cc102650 2024-03-27T06:41:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.509+0000 7f12d4716640 1 -- 172.21.15.16:0/514112504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12cc1093a0 con 0x7f12cc102650 2024-03-27T06:41:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.510+0000 7f12c37fe640 1 -- 172.21.15.16:0/514112504 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f12c8018b20 con 0x7f12cc102650 2024-03-27T06:41:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.511+0000 7f12c37fe640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f12b80a3610 0x7f12b80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.511+0000 7f12d1cb1640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f12b80a3610 0x7f12b80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.511+0000 7f12c37fe640 1 -- 172.21.15.16:0/514112504 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f12c80cfcc0 con 0x7f12cc102650 2024-03-27T06:41:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.511+0000 7f12d1cb1640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f12b80a3610 0x7f12b80a5ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f12bc006010 tx=0x7f12bc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:27.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.516+0000 7f12c37fe640 1 -- 172.21.15.16:0/514112504 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f12c80d6080 con 0x7f12cc102650 2024-03-27T06:41:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:27 smithi016 ceph-mon[31908]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:27 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2577260424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-27T06:41:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:27 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:27 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:27 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:27.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:27 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:41:27.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:27 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:27.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:27 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:41:27.619 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:27 smithi170 ceph-mon[36438]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:27.619 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:27 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2577260424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-27T06:41:27.619 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:27 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:27.619 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:27 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:27.619 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:27 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:27.619 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:27 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:41:27.619 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:27 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:27.619 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:27 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:41:27.626 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:27 smithi156 ceph-mon[36261]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:27.626 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:27 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2577260424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-27T06:41:27.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:27 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:27.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:27 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:27.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:27 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:27.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:27 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:41:27.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:27 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:27.627 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:27 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:41:27.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:27.703+0000 7f12d4716640 1 -- 172.21.15.16:0/514112504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f12cc02d370 con 0x7f12cc102650 2024-03-27T06:41:28.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.380+0000 7f12c37fe640 1 -- 172.21.15.16:0/514112504 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f12c8014030 con 0x7f12cc102650 2024-03-27T06:41:28.374 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-03-27T06:41:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.383+0000 7f12d4716640 1 -- 172.21.15.16:0/514112504 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f12b80a3610 msgr2=0x7f12b80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.383+0000 7f12d4716640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f12b80a3610 0x7f12b80a5ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f12bc006010 tx=0x7f12bc000bc0 comp rx=0 tx=0).stop 2024-03-27T06:41:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.383+0000 7f12d4716640 1 -- 172.21.15.16:0/514112504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc102650 msgr2=0x7f12cc19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.383+0000 7f12d4716640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc102650 0x7f12cc19a180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12c800c920 tx=0x7f12c800cde0 comp rx=0 tx=0).stop 2024-03-27T06:41:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.383+0000 7f12d4716640 1 -- 172.21.15.16:0/514112504 shutdown_connections 2024-03-27T06:41:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.383+0000 7f12d4716640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f12b80a3610 0x7f12b80a5ad0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.383+0000 7f12d4716640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12cc1078b0 0x7f12cc1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.383+0000 7f12d4716640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc104f80 0x7f12cc19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.383+0000 7f12d4716640 1 --2- 172.21.15.16:0/514112504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc102650 0x7f12cc19a180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:28.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.383+0000 7f12d4716640 1 -- 172.21.15.16:0/514112504 >> 172.21.15.16:0/514112504 conn(0x7f12cc0fc000 msgr2=0x7f12cc101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.385+0000 7f12d4716640 1 -- 172.21.15.16:0/514112504 shutdown_connections 2024-03-27T06:41:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.385+0000 7f12d4716640 1 -- 172.21.15.16:0/514112504 wait complete. 2024-03-27T06:41:28.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-27T06:41:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.511+0000 7fd74c4df640 1 -- 172.21.15.16:0/94830974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd744102630 msgr2=0x7fd744104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.511+0000 7fd74c4df640 1 --2- 172.21.15.16:0/94830974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd744102630 0x7fd744104a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009900 tx=0x7fd73c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.511+0000 7fd74c4df640 1 -- 172.21.15.16:0/94830974 shutdown_connections 2024-03-27T06:41:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.511+0000 7fd74c4df640 1 --2- 172.21.15.16:0/94830974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd744107890 0x7fd744109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.511+0000 7fd74c4df640 1 --2- 172.21.15.16:0/94830974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd744104f60 0x7fd744107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.511+0000 7fd74c4df640 1 --2- 172.21.15.16:0/94830974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd744102630 0x7fd744104a20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.511+0000 7fd74c4df640 1 -- 172.21.15.16:0/94830974 >> 172.21.15.16:0/94830974 conn(0x7fd7440fbfe0 msgr2=0x7fd7440fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.511+0000 7fd74c4df640 1 -- 172.21.15.16:0/94830974 shutdown_connections 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.511+0000 7fd74c4df640 1 -- 172.21.15.16:0/94830974 wait complete. 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd74c4df640 1 Processor -- start 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd74c4df640 1 -- start start 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd74c4df640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd744102630 0x7fd744197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd74c4df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd744104f60 0x7fd744198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd74c4df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd744107890 0x7fd74419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd74c4df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd74410d8b0 con 0x7fd744104f60 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd74c4df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd74410d730 con 0x7fd744107890 2024-03-27T06:41:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd74c4df640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd74410da30 con 0x7fd744102630 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd749a7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd744104f60 0x7fd744198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd749a7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd744104f60 0x7fd744198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37300/0 (socket says 172.21.15.16:37300) 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd749a7a640 1 -- 172.21.15.16:0/3952378449 learned_addr learned my addr 172.21.15.16:0/3952378449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd74aa7c640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd744107890 0x7fd74419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.512+0000 7fd74a27b640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd744102630 0x7fd744197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.513+0000 7fd749a7a640 1 -- 172.21.15.16:0/3952378449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd744102630 msgr2=0x7fd744197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.513+0000 7fd749a7a640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd744102630 0x7fd744197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.513+0000 7fd749a7a640 1 -- 172.21.15.16:0/3952378449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd744107890 msgr2=0x7fd74419f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.513+0000 7fd749a7a640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd744107890 0x7fd74419f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.513+0000 7fd749a7a640 1 -- 172.21.15.16:0/3952378449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74419fb70 con 0x7fd744104f60 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.513+0000 7fd749a7a640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd744104f60 0x7fd744198490 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd73800c970 tx=0x7fd73800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.513+0000 7fd7437fe640 1 -- 172.21.15.16:0/3952378449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738014070 con 0x7fd744104f60 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.513+0000 7fd7437fe640 1 -- 172.21.15.16:0/3952378449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd738004440 con 0x7fd744104f60 2024-03-27T06:41:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.513+0000 7fd74c4df640 1 -- 172.21.15.16:0/3952378449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74419fe60 con 0x7fd744104f60 2024-03-27T06:41:28.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.513+0000 7fd7437fe640 1 -- 172.21.15.16:0/3952378449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73800f450 con 0x7fd744104f60 2024-03-27T06:41:28.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.513+0000 7fd74c4df640 1 -- 172.21.15.16:0/3952378449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7441a0320 con 0x7fd744104f60 2024-03-27T06:41:28.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.514+0000 7fd74c4df640 1 -- 172.21.15.16:0/3952378449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7440fdad0 con 0x7fd744104f60 2024-03-27T06:41:28.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.516+0000 7fd7437fe640 1 -- 172.21.15.16:0/3952378449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fd738017070 con 0x7fd744104f60 2024-03-27T06:41:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.516+0000 7fd7437fe640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd72c0a3610 0x7fd72c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.516+0000 7fd74a27b640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd72c0a3610 0x7fd72c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.516+0000 7fd7437fe640 1 -- 172.21.15.16:0/3952378449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fd7380cff10 con 0x7fd744104f60 2024-03-27T06:41:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.517+0000 7fd74a27b640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd72c0a3610 0x7fd72c0a5ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd73c006010 tx=0x7fd73c02c9f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:28.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.520+0000 7fd7437fe640 1 -- 172.21.15.16:0/3952378449 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd73801e030 con 0x7fd744104f60 2024-03-27T06:41:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2577260424' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-27T06:41:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: osdmap e83: 12 total, 12 up, 12 in 2024-03-27T06:41:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/514112504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-27T06:41:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:28.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:41:28.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:28.745+0000 7fd74c4df640 1 -- 172.21.15.16:0/3952378449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd74402d370 con 0x7fd744104f60 2024-03-27T06:41:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2577260424' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-27T06:41:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: osdmap e83: 12 total, 12 up, 12 in 2024-03-27T06:41:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/514112504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-27T06:41:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:28.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:41:28.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2577260424' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-27T06:41:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: osdmap e83: 12 total, 12 up, 12 in 2024-03-27T06:41:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/514112504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-27T06:41:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:28.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:41:28.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:41:28.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:41:29.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.382+0000 7fd7437fe640 1 -- 172.21.15.16:0/3952378449 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd73809cc90 con 0x7fd744104f60 2024-03-27T06:41:29.377 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-03-27T06:41:29.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.386+0000 7fd74c4df640 1 -- 172.21.15.16:0/3952378449 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd72c0a3610 msgr2=0x7fd72c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:29.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.386+0000 7fd74c4df640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd72c0a3610 0x7fd72c0a5ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd73c006010 tx=0x7fd73c02c9f0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.386+0000 7fd74c4df640 1 -- 172.21.15.16:0/3952378449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd744104f60 msgr2=0x7fd744198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:29.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.386+0000 7fd74c4df640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd744104f60 0x7fd744198490 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd73800c970 tx=0x7fd73800ce30 comp rx=0 tx=0).stop 2024-03-27T06:41:29.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.386+0000 7fd74c4df640 1 -- 172.21.15.16:0/3952378449 shutdown_connections 2024-03-27T06:41:29.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.386+0000 7fd74c4df640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd72c0a3610 0x7fd72c0a5ad0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.386+0000 7fd74c4df640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd744107890 0x7fd74419f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.386+0000 7fd74c4df640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd744104f60 0x7fd744198490 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.386+0000 7fd74c4df640 1 --2- 172.21.15.16:0/3952378449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd744102630 0x7fd744197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.386+0000 7fd74c4df640 1 -- 172.21.15.16:0/3952378449 >> 172.21.15.16:0/3952378449 conn(0x7fd7440fbfe0 msgr2=0x7fd744102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.387+0000 7fd74c4df640 1 -- 172.21.15.16:0/3952378449 shutdown_connections 2024-03-27T06:41:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.387+0000 7fd74c4df640 1 -- 172.21.15.16:0/3952378449 wait complete. 2024-03-27T06:41:29.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:41:29.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.520+0000 7f206f702640 1 -- 172.21.15.16:0/1985424570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2068100da0 msgr2=0x7f2068101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.520+0000 7f206f702640 1 --2- 172.21.15.16:0/1985424570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2068100da0 0x7f2068101200 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f205c009900 tx=0x7f205c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 -- 172.21.15.16:0/1985424570 shutdown_connections 2024-03-27T06:41:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 --2- 172.21.15.16:0/1985424570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068101740 0x7f2068101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 --2- 172.21.15.16:0/1985424570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2068100da0 0x7f2068101200 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 --2- 172.21.15.16:0/1985424570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2068100440 0x7f2068100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 -- 172.21.15.16:0/1985424570 >> 172.21.15.16:0/1985424570 conn(0x7f20680fbfe0 msgr2=0x7f20680fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 -- 172.21.15.16:0/1985424570 shutdown_connections 2024-03-27T06:41:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 -- 172.21.15.16:0/1985424570 wait complete. 2024-03-27T06:41:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 Processor -- start 2024-03-27T06:41:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 -- start start 2024-03-27T06:41:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2068100440 0x7f206819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2068100da0 0x7f206819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068101740 0x7f20681a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f206810fae0 con 0x7f2068100440 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f206810f960 con 0x7f2068100da0 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206f702640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f206810fc60 con 0x7f2068101740 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.521+0000 7f206d49e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2068100440 0x7f206819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206d49e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2068100440 0x7f206819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37328/0 (socket says 172.21.15.16:37328) 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206d49e640 1 -- 172.21.15.16:0/3381654248 learned_addr learned my addr 172.21.15.16:0/3381654248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206cc9d640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2068100da0 0x7f206819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206d49e640 1 -- 172.21.15.16:0/3381654248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068101740 msgr2=0x7f20681a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206d49e640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068101740 0x7f20681a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206d49e640 1 -- 172.21.15.16:0/3381654248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2068100da0 msgr2=0x7f206819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206d49e640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2068100da0 0x7f206819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206d49e640 1 -- 172.21.15.16:0/3381654248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20681a1d30 con 0x7f2068100440 2024-03-27T06:41:29.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206cc9d640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2068100da0 0x7f206819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:41:29.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206d49e640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2068100440 0x7f206819a110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f206000d950 tx=0x7f206000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:29.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f20667fc640 1 -- 172.21.15.16:0/3381654248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060017070 con 0x7f2068100440 2024-03-27T06:41:29.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f20667fc640 1 -- 172.21.15.16:0/3381654248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2060009c30 con 0x7f2068100440 2024-03-27T06:41:29.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f20667fc640 1 -- 172.21.15.16:0/3381654248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206000f740 con 0x7f2068100440 2024-03-27T06:41:29.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206f702640 1 -- 172.21.15.16:0/3381654248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20681a1fc0 con 0x7f2068100440 2024-03-27T06:41:29.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.522+0000 7f206f702640 1 -- 172.21.15.16:0/3381654248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20681a2500 con 0x7f2068100440 2024-03-27T06:41:29.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.523+0000 7f206f702640 1 -- 172.21.15.16:0/3381654248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f206810b330 con 0x7f2068100440 2024-03-27T06:41:29.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.524+0000 7f20667fc640 1 -- 172.21.15.16:0/3381654248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f2060010a80 con 0x7f2068100440 2024-03-27T06:41:29.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.525+0000 7f20667fc640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f20500a3690 0x7f20500a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:29.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.525+0000 7f206cc9d640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f20500a3690 0x7f20500a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:29.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.525+0000 7f20667fc640 1 -- 172.21.15.16:0/3381654248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f20600cf7f0 con 0x7f2068100440 2024-03-27T06:41:29.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.525+0000 7f206cc9d640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f20500a3690 0x7f20500a5b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f205c036040 tx=0x7f205c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:29.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.528+0000 7f20667fc640 1 -- 172.21.15.16:0/3381654248 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f2060009de0 con 0x7f2068100440 2024-03-27T06:41:29.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.715+0000 7f206f702640 1 -- 172.21.15.16:0/3381654248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20680627f0 con 0x7f2068100440 2024-03-27T06:41:29.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.717+0000 7f20667fc640 1 -- 172.21.15.16:0/3381654248 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22186 (secure 0 0 0) 0x7f206009c590 con 0x7f2068100440 2024-03-27T06:41:29.711 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:29.712 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:41:28.746169+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:41:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.718+0000 7f206f702640 1 -- 172.21.15.16:0/3381654248 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f20500a3690 msgr2=0x7f20500a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.718+0000 7f206f702640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f20500a3690 0x7f20500a5b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f205c036040 tx=0x7f205c005fd0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.719+0000 7f206f702640 1 -- 172.21.15.16:0/3381654248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2068100440 msgr2=0x7f206819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.719+0000 7f206f702640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2068100440 0x7f206819a110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f206000d950 tx=0x7f206000de10 comp rx=0 tx=0).stop 2024-03-27T06:41:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.719+0000 7f206f702640 1 -- 172.21.15.16:0/3381654248 shutdown_connections 2024-03-27T06:41:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.719+0000 7f206f702640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f20500a3690 0x7f20500a5b50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.719+0000 7f206f702640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068101740 0x7f20681a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.719+0000 7f206f702640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2068100da0 0x7f206819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.719+0000 7f206f702640 1 --2- 172.21.15.16:0/3381654248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2068100440 0x7f206819a110 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.719+0000 7f206f702640 1 -- 172.21.15.16:0/3381654248 >> 172.21.15.16:0/3381654248 conn(0x7f20680fbfe0 msgr2=0x7f20680fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.719+0000 7f206f702640 1 -- 172.21.15.16:0/3381654248 shutdown_connections 2024-03-27T06:41:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.719+0000 7f206f702640 1 -- 172.21.15.16:0/3381654248 wait complete. 2024-03-27T06:41:29.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:29 smithi156 ceph-mon[36261]: pgmap v287: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:29.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:29 smithi156 ceph-mon[36261]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-27T06:41:29.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:29 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/514112504' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-27T06:41:29.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:29 smithi156 ceph-mon[36261]: osdmap e84: 12 total, 12 up, 12 in 2024-03-27T06:41:29.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:29 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3952378449' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-27T06:41:29.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:29 smithi156 ceph-mon[36261]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-27T06:41:29.723 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:29 smithi016 ceph-mon[31908]: pgmap v287: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:29.723 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:29 smithi016 ceph-mon[31908]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-27T06:41:29.723 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:29 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/514112504' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-27T06:41:29.723 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:29 smithi016 ceph-mon[31908]: osdmap e84: 12 total, 12 up, 12 in 2024-03-27T06:41:29.723 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:29 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3952378449' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-27T06:41:29.723 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:29 smithi016 ceph-mon[31908]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-27T06:41:29.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:41:29.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.833+0000 7f78b948d640 1 -- 172.21.15.16:0/2564445295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b4104fa0 msgr2=0x7f78b4107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:29.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.833+0000 7f78b948d640 1 --2- 172.21.15.16:0/2564445295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b4104fa0 0x7f78b4107390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009900 tx=0x7f78a802bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:29.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.833+0000 7f78b948d640 1 -- 172.21.15.16:0/2564445295 shutdown_connections 2024-03-27T06:41:29.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.833+0000 7f78b948d640 1 --2- 172.21.15.16:0/2564445295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78b41078d0 0x7f78b4109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.833+0000 7f78b948d640 1 --2- 172.21.15.16:0/2564445295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b4104fa0 0x7f78b4107390 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.833+0000 7f78b948d640 1 --2- 172.21.15.16:0/2564445295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78b4102670 0x7f78b4104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.833+0000 7f78b948d640 1 -- 172.21.15.16:0/2564445295 >> 172.21.15.16:0/2564445295 conn(0x7f78b40fc080 msgr2=0x7f78b40fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:29.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.833+0000 7f78b948d640 1 -- 172.21.15.16:0/2564445295 shutdown_connections 2024-03-27T06:41:29.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.833+0000 7f78b948d640 1 -- 172.21.15.16:0/2564445295 wait complete. 2024-03-27T06:41:29.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.833+0000 7f78b948d640 1 Processor -- start 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b948d640 1 -- start start 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b948d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b4102670 0x7f78b4101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b948d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78b4104fa0 0x7f78b4100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b948d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78b41078d0 0x7f78b4100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b948d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78b410d8e0 con 0x7f78b4102670 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b948d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f78b410d760 con 0x7f78b41078d0 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b948d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f78b410da60 con 0x7f78b4104fa0 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b4102670 0x7f78b4101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b4102670 0x7f78b4101fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37348/0 (socket says 172.21.15.16:37348) 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b2ffd640 1 -- 172.21.15.16:0/2342421009 learned_addr learned my addr 172.21.15.16:0/2342421009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b27fc640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78b4104fa0 0x7f78b4100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:29.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b2ffd640 1 -- 172.21.15.16:0/2342421009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78b4104fa0 msgr2=0x7f78b4100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b2ffd640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78b4104fa0 0x7f78b4100620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b2ffd640 1 -- 172.21.15.16:0/2342421009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78b41078d0 msgr2=0x7f78b4100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b37fe640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78b41078d0 0x7f78b4100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b2ffd640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78b41078d0 0x7f78b4100b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b2ffd640 1 -- 172.21.15.16:0/2342421009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b41013f0 con 0x7f78b4102670 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f78b2ffd640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b4102670 0x7f78b4101fe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c890 tx=0x7f78a400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.834+0000 7f789bfff640 1 -- 172.21.15.16:0/2342421009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4004220 con 0x7f78b4102670 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.835+0000 7f789bfff640 1 -- 172.21.15.16:0/2342421009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f78a40043c0 con 0x7f78b4102670 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.835+0000 7f78b948d640 1 -- 172.21.15.16:0/2342421009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b41a20b0 con 0x7f78b4102670 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.835+0000 7f789bfff640 1 -- 172.21.15.16:0/2342421009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4004b10 con 0x7f78b4102670 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.835+0000 7f78b948d640 1 -- 172.21.15.16:0/2342421009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b41a2600 con 0x7f78b4102670 2024-03-27T06:41:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.835+0000 7f78b948d640 1 -- 172.21.15.16:0/2342421009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b40fdb70 con 0x7f78b4102670 2024-03-27T06:41:29.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.837+0000 7f789bfff640 1 -- 172.21.15.16:0/2342421009 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f78a40051b0 con 0x7f78b4102670 2024-03-27T06:41:29.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.837+0000 7f789bfff640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f789c0a3610 0x7f789c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:29.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.837+0000 7f78b27fc640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f789c0a3610 0x7f789c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:29.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.837+0000 7f789bfff640 1 -- 172.21.15.16:0/2342421009 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f78a40cfd60 con 0x7f78b4102670 2024-03-27T06:41:29.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.838+0000 7f78b27fc640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f789c0a3610 0x7f789c0a5ad0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f78a8036040 tx=0x7f78a8005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:29.842+0000 7f789bfff640 1 -- 172.21.15.16:0/2342421009 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f78a4014030 con 0x7f78b4102670 2024-03-27T06:41:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:29 smithi170 ceph-mon[36438]: pgmap v287: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:41:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:29 smithi170 ceph-mon[36438]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-27T06:41:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:29 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/514112504' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-27T06:41:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:29 smithi170 ceph-mon[36438]: osdmap e84: 12 total, 12 up, 12 in 2024-03-27T06:41:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:29 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3952378449' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-27T06:41:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:29 smithi170 ceph-mon[36438]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-27T06:41:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.028+0000 7f78b948d640 1 -- 172.21.15.16:0/2342421009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78b4062750 con 0x7f78b4102670 2024-03-27T06:41:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.029+0000 7f789bfff640 1 -- 172.21.15.16:0/2342421009 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22186 (secure 0 0 0) 0x7f78a409ca80 con 0x7f78b4102670 2024-03-27T06:41:30.024 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:30.024 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:41:28.746169+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:41:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 -- 172.21.15.16:0/2342421009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f789c0a3610 msgr2=0x7f789c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f789c0a3610 0x7f789c0a5ad0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f78a8036040 tx=0x7f78a8005fd0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 -- 172.21.15.16:0/2342421009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b4102670 msgr2=0x7f78b4101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b4102670 0x7f78b4101fe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c890 tx=0x7f78a400cd50 comp rx=0 tx=0).stop 2024-03-27T06:41:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 -- 172.21.15.16:0/2342421009 shutdown_connections 2024-03-27T06:41:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f789c0a3610 0x7f789c0a5ad0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f78b41078d0 0x7f78b4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78b4104fa0 0x7f78b4100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 --2- 172.21.15.16:0/2342421009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b4102670 0x7f78b4101fe0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 -- 172.21.15.16:0/2342421009 >> 172.21.15.16:0/2342421009 conn(0x7f78b40fc080 msgr2=0x7f78b4102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 -- 172.21.15.16:0/2342421009 shutdown_connections 2024-03-27T06:41:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.031+0000 7f78b948d640 1 -- 172.21.15.16:0/2342421009 wait complete. 2024-03-27T06:41:30.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-27T06:41:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 -- 172.21.15.16:0/2421849749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93f8101c50 msgr2=0x7f93f81020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 --2- 172.21.15.16:0/2421849749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93f8101c50 0x7f93f81020d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009ca0 tx=0x7f93e802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 -- 172.21.15.16:0/2421849749 shutdown_connections 2024-03-27T06:41:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 --2- 172.21.15.16:0/2421849749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93f8102610 0x7f93f8108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 --2- 172.21.15.16:0/2421849749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93f8101c50 0x7f93f81020d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 --2- 172.21.15.16:0/2421849749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93f8100a60 0x7f93f8100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 -- 172.21.15.16:0/2421849749 >> 172.21.15.16:0/2421849749 conn(0x7f93f80fbfd0 msgr2=0x7f93f80fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 -- 172.21.15.16:0/2421849749 shutdown_connections 2024-03-27T06:41:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 -- 172.21.15.16:0/2421849749 wait complete. 2024-03-27T06:41:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 Processor -- start 2024-03-27T06:41:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 -- start start 2024-03-27T06:41:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93f8100a60 0x7f93f806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93f8101c50 0x7f93f819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93f8102610 0x7f93f819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93f810c810 con 0x7f93f8100a60 2024-03-27T06:41:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f93f810c690 con 0x7f93f8101c50 2024-03-27T06:41:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93ff0df640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f93f810c990 con 0x7f93f8102610 2024-03-27T06:41:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.149+0000 7f93fce7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93f8100a60 0x7f93f806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93fce7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93f8100a60 0x7f93f806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37376/0 (socket says 172.21.15.16:37376) 2024-03-27T06:41:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93fce7b640 1 -- 172.21.15.16:0/1074618848 learned_addr learned my addr 172.21.15.16:0/1074618848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93fd67c640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93f8102610 0x7f93f819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93fce7b640 1 -- 172.21.15.16:0/1074618848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93f8102610 msgr2=0x7f93f819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93fce7b640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93f8102610 0x7f93f819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93fce7b640 1 -- 172.21.15.16:0/1074618848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93f8101c50 msgr2=0x7f93f819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93fce7b640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93f8101c50 0x7f93f819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93fce7b640 1 -- 172.21.15.16:0/1074618848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f819fae0 con 0x7f93f8100a60 2024-03-27T06:41:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93fce7b640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93f8100a60 0x7f93f806ad70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f93f000d950 tx=0x7f93f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93f5ffb640 1 -- 172.21.15.16:0/1074618848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0017070 con 0x7f93f8100a60 2024-03-27T06:41:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93ff0df640 1 -- 172.21.15.16:0/1074618848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f819fdd0 con 0x7f93f8100a60 2024-03-27T06:41:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93f5ffb640 1 -- 172.21.15.16:0/1074618848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f93f0009c30 con 0x7f93f8100a60 2024-03-27T06:41:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93f5ffb640 1 -- 172.21.15.16:0/1074618848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f000f6b0 con 0x7f93f8100a60 2024-03-27T06:41:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.150+0000 7f93ff0df640 1 -- 172.21.15.16:0/1074618848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f81a0240 con 0x7f93f8100a60 2024-03-27T06:41:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.151+0000 7f93ff0df640 1 -- 172.21.15.16:0/1074618848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f8107cb0 con 0x7f93f8100a60 2024-03-27T06:41:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.152+0000 7f93f5ffb640 1 -- 172.21.15.16:0/1074618848 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f93f001a070 con 0x7f93f8100a60 2024-03-27T06:41:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.153+0000 7f93f5ffb640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93e40a3610 0x7f93e40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.153+0000 7f93f5ffb640 1 -- 172.21.15.16:0/1074618848 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f93f00cf760 con 0x7f93f8100a60 2024-03-27T06:41:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.153+0000 7f93f7fff640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93e40a3610 0x7f93e40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.154+0000 7f93f7fff640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93e40a3610 0x7f93e40a5ad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93e8006010 tx=0x7f93e80093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.158+0000 7f93f5ffb640 1 -- 172.21.15.16:0/1074618848 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f93f0009de0 con 0x7f93f8100a60 2024-03-27T06:41:30.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.386+0000 7f93ff0df640 1 -- 172.21.15.16:0/1074618848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f93f8062850 con 0x7f93f8100a60 2024-03-27T06:41:30.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.403+0000 7f93f5ffb640 1 -- 172.21.15.16:0/1074618848 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f93f009c3e0 con 0x7f93f8100a60 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 -- 172.21.15.16:0/1074618848 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93e40a3610 msgr2=0x7f93e40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93e40a3610 0x7f93e40a5ad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93e8006010 tx=0x7f93e80093f0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 -- 172.21.15.16:0/1074618848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93f8100a60 msgr2=0x7f93f806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93f8100a60 0x7f93f806ad70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f93f000d950 tx=0x7f93f000de10 comp rx=0 tx=0).stop 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 -- 172.21.15.16:0/1074618848 shutdown_connections 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93e40a3610 0x7f93e40a5ad0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93f8102610 0x7f93f819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93f8101c50 0x7f93f819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 --2- 172.21.15.16:0/1074618848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93f8100a60 0x7f93f806ad70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 -- 172.21.15.16:0/1074618848 >> 172.21.15.16:0/1074618848 conn(0x7f93f80fbfd0 msgr2=0x7f93f8104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 -- 172.21.15.16:0/1074618848 shutdown_connections 2024-03-27T06:41:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.405+0000 7f93ff0df640 1 -- 172.21.15.16:0/1074618848 wait complete. 2024-03-27T06:41:30.410 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-27T06:41:30.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-27T06:41:30.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.525+0000 7f4e90a6f640 1 -- 172.21.15.16:0/2000407728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e8c104f00 msgr2=0x7f4e8c1072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.525+0000 7f4e90a6f640 1 --2- 172.21.15.16:0/2000407728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e8c104f00 0x7f4e8c1072f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009900 tx=0x7f4e8002bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.525+0000 7f4e90a6f640 1 -- 172.21.15.16:0/2000407728 shutdown_connections 2024-03-27T06:41:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.525+0000 7f4e90a6f640 1 --2- 172.21.15.16:0/2000407728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c107830 0x7f4e8c109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.525+0000 7f4e90a6f640 1 --2- 172.21.15.16:0/2000407728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e8c104f00 0x7f4e8c1072f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.525+0000 7f4e90a6f640 1 --2- 172.21.15.16:0/2000407728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e8c1025d0 0x7f4e8c1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.525+0000 7f4e90a6f640 1 -- 172.21.15.16:0/2000407728 >> 172.21.15.16:0/2000407728 conn(0x7f4e8c0fbfc0 msgr2=0x7f4e8c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.525+0000 7f4e90a6f640 1 -- 172.21.15.16:0/2000407728 shutdown_connections 2024-03-27T06:41:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.525+0000 7f4e90a6f640 1 -- 172.21.15.16:0/2000407728 wait complete. 2024-03-27T06:41:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.526+0000 7f4e90a6f640 1 Processor -- start 2024-03-27T06:41:30.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.526+0000 7f4e90a6f640 1 -- start start 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.526+0000 7f4e90a6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e8c1025d0 0x7f4e8c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.526+0000 7f4e90a6f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e8c104f00 0x7f4e8c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.526+0000 7f4e90a6f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c107830 0x7f4e8c19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.526+0000 7f4e90a6f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c10d8b0 con 0x7f4e8c1025d0 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.526+0000 7f4e90a6f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c10d730 con 0x7f4e8c107830 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.526+0000 7f4e90a6f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c10da30 con 0x7f4e8c104f00 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.526+0000 7f4e8a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e8c1025d0 0x7f4e8c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.526+0000 7f4e8a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e8c1025d0 0x7f4e8c197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37396/0 (socket says 172.21.15.16:37396) 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.526+0000 7f4e8a59c640 1 -- 172.21.15.16:0/3227226748 learned_addr learned my addr 172.21.15.16:0/3227226748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e8ad9d640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c107830 0x7f4e8c19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e89d9b640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e8c104f00 0x7f4e8c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e8a59c640 1 -- 172.21.15.16:0/3227226748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e8c104f00 msgr2=0x7f4e8c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:30.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e8a59c640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e8c104f00 0x7f4e8c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e8a59c640 1 -- 172.21.15.16:0/3227226748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c107830 msgr2=0x7f4e8c19f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e8a59c640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c107830 0x7f4e8c19f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e8a59c640 1 -- 172.21.15.16:0/3227226748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c19fba0 con 0x7f4e8c1025d0 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e89d9b640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e8c104f00 0x7f4e8c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e8ad9d640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c107830 0x7f4e8c19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e8a59c640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e8c1025d0 0x7f4e8c197f80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400d770 tx=0x7f4e8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e7b7fe640 1 -- 172.21.15.16:0/3227226748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e84017070 con 0x7f4e8c1025d0 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e7b7fe640 1 -- 172.21.15.16:0/3227226748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4e84005520 con 0x7f4e8c1025d0 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e90a6f640 1 -- 172.21.15.16:0/3227226748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8c19fe90 con 0x7f4e8c1025d0 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e7b7fe640 1 -- 172.21.15.16:0/3227226748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e84010460 con 0x7f4e8c1025d0 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.527+0000 7f4e90a6f640 1 -- 172.21.15.16:0/3227226748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8c1a02a0 con 0x7f4e8c1025d0 2024-03-27T06:41:30.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.528+0000 7f4e90a6f640 1 -- 172.21.15.16:0/3227226748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e8c0fdab0 con 0x7f4e8c1025d0 2024-03-27T06:41:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.529+0000 7f4e7b7fe640 1 -- 172.21.15.16:0/3227226748 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f4e84004ea0 con 0x7f4e8c1025d0 2024-03-27T06:41:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.530+0000 7f4e7b7fe640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4e740a3610 0x7f4e740a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.530+0000 7f4e89d9b640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4e740a3610 0x7f4e740a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:30.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.530+0000 7f4e7b7fe640 1 -- 172.21.15.16:0/3227226748 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4e840cf830 con 0x7f4e8c1025d0 2024-03-27T06:41:30.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.530+0000 7f4e89d9b640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4e740a3610 0x7f4e740a5ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4e80036040 tx=0x7f4e80005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:30.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.534+0000 7f4e7b7fe640 1 -- 172.21.15.16:0/3227226748 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4e8409c4b0 con 0x7f4e8c1025d0 2024-03-27T06:41:30.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:30 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3952378449' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-27T06:41:30.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:30 smithi156 ceph-mon[36261]: osdmap e85: 12 total, 12 up, 12 in 2024-03-27T06:41:30.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:30 smithi156 ceph-mon[36261]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-27T06:41:30.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:30 smithi156 ceph-mon[36261]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-27T06:41:30.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:30 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3381654248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:30.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:30 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2342421009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:30.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:30 smithi156 ceph-mon[36261]: osdmap e86: 12 total, 12 up, 12 in 2024-03-27T06:41:30.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:30 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1074618848' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-27T06:41:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:30.760+0000 7f4e90a6f640 1 -- 172.21.15.16:0/3227226748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f4e8c062820 con 0x7f4e8c1025d0 2024-03-27T06:41:30.785 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:30 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3952378449' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-27T06:41:30.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:30 smithi016 ceph-mon[31908]: osdmap e85: 12 total, 12 up, 12 in 2024-03-27T06:41:30.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:30 smithi016 ceph-mon[31908]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-27T06:41:30.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:30 smithi016 ceph-mon[31908]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-27T06:41:30.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:30 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3381654248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:30.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:30 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2342421009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:30.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:30 smithi016 ceph-mon[31908]: osdmap e86: 12 total, 12 up, 12 in 2024-03-27T06:41:30.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:30 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1074618848' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-27T06:41:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:30 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3952378449' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-27T06:41:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:30 smithi170 ceph-mon[36438]: osdmap e85: 12 total, 12 up, 12 in 2024-03-27T06:41:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:30 smithi170 ceph-mon[36438]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-27T06:41:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:30 smithi170 ceph-mon[36438]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-27T06:41:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:30 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3381654248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:30 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2342421009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:30 smithi170 ceph-mon[36438]: osdmap e86: 12 total, 12 up, 12 in 2024-03-27T06:41:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:30 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1074618848' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-27T06:41:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.471+0000 7f4e7b7fe640 1 -- 172.21.15.16:0/3227226748 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f4e8409bdb0 con 0x7f4e8c1025d0 2024-03-27T06:41:31.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 -- 172.21.15.16:0/3227226748 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4e740a3610 msgr2=0x7f4e740a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4e740a3610 0x7f4e740a5ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4e80036040 tx=0x7f4e80005fd0 comp rx=0 tx=0).stop 2024-03-27T06:41:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 -- 172.21.15.16:0/3227226748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e8c1025d0 msgr2=0x7f4e8c197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e8c1025d0 0x7f4e8c197f80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400d770 tx=0x7f4e8400dc30 comp rx=0 tx=0).stop 2024-03-27T06:41:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 -- 172.21.15.16:0/3227226748 shutdown_connections 2024-03-27T06:41:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4e740a3610 0x7f4e740a5ad0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e8c107830 0x7f4e8c19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e8c104f00 0x7f4e8c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 --2- 172.21.15.16:0/3227226748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e8c1025d0 0x7f4e8c197f80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 -- 172.21.15.16:0/3227226748 >> 172.21.15.16:0/3227226748 conn(0x7f4e8c0fbfc0 msgr2=0x7f4e8c103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 -- 172.21.15.16:0/3227226748 shutdown_connections 2024-03-27T06:41:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.474+0000 7f4e90a6f640 1 -- 172.21.15.16:0/3227226748 wait complete. 2024-03-27T06:41:31.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-27T06:41:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.604+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/972162516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4103700 msgr2=0x7f0cf4109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.604+0000 7f0cf8b1e640 1 --2- 172.21.15.16:0/972162516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4103700 0x7f0cf4109f90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009ca0 tx=0x7f0ce402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:41:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.604+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/972162516 shutdown_connections 2024-03-27T06:41:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.604+0000 7f0cf8b1e640 1 --2- 172.21.15.16:0/972162516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4103700 0x7f0cf4109f90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.604+0000 7f0cf8b1e640 1 --2- 172.21.15.16:0/972162516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf4102d40 0x7f0cf41031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.604+0000 7f0cf8b1e640 1 --2- 172.21.15.16:0/972162516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf4101b50 0x7f0cf4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.604+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/972162516 >> 172.21.15.16:0/972162516 conn(0x7f0cf40fd0c0 msgr2=0x7f0cf40ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.604+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/972162516 shutdown_connections 2024-03-27T06:41:31.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.604+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/972162516 wait complete. 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf8b1e640 1 Processor -- start 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf8b1e640 1 -- start start 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf8b1e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf4101b50 0x7f0cf4197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf8b1e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf4102d40 0x7f0cf4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf8b1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4103700 0x7f0cf419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf8b1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410d900 con 0x7f0cf4103700 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf8b1e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410d780 con 0x7f0cf4102d40 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf8b1e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410da80 con 0x7f0cf4101b50 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4103700 0x7f0cf419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4103700 0x7f0cf419f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37410/0 (socket says 172.21.15.16:37410) 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf2d9d640 1 -- 172.21.15.16:0/3234631009 learned_addr learned my addr 172.21.15.16:0/3234631009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf259c640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf4101b50 0x7f0cf4197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf1d9b640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf4102d40 0x7f0cf4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf2d9d640 1 -- 172.21.15.16:0/3234631009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf4101b50 msgr2=0x7f0cf4197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf2d9d640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf4101b50 0x7f0cf4197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf2d9d640 1 -- 172.21.15.16:0/3234631009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf4102d40 msgr2=0x7f0cf4198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf2d9d640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf4102d40 0x7f0cf4198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf2d9d640 1 -- 172.21.15.16:0/3234631009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf419fb00 con 0x7f0cf4103700 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf1d9b640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf4102d40 0x7f0cf4198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf2d9d640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4103700 0x7f0cf419f490 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ce402c700 tx=0x7f0ce40346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0ceb7fe640 1 -- 172.21.15.16:0/3234631009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4039040 con 0x7f0cf4103700 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.605+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/3234631009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf419fd90 con 0x7f0cf4103700 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.606+0000 7f0ceb7fe640 1 -- 172.21.15.16:0/3234631009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0ce4044070 con 0x7f0cf4103700 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.606+0000 7f0ceb7fe640 1 -- 172.21.15.16:0/3234631009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40044d0 con 0x7f0cf4103700 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.606+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/3234631009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf41a0220 con 0x7f0cf4103700 2024-03-27T06:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.606+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/3234631009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf4108da0 con 0x7f0cf4103700 2024-03-27T06:41:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.608+0000 7f0ceb7fe640 1 -- 172.21.15.16:0/3234631009 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f0ce4004670 con 0x7f0cf4103700 2024-03-27T06:41:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.608+0000 7f0ceb7fe640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0ce00a3610 0x7f0ce00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.608+0000 7f0ceb7fe640 1 -- 172.21.15.16:0/3234631009 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0ce40f29f0 con 0x7f0cf4103700 2024-03-27T06:41:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.608+0000 7f0cf259c640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0ce00a3610 0x7f0ce00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.609+0000 7f0cf259c640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0ce00a3610 0x7f0ce00a5ad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0cec005fd0 tx=0x7f0cec00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.612+0000 7f0ceb7fe640 1 -- 172.21.15.16:0/3234631009 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f0ce40bf710 con 0x7f0cf4103700 2024-03-27T06:41:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:31 smithi156 ceph-mon[36261]: pgmap v290: 97 pgs: 20 creating+peering, 10 active+clean, 67 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-27T06:41:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:31 smithi156 ceph-mon[36261]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-27T06:41:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:31 smithi156 ceph-mon[36261]: Cluster is now healthy 2024-03-27T06:41:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:31 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1074618848' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-27T06:41:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:31 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] up:active 2024-03-27T06:41:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:31 smithi156 ceph-mon[36261]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T06:41:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:31 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3227226748' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-27T06:41:31.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:31.839+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/3234631009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0cf402d370 con 0x7f0cf4103700 2024-03-27T06:41:31.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:31 smithi170 ceph-mon[36438]: pgmap v290: 97 pgs: 20 creating+peering, 10 active+clean, 67 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-27T06:41:31.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:31 smithi170 ceph-mon[36438]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-27T06:41:31.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:31 smithi170 ceph-mon[36438]: Cluster is now healthy 2024-03-27T06:41:31.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:31 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1074618848' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-27T06:41:31.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:31 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] up:active 2024-03-27T06:41:31.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:31 smithi170 ceph-mon[36438]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T06:41:31.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:31 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3227226748' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-27T06:41:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:31 smithi016 ceph-mon[31908]: pgmap v290: 97 pgs: 20 creating+peering, 10 active+clean, 67 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-27T06:41:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:31 smithi016 ceph-mon[31908]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-27T06:41:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:31 smithi016 ceph-mon[31908]: Cluster is now healthy 2024-03-27T06:41:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:31 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1074618848' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-27T06:41:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:31 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] up:active 2024-03-27T06:41:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:31 smithi016 ceph-mon[31908]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T06:41:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:31 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3227226748' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-27T06:41:32.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.481+0000 7f0ceb7fe640 1 -- 172.21.15.16:0/3234631009 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f0ce40bf010 con 0x7f0cf4103700 2024-03-27T06:41:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.483+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/3234631009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0ce00a3610 msgr2=0x7f0ce00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.483+0000 7f0cf8b1e640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0ce00a3610 0x7f0ce00a5ad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0cec005fd0 tx=0x7f0cec00b390 comp rx=0 tx=0).stop 2024-03-27T06:41:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.483+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/3234631009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4103700 msgr2=0x7f0cf419f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.483+0000 7f0cf8b1e640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4103700 0x7f0cf419f490 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ce402c700 tx=0x7f0ce40346d0 comp rx=0 tx=0).stop 2024-03-27T06:41:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.484+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/3234631009 shutdown_connections 2024-03-27T06:41:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.484+0000 7f0cf8b1e640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0ce00a3610 0x7f0ce00a5ad0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.484+0000 7f0cf8b1e640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4103700 0x7f0cf419f490 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.484+0000 7f0cf8b1e640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf4102d40 0x7f0cf4198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.484+0000 7f0cf8b1e640 1 --2- 172.21.15.16:0/3234631009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf4101b50 0x7f0cf4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.484+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/3234631009 >> 172.21.15.16:0/3234631009 conn(0x7f0cf40fd0c0 msgr2=0x7f0cf4105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.484+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/3234631009 shutdown_connections 2024-03-27T06:41:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.484+0000 7f0cf8b1e640 1 -- 172.21.15.16:0/3234631009 wait complete. 2024-03-27T06:41:32.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-27T06:41:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.612+0000 7f757859a640 1 -- 172.21.15.16:0/391201127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570101b60 msgr2=0x7f7570101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.612+0000 7f757859a640 1 --2- 172.21.15.16:0/391201127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570101b60 0x7f7570101f80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f756c009900 tx=0x7f756c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.612+0000 7f757859a640 1 -- 172.21.15.16:0/391201127 shutdown_connections 2024-03-27T06:41:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.612+0000 7f757859a640 1 --2- 172.21.15.16:0/391201127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7570103710 0x7f7570109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.612+0000 7f757859a640 1 --2- 172.21.15.16:0/391201127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7570102d50 0x7f75701031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.612+0000 7f757859a640 1 --2- 172.21.15.16:0/391201127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570101b60 0x7f7570101f80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.612+0000 7f757859a640 1 -- 172.21.15.16:0/391201127 >> 172.21.15.16:0/391201127 conn(0x7f75700fd0f0 msgr2=0x7f75700ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.612+0000 7f757859a640 1 -- 172.21.15.16:0/391201127 shutdown_connections 2024-03-27T06:41:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.612+0000 7f757859a640 1 -- 172.21.15.16:0/391201127 wait complete. 2024-03-27T06:41:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.612+0000 7f757859a640 1 Processor -- start 2024-03-27T06:41:32.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f757859a640 1 -- start start 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f757859a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7570101b60 0x7f7570197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f757859a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570102d50 0x7f7570198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f757859a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7570103710 0x7f757019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f757859a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f757010d920 con 0x7f7570102d50 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f757859a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f757010d7a0 con 0x7f7570101b60 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f757859a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f757010daa0 con 0x7f7570103710 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f7575b35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570102d50 0x7f7570198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f7575b35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570102d50 0x7f7570198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37434/0 (socket says 172.21.15.16:37434) 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f7575b35640 1 -- 172.21.15.16:0/2214958890 learned_addr learned my addr 172.21.15.16:0/2214958890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f7576336640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7570101b60 0x7f7570197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f7576b37640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7570103710 0x7f757019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:32.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f7575b35640 1 -- 172.21.15.16:0/2214958890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7570103710 msgr2=0x7f757019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:32.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f7575b35640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7570103710 0x7f757019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:32.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f7575b35640 1 -- 172.21.15.16:0/2214958890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7570101b60 msgr2=0x7f7570197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:32.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f7575b35640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7570101b60 0x7f7570197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:32.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f7575b35640 1 -- 172.21.15.16:0/2214958890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757019fc10 con 0x7f7570102d50 2024-03-27T06:41:32.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f7575b35640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570102d50 0x7f7570198530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f756000b4a0 tx=0x7f756000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:32.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f75677fe640 1 -- 172.21.15.16:0/2214958890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560014070 con 0x7f7570102d50 2024-03-27T06:41:32.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f75677fe640 1 -- 172.21.15.16:0/2214958890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7560010b10 con 0x7f7570102d50 2024-03-27T06:41:32.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f757859a640 1 -- 172.21.15.16:0/2214958890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757019ff00 con 0x7f7570102d50 2024-03-27T06:41:32.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.613+0000 7f75677fe640 1 -- 172.21.15.16:0/2214958890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560017d60 con 0x7f7570102d50 2024-03-27T06:41:32.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.614+0000 7f757859a640 1 -- 172.21.15.16:0/2214958890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701a0440 con 0x7f7570102d50 2024-03-27T06:41:32.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.615+0000 7f757859a640 1 -- 172.21.15.16:0/2214958890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7570108f00 con 0x7f7570102d50 2024-03-27T06:41:32.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.616+0000 7f75677fe640 1 -- 172.21.15.16:0/2214958890 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f7560017070 con 0x7f7570102d50 2024-03-27T06:41:32.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.616+0000 7f75677fe640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f755c0a3610 0x7f755c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:32.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.617+0000 7f7576336640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f755c0a3610 0x7f755c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:32.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.617+0000 7f75677fe640 1 -- 172.21.15.16:0/2214958890 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f75600cef60 con 0x7f7570102d50 2024-03-27T06:41:32.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.617+0000 7f7576336640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f755c0a3610 0x7f755c0a5ad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f756c000c00 tx=0x7f756c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:32.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.621+0000 7f75677fe640 1 -- 172.21.15.16:0/2214958890 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f756009bc80 con 0x7f7570102d50 2024-03-27T06:41:32.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:32 smithi156 ceph-mon[36261]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-27T06:41:32.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:32 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3227226748' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-27T06:41:32.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:32 smithi156 ceph-mon[36261]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-27T06:41:32.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:32 smithi156 ceph-mon[36261]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T06:41:32.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:32 smithi156 ceph-mon[36261]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-27T06:41:32.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:32 smithi156 ceph-mon[36261]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-27T06:41:32.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:32 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3234631009' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-27T06:41:32.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:32 smithi170 ceph-mon[36438]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-27T06:41:32.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:32 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3227226748' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-27T06:41:32.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:32 smithi170 ceph-mon[36438]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-27T06:41:32.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:32 smithi170 ceph-mon[36438]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T06:41:32.745 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:32 smithi170 ceph-mon[36438]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-27T06:41:32.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:32 smithi170 ceph-mon[36438]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-27T06:41:32.746 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:32 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3234631009' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-27T06:41:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:32.847+0000 7f757859a640 1 -- 172.21.15.16:0/2214958890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f757002d570 con 0x7f7570102d50 2024-03-27T06:41:32.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:32 smithi016 ceph-mon[31908]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-27T06:41:32.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:32 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3227226748' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-27T06:41:32.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:32 smithi016 ceph-mon[31908]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-27T06:41:32.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:32 smithi016 ceph-mon[31908]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-27T06:41:32.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:32 smithi016 ceph-mon[31908]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-27T06:41:32.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:32 smithi016 ceph-mon[31908]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-27T06:41:32.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:32 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3234631009' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-27T06:41:33.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.486+0000 7f75677fe640 1 -- 172.21.15.16:0/2214958890 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f756009b580 con 0x7f7570102d50 2024-03-27T06:41:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 -- 172.21.15.16:0/2214958890 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f755c0a3610 msgr2=0x7f755c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f755c0a3610 0x7f755c0a5ad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f756c000c00 tx=0x7f756c005fd0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 -- 172.21.15.16:0/2214958890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570102d50 msgr2=0x7f7570198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570102d50 0x7f7570198530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f756000b4a0 tx=0x7f756000b960 comp rx=0 tx=0).stop 2024-03-27T06:41:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 -- 172.21.15.16:0/2214958890 shutdown_connections 2024-03-27T06:41:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f755c0a3610 0x7f755c0a5ad0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7570103710 0x7f757019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7570102d50 0x7f7570198530 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 --2- 172.21.15.16:0/2214958890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7570101b60 0x7f7570197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 -- 172.21.15.16:0/2214958890 >> 172.21.15.16:0/2214958890 conn(0x7f75700fd0f0 msgr2=0x7f75701060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 -- 172.21.15.16:0/2214958890 shutdown_connections 2024-03-27T06:41:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.489+0000 7f757859a640 1 -- 172.21.15.16:0/2214958890 wait complete. 2024-03-27T06:41:33.501 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-27T06:41:33.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-27T06:41:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.618+0000 7f7c853bc640 1 -- 172.21.15.16:0/743899870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80102620 msgr2=0x7f7c80108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.618+0000 7f7c853bc640 1 --2- 172.21.15.16:0/743899870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80102620 0x7f7c80108eb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009ca0 tx=0x7f7c7002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.618+0000 7f7c853bc640 1 -- 172.21.15.16:0/743899870 shutdown_connections 2024-03-27T06:41:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.618+0000 7f7c853bc640 1 --2- 172.21.15.16:0/743899870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80102620 0x7f7c80108eb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.618+0000 7f7c853bc640 1 --2- 172.21.15.16:0/743899870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c80101c60 0x7f7c801020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.618+0000 7f7c853bc640 1 --2- 172.21.15.16:0/743899870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80100a70 0x7f7c80100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.618+0000 7f7c853bc640 1 -- 172.21.15.16:0/743899870 >> 172.21.15.16:0/743899870 conn(0x7f7c800fc000 msgr2=0x7f7c800fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.618+0000 7f7c853bc640 1 -- 172.21.15.16:0/743899870 shutdown_connections 2024-03-27T06:41:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.618+0000 7f7c853bc640 1 -- 172.21.15.16:0/743899870 wait complete. 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.618+0000 7f7c853bc640 1 Processor -- start 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c853bc640 1 -- start start 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c853bc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c80100a70 0x7f7c8006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c853bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80101c60 0x7f7c8019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c853bc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80102620 0x7f7c8019f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c853bc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8010c820 con 0x7f7c80101c60 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c853bc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8010c6a0 con 0x7f7c80100a70 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c853bc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8010c9a0 con 0x7f7c80102620 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c7e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80101c60 0x7f7c8019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c7e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80101c60 0x7f7c8019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37460/0 (socket says 172.21.15.16:37460) 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c7effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c80100a70 0x7f7c8006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c7e7fc640 1 -- 172.21.15.16:0/2569653199 learned_addr learned my addr 172.21.15.16:0/2569653199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c7f7fe640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80102620 0x7f7c8019f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c7e7fc640 1 -- 172.21.15.16:0/2569653199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80102620 msgr2=0x7f7c8019f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c7e7fc640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80102620 0x7f7c8019f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c7e7fc640 1 -- 172.21.15.16:0/2569653199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c80100a70 msgr2=0x7f7c8006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c7e7fc640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c80100a70 0x7f7c8006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c7e7fc640 1 -- 172.21.15.16:0/2569653199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8019fb50 con 0x7f7c80101c60 2024-03-27T06:41:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c7e7fc640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80101c60 0x7f7c8019cc20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800e9f0 tx=0x7f7c6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c77fff640 1 -- 172.21.15.16:0/2569653199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6800cd80 con 0x7f7c80101c60 2024-03-27T06:41:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c853bc640 1 -- 172.21.15.16:0/2569653199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8019fde0 con 0x7f7c80101c60 2024-03-27T06:41:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c77fff640 1 -- 172.21.15.16:0/2569653199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7c68010930 con 0x7f7c80101c60 2024-03-27T06:41:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c77fff640 1 -- 172.21.15.16:0/2569653199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68018b00 con 0x7f7c80101c60 2024-03-27T06:41:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.619+0000 7f7c853bc640 1 -- 172.21.15.16:0/2569653199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c801a02a0 con 0x7f7c80101c60 2024-03-27T06:41:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.620+0000 7f7c853bc640 1 -- 172.21.15.16:0/2569653199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c80107d30 con 0x7f7c80101c60 2024-03-27T06:41:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.621+0000 7f7c77fff640 1 -- 172.21.15.16:0/2569653199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f7c68018ca0 con 0x7f7c80101c60 2024-03-27T06:41:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.622+0000 7f7c77fff640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c6c0a3610 0x7f7c6c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.622+0000 7f7c7effd640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c6c0a3610 0x7f7c6c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.622+0000 7f7c77fff640 1 -- 172.21.15.16:0/2569653199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7c680cffd0 con 0x7f7c80101c60 2024-03-27T06:41:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.622+0000 7f7c7effd640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c6c0a3610 0x7f7c6c0a5ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7c80101ac0 tx=0x7f7c78005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:33.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.626+0000 7f7c77fff640 1 -- 172.21.15.16:0/2569653199 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7c6809c600 con 0x7f7c80101c60 2024-03-27T06:41:33.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.837+0000 7f7c853bc640 1 -- 172.21.15.16:0/2569653199 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7c80061f90 con 0x7f7c6c0a3610 2024-03-27T06:41:33.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-mon[31908]: pgmap v292: 97 pgs: 20 creating+peering, 38 active+clean, 39 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 244 B/s wr, 5 op/s 2024-03-27T06:41:33.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3234631009' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-27T06:41:33.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] up:active 2024-03-27T06:41:33.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-mon[31908]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-27T06:41:33.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-mon[31908]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-27T06:41:33.838 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-mon[31908]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-03-27T06:41:33.838 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-mon[31908]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-27T06:41:33.838 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2214958890' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-27T06:41:33.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:33 smithi170 ceph-mon[36438]: pgmap v292: 97 pgs: 20 creating+peering, 38 active+clean, 39 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 244 B/s wr, 5 op/s 2024-03-27T06:41:33.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:33 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3234631009' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-27T06:41:33.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:33 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] up:active 2024-03-27T06:41:33.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:33 smithi170 ceph-mon[36438]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-27T06:41:33.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:33 smithi170 ceph-mon[36438]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-27T06:41:33.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:33 smithi170 ceph-mon[36438]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-03-27T06:41:33.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:33 smithi170 ceph-mon[36438]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-27T06:41:33.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:33 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2214958890' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-27T06:41:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.862+0000 7f7c77fff640 1 -- 172.21.15.16:0/2569653199 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7c80061f90 con 0x7f7c6c0a3610 2024-03-27T06:41:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 -- 172.21.15.16:0/2569653199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c6c0a3610 msgr2=0x7f7c6c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c6c0a3610 0x7f7c6c0a5ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7c80101ac0 tx=0x7f7c78005f90 comp rx=0 tx=0).stop 2024-03-27T06:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 -- 172.21.15.16:0/2569653199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80101c60 msgr2=0x7f7c8019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80101c60 0x7f7c8019cc20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800e9f0 tx=0x7f7c6800eeb0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 -- 172.21.15.16:0/2569653199 shutdown_connections 2024-03-27T06:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f7c6c0a3610 0x7f7c6c0a5ad0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80102620 0x7f7c8019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c80101c60 0x7f7c8019cc20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 --2- 172.21.15.16:0/2569653199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c80100a70 0x7f7c8006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 -- 172.21.15.16:0/2569653199 >> 172.21.15.16:0/2569653199 conn(0x7f7c800fc000 msgr2=0x7f7c80104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 -- 172.21.15.16:0/2569653199 shutdown_connections 2024-03-27T06:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.864+0000 7f7c853bc640 1 -- 172.21.15.16:0/2569653199 wait complete. 2024-03-27T06:41:33.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-27T06:41:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:33 smithi156 ceph-mon[36261]: pgmap v292: 97 pgs: 20 creating+peering, 38 active+clean, 39 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 244 B/s wr, 5 op/s 2024-03-27T06:41:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:33 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3234631009' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-27T06:41:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:33 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] up:active 2024-03-27T06:41:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:33 smithi156 ceph-mon[36261]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-27T06:41:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:33 smithi156 ceph-mon[36261]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-27T06:41:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:33 smithi156 ceph-mon[36261]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-03-27T06:41:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:33 smithi156 ceph-mon[36261]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-27T06:41:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:33 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2214958890' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.983+0000 7f588da0f640 1 -- 172.21.15.16:0/2978798025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5888102d50 msgr2=0x7f58881031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.983+0000 7f588da0f640 1 --2- 172.21.15.16:0/2978798025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5888102d50 0x7f58881031d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f587c009900 tx=0x7f587c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.983+0000 7f588da0f640 1 -- 172.21.15.16:0/2978798025 shutdown_connections 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.983+0000 7f588da0f640 1 --2- 172.21.15.16:0/2978798025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5888103710 0x7f5888109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.983+0000 7f588da0f640 1 --2- 172.21.15.16:0/2978798025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5888102d50 0x7f58881031d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.983+0000 7f588da0f640 1 --2- 172.21.15.16:0/2978798025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5888101b60 0x7f5888101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.983+0000 7f588da0f640 1 -- 172.21.15.16:0/2978798025 >> 172.21.15.16:0/2978798025 conn(0x7f58880fd0f0 msgr2=0x7f58880ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.983+0000 7f588da0f640 1 -- 172.21.15.16:0/2978798025 shutdown_connections 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.983+0000 7f588da0f640 1 -- 172.21.15.16:0/2978798025 wait complete. 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f588da0f640 1 Processor -- start 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f588da0f640 1 -- start start 2024-03-27T06:41:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f588da0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5888101b60 0x7f5888197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f588da0f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5888102d50 0x7f5888198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f588da0f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5888103710 0x7f588819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f588da0f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f588810d8c0 con 0x7f5888101b60 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f588da0f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f588810d740 con 0x7f5888103710 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f588da0f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f588810da40 con 0x7f5888102d50 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f5886ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5888101b60 0x7f5888197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f5886ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5888101b60 0x7f5888197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37498/0 (socket says 172.21.15.16:37498) 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f5886ffd640 1 -- 172.21.15.16:0/262841477 learned_addr learned my addr 172.21.15.16:0/262841477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f58877fe640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5888103710 0x7f588819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f58867fc640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5888102d50 0x7f5888198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f5886ffd640 1 -- 172.21.15.16:0/262841477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5888102d50 msgr2=0x7f5888198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f5886ffd640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5888102d50 0x7f5888198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f5886ffd640 1 -- 172.21.15.16:0/262841477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5888103710 msgr2=0x7f588819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f5886ffd640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5888103710 0x7f588819f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.984+0000 7f5886ffd640 1 -- 172.21.15.16:0/262841477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588819fb70 con 0x7f5888101b60 2024-03-27T06:41:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.985+0000 7f5886ffd640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5888101b60 0x7f5888197f50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5878009fd0 tx=0x7f587800c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.985+0000 7f588ca0d640 1 -- 172.21.15.16:0/262841477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878011470 con 0x7f5888101b60 2024-03-27T06:41:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.985+0000 7f588ca0d640 1 -- 172.21.15.16:0/262841477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5878011b10 con 0x7f5888101b60 2024-03-27T06:41:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.985+0000 7f588da0f640 1 -- 172.21.15.16:0/262841477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588819fe60 con 0x7f5888101b60 2024-03-27T06:41:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.985+0000 7f588ca0d640 1 -- 172.21.15.16:0/262841477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587801acf0 con 0x7f5888101b60 2024-03-27T06:41:33.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.985+0000 7f588da0f640 1 -- 172.21.15.16:0/262841477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58881a02c0 con 0x7f5888101b60 2024-03-27T06:41:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.986+0000 7f588da0f640 1 -- 172.21.15.16:0/262841477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5888108e80 con 0x7f5888101b60 2024-03-27T06:41:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.987+0000 7f588ca0d640 1 -- 172.21.15.16:0/262841477 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7f5878011cc0 con 0x7f5888101b60 2024-03-27T06:41:33.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.987+0000 7f588ca0d640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f58700a3610 0x7f58700a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.987+0000 7f58867fc640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f58700a3610 0x7f58700a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.987+0000 7f588ca0d640 1 -- 172.21.15.16:0/262841477 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f58780d00b0 con 0x7f5888101b60 2024-03-27T06:41:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.988+0000 7f58867fc640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f58700a3610 0x7f58700a5ad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f587c000c00 tx=0x7f587c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:33.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:33.992+0000 7f588ca0d640 1 -- 172.21.15.16:0/262841477 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5878016030 con 0x7f5888101b60 2024-03-27T06:41:34.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:41:33.842+0000 7f9c33f4f700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:41:34.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:41:33.842+0000 7f9c33f4f700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:41:34.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:41:33.842+0000 7f9c33f4f700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:41:34.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:41:33.842+0000 7f9c33f4f700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:41:34.112 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:41:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:41:33.842+0000 7f9c33f4f700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:41:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.207+0000 7f588da0f640 1 -- 172.21.15.16:0/262841477 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5888061f90 con 0x7f58700a3610 2024-03-27T06:41:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.232+0000 7f588ca0d640 1 -- 172.21.15.16:0/262841477 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5888061f90 con 0x7f58700a3610 2024-03-27T06:41:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 -- 172.21.15.16:0/262841477 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f58700a3610 msgr2=0x7f58700a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f58700a3610 0x7f58700a5ad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f587c000c00 tx=0x7f587c005fd0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 -- 172.21.15.16:0/262841477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5888101b60 msgr2=0x7f5888197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5888101b60 0x7f5888197f50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5878009fd0 tx=0x7f587800c270 comp rx=0 tx=0).stop 2024-03-27T06:41:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 -- 172.21.15.16:0/262841477 shutdown_connections 2024-03-27T06:41:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f58700a3610 0x7f58700a5ad0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5888103710 0x7f588819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5888102d50 0x7f5888198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 --2- 172.21.15.16:0/262841477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5888101b60 0x7f5888197f50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 -- 172.21.15.16:0/262841477 >> 172.21.15.16:0/262841477 conn(0x7f58880fd0f0 msgr2=0x7f5888106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 -- 172.21.15.16:0/262841477 shutdown_connections 2024-03-27T06:41:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.234+0000 7f588da0f640 1 -- 172.21.15.16:0/262841477 wait complete. 2024-03-27T06:41:34.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-27T06:41:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.353+0000 7fe1d715c640 1 -- 172.21.15.16:0/774135644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d0102600 msgr2=0x7fe1d0108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.353+0000 7fe1d715c640 1 --2- 172.21.15.16:0/774135644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d0102600 0x7fe1d0108e90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009ca0 tx=0x7fe1c802c100 comp rx=0 tx=0).stop 2024-03-27T06:41:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.353+0000 7fe1d715c640 1 -- 172.21.15.16:0/774135644 shutdown_connections 2024-03-27T06:41:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.353+0000 7fe1d715c640 1 --2- 172.21.15.16:0/774135644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d0102600 0x7fe1d0108e90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.353+0000 7fe1d715c640 1 --2- 172.21.15.16:0/774135644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1d0101c40 0x7fe1d01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.353+0000 7fe1d715c640 1 --2- 172.21.15.16:0/774135644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d0100a50 0x7fe1d0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.353+0000 7fe1d715c640 1 -- 172.21.15.16:0/774135644 >> 172.21.15.16:0/774135644 conn(0x7fe1d00fbfe0 msgr2=0x7fe1d00fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.353+0000 7fe1d715c640 1 -- 172.21.15.16:0/774135644 shutdown_connections 2024-03-27T06:41:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d715c640 1 -- 172.21.15.16:0/774135644 wait complete. 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d715c640 1 Processor -- start 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d715c640 1 -- start start 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d715c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d0100a50 0x7fe1d0197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d715c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1d0101c40 0x7fe1d0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d715c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d0102600 0x7fe1d019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d715c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d010d910 con 0x7fe1d0102600 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d715c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d010d790 con 0x7fe1d0100a50 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d715c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d010da90 con 0x7fe1d0101c40 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d56f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d0102600 0x7fe1d019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d56f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d0102600 0x7fe1d019f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37512/0 (socket says 172.21.15.16:37512) 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d56f9640 1 -- 172.21.15.16:0/1899256772 learned_addr learned my addr 172.21.15.16:0/1899256772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1d4ef8640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d0100a50 0x7fe1d0197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.354+0000 7fe1c7fff640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1d0101c40 0x7fe1d0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1d56f9640 1 -- 172.21.15.16:0/1899256772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1d0101c40 msgr2=0x7fe1d0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1d56f9640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1d0101c40 0x7fe1d0198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1d56f9640 1 -- 172.21.15.16:0/1899256772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d0100a50 msgr2=0x7fe1d0197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1d56f9640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d0100a50 0x7fe1d0197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1d56f9640 1 -- 172.21.15.16:0/1899256772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d019fb40 con 0x7fe1d0102600 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1d4ef8640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d0100a50 0x7fe1d0197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1d56f9640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d0102600 0x7fe1d019f4d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe1c802c710 tx=0x7fe1c80346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1c5ffb640 1 -- 172.21.15.16:0/1899256772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8039040 con 0x7fe1d0102600 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1d715c640 1 -- 172.21.15.16:0/1899256772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d019fdd0 con 0x7fe1d0102600 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1c5ffb640 1 -- 172.21.15.16:0/1899256772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe1c8044070 con 0x7fe1d0102600 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1c5ffb640 1 -- 172.21.15.16:0/1899256772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8004460 con 0x7fe1d0102600 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.355+0000 7fe1d715c640 1 -- 172.21.15.16:0/1899256772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d01a0260 con 0x7fe1d0102600 2024-03-27T06:41:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.356+0000 7fe1d715c640 1 -- 172.21.15.16:0/1899256772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d0107d10 con 0x7fe1d0102600 2024-03-27T06:41:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.357+0000 7fe1c5ffb640 1 -- 172.21.15.16:0/1899256772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138370+0+0 (secure 0 0 0) 0x7fe1c8004600 con 0x7fe1d0102600 2024-03-27T06:41:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.358+0000 7fe1c5ffb640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe1b80a3610 0x7fe1b80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.358+0000 7fe1d4ef8640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe1b80a3610 0x7fe1b80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.358+0000 7fe1c5ffb640 1 -- 172.21.15.16:0/1899256772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe1c80f2850 con 0x7fe1d0102600 2024-03-27T06:41:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.358+0000 7fe1d4ef8640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe1b80a3610 0x7fe1b80a5ad0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0101aa0 tx=0x7fe1cc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:34.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.362+0000 7fe1c5ffb640 1 -- 172.21.15.16:0/1899256772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fe1c80bf4d0 con 0x7fe1d0102600 2024-03-27T06:41:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.506+0000 7fe1c5ffb640 1 -- 172.21.15.16:0/1899256772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fe1c80bb440 con 0x7fe1d0102600 2024-03-27T06:41:34.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.585+0000 7fe1d715c640 1 -- 172.21.15.16:0/1899256772 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fe1d0061f90 con 0x7fe1b80a3610 2024-03-27T06:41:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.590+0000 7fe1c5ffb640 1 -- 172.21.15.16:0/1899256772 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe1d0061f90 con 0x7fe1b80a3610 2024-03-27T06:41:34.585 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_0/5fcea376-87d6-4a4f-9253-cd57a1869eab 2024-03-27T06:41:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.592+0000 7fe1d715c640 1 -- 172.21.15.16:0/1899256772 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe1b80a3610 msgr2=0x7fe1b80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.592+0000 7fe1d715c640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe1b80a3610 0x7fe1b80a5ad0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0101aa0 tx=0x7fe1cc005f90 comp rx=0 tx=0).stop 2024-03-27T06:41:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.592+0000 7fe1d715c640 1 -- 172.21.15.16:0/1899256772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d0102600 msgr2=0x7fe1d019f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.592+0000 7fe1d715c640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d0102600 0x7fe1d019f4d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe1c802c710 tx=0x7fe1c80346f0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.592+0000 7fe1d715c640 1 -- 172.21.15.16:0/1899256772 shutdown_connections 2024-03-27T06:41:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.592+0000 7fe1d715c640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe1b80a3610 0x7fe1b80a5ad0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.592+0000 7fe1d715c640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d0102600 0x7fe1d019f4d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.592+0000 7fe1d715c640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1d0101c40 0x7fe1d0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.592+0000 7fe1d715c640 1 --2- 172.21.15.16:0/1899256772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d0100a50 0x7fe1d0197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.592+0000 7fe1d715c640 1 -- 172.21.15.16:0/1899256772 >> 172.21.15.16:0/1899256772 conn(0x7fe1d00fbfe0 msgr2=0x7fe1d0104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.592+0000 7fe1d715c640 1 -- 172.21.15.16:0/1899256772 shutdown_connections 2024-03-27T06:41:34.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.593+0000 7fe1d715c640 1 -- 172.21.15.16:0/1899256772 wait complete. 2024-03-27T06:41:34.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:34 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2214958890' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-27T06:41:34.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:34 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] up:active 2024-03-27T06:41:34.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:34 smithi016 ceph-mon[31908]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-27T06:41:34.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:34 smithi016 ceph-mon[31908]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-03-27T06:41:34.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:34 smithi016 ceph-mon[31908]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-03-27T06:41:34.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:34 smithi016 ceph-mon[31908]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-27T06:41:34.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-27T06:41:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.714+0000 7fb84f088640 1 -- 172.21.15.16:0/169534331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848102600 msgr2=0x7fb848108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.714+0000 7fb84f088640 1 --2- 172.21.15.16:0/169534331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848102600 0x7fb848108e90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb830009900 tx=0x7fb83002bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.714+0000 7fb84f088640 1 -- 172.21.15.16:0/169534331 shutdown_connections 2024-03-27T06:41:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.714+0000 7fb84f088640 1 --2- 172.21.15.16:0/169534331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848102600 0x7fb848108e90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.714+0000 7fb84f088640 1 --2- 172.21.15.16:0/169534331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848101c40 0x7fb8481020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.714+0000 7fb84f088640 1 --2- 172.21.15.16:0/169534331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb848100a50 0x7fb848100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.714+0000 7fb84f088640 1 -- 172.21.15.16:0/169534331 >> 172.21.15.16:0/169534331 conn(0x7fb8480fbfe0 msgr2=0x7fb8480fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.714+0000 7fb84f088640 1 -- 172.21.15.16:0/169534331 shutdown_connections 2024-03-27T06:41:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.714+0000 7fb84f088640 1 -- 172.21.15.16:0/169534331 wait complete. 2024-03-27T06:41:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84f088640 1 Processor -- start 2024-03-27T06:41:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84f088640 1 -- start start 2024-03-27T06:41:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84f088640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848100a50 0x7fb848069820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84f088640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb848101c40 0x7fb84819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84f088640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848102600 0x7fb84819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84f088640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb84810c7b0 con 0x7fb848102600 2024-03-27T06:41:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84f088640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fb84810c630 con 0x7fb848101c40 2024-03-27T06:41:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84f088640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb84810c930 con 0x7fb848100a50 2024-03-27T06:41:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84d625640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848102600 0x7fb84819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84d625640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848102600 0x7fb84819f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37522/0 (socket says 172.21.15.16:37522) 2024-03-27T06:41:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84d625640 1 -- 172.21.15.16:0/812790670 learned_addr learned my addr 172.21.15.16:0/812790670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84ce24640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848100a50 0x7fb848069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb847fff640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb848101c40 0x7fb84819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84d625640 1 -- 172.21.15.16:0/812790670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848100a50 msgr2=0x7fb848069820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84d625640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848100a50 0x7fb848069820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84d625640 1 -- 172.21.15.16:0/812790670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb848101c40 msgr2=0x7fb84819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84d625640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb848101c40 0x7fb84819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.715+0000 7fb84d625640 1 -- 172.21.15.16:0/812790670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84819fbb0 con 0x7fb848102600 2024-03-27T06:41:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.716+0000 7fb84d625640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848102600 0x7fb84819f2a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb830005ab0 tx=0x7fb830004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.716+0000 7fb845ffb640 1 -- 172.21.15.16:0/812790670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830043020 con 0x7fb848102600 2024-03-27T06:41:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.716+0000 7fb845ffb640 1 -- 172.21.15.16:0/812790670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb8300042d0 con 0x7fb848102600 2024-03-27T06:41:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.716+0000 7fb84f088640 1 -- 172.21.15.16:0/812790670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84819fde0 con 0x7fb848102600 2024-03-27T06:41:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.716+0000 7fb845ffb640 1 -- 172.21.15.16:0/812790670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830003c50 con 0x7fb848102600 2024-03-27T06:41:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.716+0000 7fb84f088640 1 -- 172.21.15.16:0/812790670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8481a02a0 con 0x7fb848102600 2024-03-27T06:41:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.717+0000 7fb84f088640 1 -- 172.21.15.16:0/812790670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb848107d70 con 0x7fb848102600 2024-03-27T06:41:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.718+0000 7fb845ffb640 1 -- 172.21.15.16:0/812790670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fb830046020 con 0x7fb848102600 2024-03-27T06:41:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.718+0000 7fb845ffb640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb8340a36a0 0x7fb8340a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.719+0000 7fb84ce24640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb8340a36a0 0x7fb8340a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.719+0000 7fb845ffb640 1 -- 172.21.15.16:0/812790670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb8300f1df0 con 0x7fb848102600 2024-03-27T06:41:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.719+0000 7fb84ce24640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb8340a36a0 0x7fb8340a5b60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb848101aa0 tx=0x7fb840009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.723+0000 7fb845ffb640 1 -- 172.21.15.16:0/812790670 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb83004d020 con 0x7fb848102600 2024-03-27T06:41:34.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:34 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2214958890' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-27T06:41:34.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:34 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] up:active 2024-03-27T06:41:34.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:34 smithi170 ceph-mon[36438]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-27T06:41:34.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:34 smithi170 ceph-mon[36438]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-03-27T06:41:34.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:34 smithi170 ceph-mon[36438]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-03-27T06:41:34.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:34 smithi170 ceph-mon[36438]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-27T06:41:34.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.935+0000 7fb84f088640 1 -- 172.21.15.16:0/812790670 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb848062020 con 0x7fb8340a36a0 2024-03-27T06:41:34.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.958+0000 7fb845ffb640 1 -- 172.21.15.16:0/812790670 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb848062020 con 0x7fb8340a36a0 2024-03-27T06:41:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 -- 172.21.15.16:0/812790670 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb8340a36a0 msgr2=0x7fb8340a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb8340a36a0 0x7fb8340a5b60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb848101aa0 tx=0x7fb840009500 comp rx=0 tx=0).stop 2024-03-27T06:41:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 -- 172.21.15.16:0/812790670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848102600 msgr2=0x7fb84819f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848102600 0x7fb84819f2a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb830005ab0 tx=0x7fb830004ad0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 -- 172.21.15.16:0/812790670 shutdown_connections 2024-03-27T06:41:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fb8340a36a0 0x7fb8340a5b60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb848102600 0x7fb84819f2a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb848101c40 0x7fb84819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 --2- 172.21.15.16:0/812790670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb848100a50 0x7fb848069820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 -- 172.21.15.16:0/812790670 >> 172.21.15.16:0/812790670 conn(0x7fb8480fbfe0 msgr2=0x7fb848104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 -- 172.21.15.16:0/812790670 shutdown_connections 2024-03-27T06:41:34.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:34.960+0000 7fb84f088640 1 -- 172.21.15.16:0/812790670 wait complete. 2024-03-27T06:41:34.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-27T06:41:34.971 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b[66776]: 2024-03-27T06:41:34.494+0000 7ff65d639700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-27T06:41:34.972 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b[66776]: 2024-03-27T06:41:34.494+0000 7ff65d639700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-27T06:41:34.972 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b[66776]: 2024-03-27T06:41:34.494+0000 7ff65d639700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-27T06:41:34.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:34 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2214958890' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-27T06:41:34.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:34 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] up:active 2024-03-27T06:41:34.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:34 smithi156 ceph-mon[36261]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-27T06:41:34.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:34 smithi156 ceph-mon[36261]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-03-27T06:41:34.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:34 smithi156 ceph-mon[36261]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-03-27T06:41:34.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:34 smithi156 ceph-mon[36261]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.078+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3423250123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b94101b70 msgr2=0x7f5b94101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.078+0000 7f5b9ac88640 1 --2- 172.21.15.16:0/3423250123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b94101b70 0x7f5b94101f90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009900 tx=0x7f5b8802bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.078+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3423250123 shutdown_connections 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.078+0000 7f5b9ac88640 1 --2- 172.21.15.16:0/3423250123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b94103720 0x7f5b94109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.078+0000 7f5b9ac88640 1 --2- 172.21.15.16:0/3423250123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b94102d60 0x7f5b941031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.078+0000 7f5b9ac88640 1 --2- 172.21.15.16:0/3423250123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b94101b70 0x7f5b94101f90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.078+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3423250123 >> 172.21.15.16:0/3423250123 conn(0x7f5b940fd0e0 msgr2=0x7f5b940ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.078+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3423250123 shutdown_connections 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.078+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3423250123 wait complete. 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b9ac88640 1 Processor -- start 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b9ac88640 1 -- start start 2024-03-27T06:41:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b9ac88640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b94101b70 0x7f5b94197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b9ac88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b94102d60 0x7f5b941984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b9ac88640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b94103720 0x7f5b9419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b9ac88640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9410d8d0 con 0x7f5b94101b70 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b9ac88640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9410d750 con 0x7f5b94103720 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b9ac88640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9410da50 con 0x7f5b94102d60 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b98a24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b94101b70 0x7f5b94197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b98a24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b94101b70 0x7f5b94197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37538/0 (socket says 172.21.15.16:37538) 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b98a24640 1 -- 172.21.15.16:0/3361085961 learned_addr learned my addr 172.21.15.16:0/3361085961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b93fff640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b94102d60 0x7f5b941984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b99225640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b94103720 0x7f5b9419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b98a24640 1 -- 172.21.15.16:0/3361085961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b94102d60 msgr2=0x7f5b941984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b98a24640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b94102d60 0x7f5b941984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b98a24640 1 -- 172.21.15.16:0/3361085961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b94103720 msgr2=0x7f5b9419f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b98a24640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b94103720 0x7f5b9419f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b98a24640 1 -- 172.21.15.16:0/3361085961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9419fb90 con 0x7f5b94101b70 2024-03-27T06:41:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.079+0000 7f5b98a24640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b94101b70 0x7f5b94197f70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5b88000c00 tx=0x7f5b88004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.080+0000 7f5b91ffb640 1 -- 172.21.15.16:0/3361085961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88043020 con 0x7f5b94101b70 2024-03-27T06:41:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.080+0000 7f5b91ffb640 1 -- 172.21.15.16:0/3361085961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b88004250 con 0x7f5b94101b70 2024-03-27T06:41:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.080+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3361085961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9419fe20 con 0x7f5b94101b70 2024-03-27T06:41:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.080+0000 7f5b91ffb640 1 -- 172.21.15.16:0/3361085961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88003c00 con 0x7f5b94101b70 2024-03-27T06:41:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.080+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3361085961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b941a0280 con 0x7f5b94101b70 2024-03-27T06:41:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.081+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3361085961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b94108eb0 con 0x7f5b94101b70 2024-03-27T06:41:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.082+0000 7f5b91ffb640 1 -- 172.21.15.16:0/3361085961 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f5b88046020 con 0x7f5b94101b70 2024-03-27T06:41:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.082+0000 7f5b91ffb640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5b840a36a0 0x7f5b840a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.083+0000 7f5b93fff640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5b840a36a0 0x7f5b840a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.083+0000 7f5b91ffb640 1 -- 172.21.15.16:0/3361085961 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5b880f21c0 con 0x7f5b94101b70 2024-03-27T06:41:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.083+0000 7f5b93fff640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5b840a36a0 0x7f5b840a5b60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5b94199480 tx=0x7f5b80005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:35.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.086+0000 7f5b91ffb640 1 -- 172.21.15.16:0/3361085961 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5b8804d020 con 0x7f5b94101b70 2024-03-27T06:41:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.300+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3361085961 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5b94062020 con 0x7f5b840a36a0 2024-03-27T06:41:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.305+0000 7f5b91ffb640 1 -- 172.21.15.16:0/3361085961 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5b94062020 con 0x7f5b840a36a0 2024-03-27T06:41:35.300 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a 2024-03-27T06:41:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3361085961 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5b840a36a0 msgr2=0x7f5b840a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5b840a36a0 0x7f5b840a5b60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5b94199480 tx=0x7f5b80005f90 comp rx=0 tx=0).stop 2024-03-27T06:41:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3361085961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b94101b70 msgr2=0x7f5b94197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b94101b70 0x7f5b94197f70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5b88000c00 tx=0x7f5b88004a30 comp rx=0 tx=0).stop 2024-03-27T06:41:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3361085961 shutdown_connections 2024-03-27T06:41:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5b840a36a0 0x7f5b840a5b60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b94103720 0x7f5b9419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b94102d60 0x7f5b941984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 --2- 172.21.15.16:0/3361085961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b94101b70 0x7f5b94197f70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3361085961 >> 172.21.15.16:0/3361085961 conn(0x7f5b940fd0e0 msgr2=0x7f5b94106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3361085961 shutdown_connections 2024-03-27T06:41:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.307+0000 7f5b9ac88640 1 -- 172.21.15.16:0/3361085961 wait complete. 2024-03-27T06:41:35.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:41:35.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.422+0000 7f77a2ff6640 1 -- 172.21.15.16:0/397393242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c1026a0 msgr2=0x7f779c108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.422+0000 7f77a2ff6640 1 --2- 172.21.15.16:0/397393242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c1026a0 0x7f779c108f30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7794009ca0 tx=0x7f779402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.422+0000 7f77a2ff6640 1 -- 172.21.15.16:0/397393242 shutdown_connections 2024-03-27T06:41:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.422+0000 7f77a2ff6640 1 --2- 172.21.15.16:0/397393242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c1026a0 0x7f779c108f30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.422+0000 7f77a2ff6640 1 --2- 172.21.15.16:0/397393242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f779c101ce0 0x7f779c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.422+0000 7f77a2ff6640 1 --2- 172.21.15.16:0/397393242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f779c100af0 0x7f779c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.422+0000 7f77a2ff6640 1 -- 172.21.15.16:0/397393242 >> 172.21.15.16:0/397393242 conn(0x7f779c0fc080 msgr2=0x7f779c0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.422+0000 7f77a2ff6640 1 -- 172.21.15.16:0/397393242 shutdown_connections 2024-03-27T06:41:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.422+0000 7f77a2ff6640 1 -- 172.21.15.16:0/397393242 wait complete. 2024-03-27T06:41:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a2ff6640 1 Processor -- start 2024-03-27T06:41:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a2ff6640 1 -- start start 2024-03-27T06:41:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a2ff6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c100af0 0x7f779c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a2ff6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f779c101ce0 0x7f779c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a2ff6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f779c1026a0 0x7f779c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a2ff6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f779c10c820 con 0x7f779c100af0 2024-03-27T06:41:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a2ff6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f779c10c6a0 con 0x7f779c101ce0 2024-03-27T06:41:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a2ff6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f779c10c9a0 con 0x7f779c1026a0 2024-03-27T06:41:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a0d92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c100af0 0x7f779c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a0d92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c100af0 0x7f779c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37560/0 (socket says 172.21.15.16:37560) 2024-03-27T06:41:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a0d92640 1 -- 172.21.15.16:0/747067189 learned_addr learned my addr 172.21.15.16:0/747067189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f779bfff640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f779c101ce0 0x7f779c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a1593640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f779c1026a0 0x7f779c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a0d92640 1 -- 172.21.15.16:0/747067189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f779c1026a0 msgr2=0x7f779c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a0d92640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f779c1026a0 0x7f779c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a0d92640 1 -- 172.21.15.16:0/747067189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f779c101ce0 msgr2=0x7f779c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a0d92640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f779c101ce0 0x7f779c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.423+0000 7f77a0d92640 1 -- 172.21.15.16:0/747067189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779c19faf0 con 0x7f779c100af0 2024-03-27T06:41:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.424+0000 7f77a0d92640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c100af0 0x7f779c197ed0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f779000e9c0 tx=0x7f779000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.424+0000 7f7799ffb640 1 -- 172.21.15.16:0/747067189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779000cde0 con 0x7f779c100af0 2024-03-27T06:41:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.424+0000 7f77a2ff6640 1 -- 172.21.15.16:0/747067189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779c19fde0 con 0x7f779c100af0 2024-03-27T06:41:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.424+0000 7f7799ffb640 1 -- 172.21.15.16:0/747067189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7790010930 con 0x7f779c100af0 2024-03-27T06:41:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.424+0000 7f7799ffb640 1 -- 172.21.15.16:0/747067189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790018a60 con 0x7f779c100af0 2024-03-27T06:41:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.424+0000 7f77a2ff6640 1 -- 172.21.15.16:0/747067189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779c1a0240 con 0x7f779c100af0 2024-03-27T06:41:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.425+0000 7f77a2ff6640 1 -- 172.21.15.16:0/747067189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f779c1a04a0 con 0x7f779c100af0 2024-03-27T06:41:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.426+0000 7f7799ffb640 1 -- 172.21.15.16:0/747067189 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f779001f020 con 0x7f779c100af0 2024-03-27T06:41:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.426+0000 7f7799ffb640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f778c0a36a0 0x7f778c0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.427+0000 7f779bfff640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f778c0a36a0 0x7f778c0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.427+0000 7f7799ffb640 1 -- 172.21.15.16:0/747067189 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f77900cfeb0 con 0x7f779c100af0 2024-03-27T06:41:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.427+0000 7f779bfff640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f778c0a36a0 0x7f778c0a5b60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f779c1993e0 tx=0x7f7788005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.431+0000 7f7799ffb640 1 -- 172.21.15.16:0/747067189 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f779009cad0 con 0x7f779c100af0 2024-03-27T06:41:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: pgmap v293: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s wr, 14 op/s 2024-03-27T06:41:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:35.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:35.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] up:active 2024-03-27T06:41:35.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-27T06:41:35.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-27T06:41:35.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: Cluster is now healthy 2024-03-27T06:41:35.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-03-27T06:41:35.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-27T06:41:35.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-27T06:41:35.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-03-27T06:41:35.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] up:active 2024-03-27T06:41:35.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:35 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:41:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.664+0000 7f77a2ff6640 1 -- 172.21.15.16:0/747067189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f779c1a0670 con 0x7f779c100af0 2024-03-27T06:41:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.665+0000 7f7799ffb640 1 -- 172.21.15.16:0/747067189 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7f779009c3d0 con 0x7f779c100af0 2024-03-27T06:41:35.659 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:35.660 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:41:35.466274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24472,24493,24500,24527,24467]},"id":1}]} 2024-03-27T06:41:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-03-27T06:41:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 -- 172.21.15.16:0/747067189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f778c0a36a0 msgr2=0x7f778c0a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f778c0a36a0 0x7f778c0a5b60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f779c1993e0 tx=0x7f7788005f90 comp rx=0 tx=0).stop 2024-03-27T06:41:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 -- 172.21.15.16:0/747067189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c100af0 msgr2=0x7f779c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c100af0 0x7f779c197ed0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f779000e9c0 tx=0x7f779000ee80 comp rx=0 tx=0).stop 2024-03-27T06:41:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 -- 172.21.15.16:0/747067189 shutdown_connections 2024-03-27T06:41:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f778c0a36a0 0x7f778c0a5b60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f779c1026a0 0x7f779c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f779c101ce0 0x7f779c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 --2- 172.21.15.16:0/747067189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c100af0 0x7f779c197ed0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 -- 172.21.15.16:0/747067189 >> 172.21.15.16:0/747067189 conn(0x7f779c0fc080 msgr2=0x7f779c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 -- 172.21.15.16:0/747067189 shutdown_connections 2024-03-27T06:41:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.667+0000 7f77a2ff6640 1 -- 172.21.15.16:0/747067189 wait complete. 2024-03-27T06:41:35.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:41:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.781+0000 7f1ef6af4640 1 -- 172.21.15.16:0/1011481772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ef0101b70 msgr2=0x7f1ef0101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.781+0000 7f1ef6af4640 1 --2- 172.21.15.16:0/1011481772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ef0101b70 0x7f1ef0101f90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009be0 tx=0x7f1ee802bef0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.781+0000 7f1ef6af4640 1 -- 172.21.15.16:0/1011481772 shutdown_connections 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.781+0000 7f1ef6af4640 1 --2- 172.21.15.16:0/1011481772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef0103720 0x7f1ef0109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.781+0000 7f1ef6af4640 1 --2- 172.21.15.16:0/1011481772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef0102d60 0x7f1ef01031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.781+0000 7f1ef6af4640 1 --2- 172.21.15.16:0/1011481772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ef0101b70 0x7f1ef0101f90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.781+0000 7f1ef6af4640 1 -- 172.21.15.16:0/1011481772 >> 172.21.15.16:0/1011481772 conn(0x7f1ef00fd100 msgr2=0x7f1ef00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.781+0000 7f1ef6af4640 1 -- 172.21.15.16:0/1011481772 shutdown_connections 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.781+0000 7f1ef6af4640 1 -- 172.21.15.16:0/1011481772 wait complete. 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.781+0000 7f1ef6af4640 1 Processor -- start 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.781+0000 7f1ef6af4640 1 -- start start 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef6af4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef0101b70 0x7f1ef0197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef6af4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef0102d60 0x7f1ef01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef6af4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ef0103720 0x7f1ef019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef6af4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef010d920 con 0x7f1ef0101b70 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef6af4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef010d7a0 con 0x7f1ef0102d60 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef6af4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef010daa0 con 0x7f1ef0103720 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef4890640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef0101b70 0x7f1ef0197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef4890640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef0101b70 0x7f1ef0197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50776/0 (socket says 172.21.15.16:50776) 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef4890640 1 -- 172.21.15.16:0/634948958 learned_addr learned my addr 172.21.15.16:0/634948958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1eeffff640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef0102d60 0x7f1ef01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef5091640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ef0103720 0x7f1ef019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef4890640 1 -- 172.21.15.16:0/634948958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ef0103720 msgr2=0x7f1ef019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef4890640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ef0103720 0x7f1ef019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef4890640 1 -- 172.21.15.16:0/634948958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef0102d60 msgr2=0x7f1ef01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef4890640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef0102d60 0x7f1ef01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef4890640 1 -- 172.21.15.16:0/634948958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef019fbb0 con 0x7f1ef0101b70 2024-03-27T06:41:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef4890640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef0101b70 0x7f1ef0197f90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8000f80 tx=0x7f1ee80338a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1eedffb640 1 -- 172.21.15.16:0/634948958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8043020 con 0x7f1ef0101b70 2024-03-27T06:41:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1ef6af4640 1 -- 172.21.15.16:0/634948958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef019fe40 con 0x7f1ef0101b70 2024-03-27T06:41:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1eedffb640 1 -- 172.21.15.16:0/634948958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1ee802ca80 con 0x7f1ef0101b70 2024-03-27T06:41:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.782+0000 7f1eedffb640 1 -- 172.21.15.16:0/634948958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee803e740 con 0x7f1ef0101b70 2024-03-27T06:41:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.783+0000 7f1ef6af4640 1 -- 172.21.15.16:0/634948958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef01a02b0 con 0x7f1ef0101b70 2024-03-27T06:41:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.784+0000 7f1ef6af4640 1 -- 172.21.15.16:0/634948958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef0108e90 con 0x7f1ef0101b70 2024-03-27T06:41:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.785+0000 7f1eedffb640 1 -- 172.21.15.16:0/634948958 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f1ee8046020 con 0x7f1ef0101b70 2024-03-27T06:41:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.785+0000 7f1eedffb640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1ed80a36a0 0x7f1ed80a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.785+0000 7f1eeffff640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1ed80a36a0 0x7f1ed80a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.785+0000 7f1eedffb640 1 -- 172.21.15.16:0/634948958 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f1ee80f22f0 con 0x7f1ef0101b70 2024-03-27T06:41:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.786+0000 7f1eeffff640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1ed80a36a0 0x7f1ed80a5b60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ef01994a0 tx=0x7f1ee4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.790+0000 7f1eedffb640 1 -- 172.21.15.16:0/634948958 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f1ee804d020 con 0x7f1ef0101b70 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: pgmap v293: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s wr, 14 op/s 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] up:active 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: Cluster is now healthy 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-03-27T06:41:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] up:active 2024-03-27T06:41:35.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:35 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:41:35.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.976+0000 7f1ef6af4640 1 -- 172.21.15.16:0/634948958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ef002d5c0 con 0x7f1ef0101b70 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: pgmap v293: 97 pgs: 8 creating+peering, 89 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s wr, 14 op/s 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] up:active 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: Cluster is now healthy 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-03-27T06:41:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] up:active 2024-03-27T06:41:35.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:35 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:41:35.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.977+0000 7f1eedffb640 1 -- 172.21.15.16:0/634948958 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22173 (secure 0 0 0) 0x7f1ee80bf020 con 0x7f1ef0101b70 2024-03-27T06:41:35.974 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:35.974 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:41:30.382838+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:41:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.979+0000 7f1ef6af4640 1 -- 172.21.15.16:0/634948958 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1ed80a36a0 msgr2=0x7f1ed80a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.979+0000 7f1ef6af4640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1ed80a36a0 0x7f1ed80a5b60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ef01994a0 tx=0x7f1ee4005f90 comp rx=0 tx=0).stop 2024-03-27T06:41:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.979+0000 7f1ef6af4640 1 -- 172.21.15.16:0/634948958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef0101b70 msgr2=0x7f1ef0197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.979+0000 7f1ef6af4640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef0101b70 0x7f1ef0197f90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8000f80 tx=0x7f1ee80338a0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.979+0000 7f1ef6af4640 1 -- 172.21.15.16:0/634948958 shutdown_connections 2024-03-27T06:41:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.979+0000 7f1ef6af4640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f1ed80a36a0 0x7f1ed80a5b60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.979+0000 7f1ef6af4640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ef0103720 0x7f1ef019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.979+0000 7f1ef6af4640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef0102d60 0x7f1ef01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.979+0000 7f1ef6af4640 1 --2- 172.21.15.16:0/634948958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef0101b70 0x7f1ef0197f90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:35.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.979+0000 7f1ef6af4640 1 -- 172.21.15.16:0/634948958 >> 172.21.15.16:0/634948958 conn(0x7f1ef00fd100 msgr2=0x7f1ef0106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.980+0000 7f1ef6af4640 1 -- 172.21.15.16:0/634948958 shutdown_connections 2024-03-27T06:41:35.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:35.980+0000 7f1ef6af4640 1 -- 172.21.15.16:0/634948958 wait complete. 2024-03-27T06:41:35.985 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-27T06:41:35.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T06:41:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.094+0000 7f4c3050f640 1 -- 172.21.15.16:0/1756867033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c28104fa0 msgr2=0x7f4c28107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.094+0000 7f4c3050f640 1 --2- 172.21.15.16:0/1756867033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c28104fa0 0x7f4c28107390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4c18009900 tx=0x7f4c1802bc30 comp rx=0 tx=0).stop 2024-03-27T06:41:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.094+0000 7f4c3050f640 1 -- 172.21.15.16:0/1756867033 shutdown_connections 2024-03-27T06:41:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.094+0000 7f4c3050f640 1 --2- 172.21.15.16:0/1756867033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c281078d0 0x7f4c28109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.094+0000 7f4c3050f640 1 --2- 172.21.15.16:0/1756867033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c28104fa0 0x7f4c28107390 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.094+0000 7f4c3050f640 1 --2- 172.21.15.16:0/1756867033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c28102670 0x7f4c28104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.094+0000 7f4c3050f640 1 -- 172.21.15.16:0/1756867033 >> 172.21.15.16:0/1756867033 conn(0x7f4c280fc080 msgr2=0x7f4c280fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.094+0000 7f4c3050f640 1 -- 172.21.15.16:0/1756867033 shutdown_connections 2024-03-27T06:41:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.094+0000 7f4c3050f640 1 -- 172.21.15.16:0/1756867033 wait complete. 2024-03-27T06:41:36.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c3050f640 1 Processor -- start 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c3050f640 1 -- start start 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c3050f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c28102670 0x7f4c28197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c3050f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c28104fa0 0x7f4c281984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c3050f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c281078d0 0x7f4c2819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c3050f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2810d8e0 con 0x7f4c28104fa0 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c3050f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2810d760 con 0x7f4c28102670 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c3050f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2810da60 con 0x7f4c281078d0 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c2daaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c28104fa0 0x7f4c281984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c2daaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c28104fa0 0x7f4c281984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50800/0 (socket says 172.21.15.16:50800) 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c2daaa640 1 -- 172.21.15.16:0/457807040 learned_addr learned my addr 172.21.15.16:0/457807040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c2e2ab640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c28102670 0x7f4c28197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:36.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c2eaac640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c281078d0 0x7f4c2819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c2daaa640 1 -- 172.21.15.16:0/457807040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c281078d0 msgr2=0x7f4c2819f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c2daaa640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c281078d0 0x7f4c2819f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c2daaa640 1 -- 172.21.15.16:0/457807040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c28102670 msgr2=0x7f4c28197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c2daaa640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c28102670 0x7f4c28197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c2daaa640 1 -- 172.21.15.16:0/457807040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2819fba0 con 0x7f4c28104fa0 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c2daaa640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c28104fa0 0x7f4c281984c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4c18000c00 tx=0x7f4c18004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c1f7fe640 1 -- 172.21.15.16:0/457807040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18043020 con 0x7f4c28104fa0 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c1f7fe640 1 -- 172.21.15.16:0/457807040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4c18004250 con 0x7f4c28104fa0 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c3050f640 1 -- 172.21.15.16:0/457807040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2819fe30 con 0x7f4c28104fa0 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.095+0000 7f4c1f7fe640 1 -- 172.21.15.16:0/457807040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18003b90 con 0x7f4c28104fa0 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.096+0000 7f4c3050f640 1 -- 172.21.15.16:0/457807040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c281a02f0 con 0x7f4c28104fa0 2024-03-27T06:41:36.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.097+0000 7f4c3050f640 1 -- 172.21.15.16:0/457807040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c280fdb70 con 0x7f4c28104fa0 2024-03-27T06:41:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.098+0000 7f4c1f7fe640 1 -- 172.21.15.16:0/457807040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f4c18046020 con 0x7f4c28104fa0 2024-03-27T06:41:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.098+0000 7f4c1f7fe640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4c140a36a0 0x7f4c140a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.098+0000 7f4c2e2ab640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4c140a36a0 0x7f4c140a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:36.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.098+0000 7f4c1f7fe640 1 -- 172.21.15.16:0/457807040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4c180f1ca0 con 0x7f4c28104fa0 2024-03-27T06:41:36.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.099+0000 7f4c2e2ab640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4c140a36a0 0x7f4c140a5b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009bd0 tx=0x7f4c24005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.103+0000 7f4c1f7fe640 1 -- 172.21.15.16:0/457807040 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4c180f6070 con 0x7f4c28104fa0 2024-03-27T06:41:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.285+0000 7f4c3050f640 1 -- 172.21.15.16:0/457807040 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c28061f90 con 0x7f4c140a36a0 2024-03-27T06:41:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.293+0000 7f4c1f7fe640 1 -- 172.21.15.16:0/457807040 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258915 (secure 0 0 0) 0x7f4c28061f90 con 0x7f4c140a36a0 2024-03-27T06:41:36.291 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-03-27T06:41:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 -- 172.21.15.16:0/457807040 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4c140a36a0 msgr2=0x7f4c140a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4c140a36a0 0x7f4c140a5b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009bd0 tx=0x7f4c24005f90 comp rx=0 tx=0).stop 2024-03-27T06:41:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 -- 172.21.15.16:0/457807040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c28104fa0 msgr2=0x7f4c281984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c28104fa0 0x7f4c281984c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4c18000c00 tx=0x7f4c18004a30 comp rx=0 tx=0).stop 2024-03-27T06:41:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 -- 172.21.15.16:0/457807040 shutdown_connections 2024-03-27T06:41:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4c140a36a0 0x7f4c140a5b60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c281078d0 0x7f4c2819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c28104fa0 0x7f4c281984c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 --2- 172.21.15.16:0/457807040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c28102670 0x7f4c28197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 -- 172.21.15.16:0/457807040 >> 172.21.15.16:0/457807040 conn(0x7f4c280fc080 msgr2=0x7f4c28102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 -- 172.21.15.16:0/457807040 shutdown_connections 2024-03-27T06:41:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.296+0000 7f4c3050f640 1 -- 172.21.15.16:0/457807040 wait complete. 2024-03-27T06:41:36.301 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-03-27T06:41:35.562261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595646,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":320,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":251,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364524,"kb_used_data":11172,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577300,"statfs":{"total":1151940427776,"available":1151567155200,"internally_reserved":0,"allocated":11440128,"data_stored":7085760,"data_compressed":106419,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5278,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.180868"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499787+0000","last_change":"2024-03-27T06:41:29.408953+0000","last_active":"2024-03-27T06:41:30.499787+0000","last_peered":"2024-03-27T06:41:30.499787+0000","last_clean":"2024-03-27T06:41:30.499787+0000","last_became_active":"2024-03-27T06:41:29.408143+0000","last_became_peered":"2024-03-27T06:41:29.408143+0000","last_unstale":"2024-03-27T06:41:30.499787+0000","last_undegraded":"2024-03-27T06:41:30.499787+0000","last_fullsized":"2024-03-27T06:41:30.499787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:22.486951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486607+0000","last_change":"2024-03-27T06:41:29.405080+0000","last_active":"2024-03-27T06:41:31.486607+0000","last_peered":"2024-03-27T06:41:31.486607+0000","last_clean":"2024-03-27T06:41:31.486607+0000","last_became_active":"2024-03-27T06:41:29.404743+0000","last_became_peered":"2024-03-27T06:41:29.404743+0000","last_unstale":"2024-03-27T06:41:31.486607+0000","last_undegraded":"2024-03-27T06:41:31.486607+0000","last_fullsized":"2024-03-27T06:41:31.486607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:24:30.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486810+0000","last_change":"2024-03-27T06:41:29.409346+0000","last_active":"2024-03-27T06:41:31.486810+0000","last_peered":"2024-03-27T06:41:31.486810+0000","last_clean":"2024-03-27T06:41:31.486810+0000","last_became_active":"2024-03-27T06:41:29.408837+0000","last_became_peered":"2024-03-27T06:41:29.408837+0000","last_unstale":"2024-03-27T06:41:31.486810+0000","last_undegraded":"2024-03-27T06:41:31.486810+0000","last_fullsized":"2024-03-27T06:41:31.486810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:11:12.274869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386987+0000","last_change":"2024-03-27T06:41:29.412115+0000","last_active":"2024-03-27T06:41:30.386987+0000","last_peered":"2024-03-27T06:41:30.386987+0000","last_clean":"2024-03-27T06:41:30.386987+0000","last_became_active":"2024-03-27T06:41:29.411702+0000","last_became_peered":"2024-03-27T06:41:29.411702+0000","last_unstale":"2024-03-27T06:41:30.386987+0000","last_undegraded":"2024-03-27T06:41:30.386987+0000","last_fullsized":"2024-03-27T06:41:30.386987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:04:42.347229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.996006+0000","last_change":"2024-03-27T06:41:29.409888+0000","last_active":"2024-03-27T06:41:30.996006+0000","last_peered":"2024-03-27T06:41:30.996006+0000","last_clean":"2024-03-27T06:41:30.996006+0000","last_became_active":"2024-03-27T06:41:29.409558+0000","last_became_peered":"2024-03-27T06:41:29.409558+0000","last_unstale":"2024-03-27T06:41:30.996006+0000","last_undegraded":"2024-03-27T06:41:30.996006+0000","last_fullsized":"2024-03-27T06:41:30.996006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:38:51.452793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485103+0000","last_change":"2024-03-27T06:41:29.404088+0000","last_active":"2024-03-27T06:41:31.485103+0000","last_peered":"2024-03-27T06:41:31.485103+0000","last_clean":"2024-03-27T06:41:31.485103+0000","last_became_active":"2024-03-27T06:41:29.403698+0000","last_became_peered":"2024-03-27T06:41:29.403698+0000","last_unstale":"2024-03-27T06:41:31.485103+0000","last_undegraded":"2024-03-27T06:41:31.485103+0000","last_fullsized":"2024-03-27T06:41:31.485103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:14:31.296523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.446893+0000","last_change":"2024-03-27T06:41:29.413157+0000","last_active":"2024-03-27T06:41:31.446893+0000","last_peered":"2024-03-27T06:41:31.446893+0000","last_clean":"2024-03-27T06:41:31.446893+0000","last_became_active":"2024-03-27T06:41:29.412669+0000","last_became_peered":"2024-03-27T06:41:29.412669+0000","last_unstale":"2024-03-27T06:41:31.446893+0000","last_undegraded":"2024-03-27T06:41:31.446893+0000","last_fullsized":"2024-03-27T06:41:31.446893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:28:20.823770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.385741+0000","last_change":"2024-03-27T06:41:29.417207+0000","last_active":"2024-03-27T06:41:30.385741+0000","last_peered":"2024-03-27T06:41:30.385741+0000","last_clean":"2024-03-27T06:41:30.385741+0000","last_became_active":"2024-03-27T06:41:29.415556+0000","last_became_peered":"2024-03-27T06:41:29.415556+0000","last_unstale":"2024-03-27T06:41:30.385741+0000","last_undegraded":"2024-03-27T06:41:30.385741+0000","last_fullsized":"2024-03-27T06:41:30.385741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:26:04.040296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486112+0000","last_change":"2024-03-27T06:41:29.400032+0000","last_active":"2024-03-27T06:41:31.486112+0000","last_peered":"2024-03-27T06:41:31.486112+0000","last_clean":"2024-03-27T06:41:31.486112+0000","last_became_active":"2024-03-27T06:41:29.399274+0000","last_became_peered":"2024-03-27T06:41:29.399274+0000","last_unstale":"2024-03-27T06:41:31.486112+0000","last_undegraded":"2024-03-27T06:41:31.486112+0000","last_fullsized":"2024-03-27T06:41:31.486112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:55.306309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485371+0000","last_change":"2024-03-27T06:41:29.412471+0000","last_active":"2024-03-27T06:41:31.485371+0000","last_peered":"2024-03-27T06:41:31.485371+0000","last_clean":"2024-03-27T06:41:31.485371+0000","last_became_active":"2024-03-27T06:41:29.412134+0000","last_became_peered":"2024-03-27T06:41:29.412134+0000","last_unstale":"2024-03-27T06:41:31.485371+0000","last_undegraded":"2024-03-27T06:41:31.485371+0000","last_fullsized":"2024-03-27T06:41:31.485371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:35:57.146944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447274+0000","last_change":"2024-03-27T06:41:29.407915+0000","last_active":"2024-03-27T06:41:31.447274+0000","last_peered":"2024-03-27T06:41:31.447274+0000","last_clean":"2024-03-27T06:41:31.447274+0000","last_became_active":"2024-03-27T06:41:29.407563+0000","last_became_peered":"2024-03-27T06:41:29.407563+0000","last_unstale":"2024-03-27T06:41:31.447274+0000","last_undegraded":"2024-03-27T06:41:31.447274+0000","last_fullsized":"2024-03-27T06:41:31.447274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:09:05.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386890+0000","last_change":"2024-03-27T06:41:29.413374+0000","last_active":"2024-03-27T06:41:30.386890+0000","last_peered":"2024-03-27T06:41:30.386890+0000","last_clean":"2024-03-27T06:41:30.386890+0000","last_became_active":"2024-03-27T06:41:29.413010+0000","last_became_peered":"2024-03-27T06:41:29.413010+0000","last_unstale":"2024-03-27T06:41:30.386890+0000","last_undegraded":"2024-03-27T06:41:30.386890+0000","last_fullsized":"2024-03-27T06:41:30.386890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:48:52.329611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486307+0000","last_change":"2024-03-27T06:41:29.408640+0000","last_active":"2024-03-27T06:41:31.486307+0000","last_peered":"2024-03-27T06:41:31.486307+0000","last_clean":"2024-03-27T06:41:31.486307+0000","last_became_active":"2024-03-27T06:41:29.408268+0000","last_became_peered":"2024-03-27T06:41:29.408268+0000","last_unstale":"2024-03-27T06:41:31.486307+0000","last_undegraded":"2024-03-27T06:41:31.486307+0000","last_fullsized":"2024-03-27T06:41:31.486307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:51:59.803596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486475+0000","last_change":"2024-03-27T06:41:29.420436+0000","last_active":"2024-03-27T06:41:31.486475+0000","last_peered":"2024-03-27T06:41:31.486475+0000","last_clean":"2024-03-27T06:41:31.486475+0000","last_became_active":"2024-03-27T06:41:29.419426+0000","last_became_peered":"2024-03-27T06:41:29.419426+0000","last_unstale":"2024-03-27T06:41:31.486475+0000","last_undegraded":"2024-03-27T06:41:31.486475+0000","last_fullsized":"2024-03-27T06:41:31.486475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:38:15.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486445+0000","last_change":"2024-03-27T06:41:29.415673+0000","last_active":"2024-03-27T06:41:31.486445+0000","last_peered":"2024-03-27T06:41:31.486445+0000","last_clean":"2024-03-27T06:41:31.486445+0000","last_became_active":"2024-03-27T06:41:29.414741+0000","last_became_peered":"2024-03-27T06:41:29.414741+0000","last_unstale":"2024-03-27T06:41:31.486445+0000","last_undegraded":"2024-03-27T06:41:31.486445+0000","last_fullsized":"2024-03-27T06:41:31.486445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:38:43.183517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484491+0000","last_change":"2024-03-27T06:41:29.408609+0000","last_active":"2024-03-27T06:41:31.484491+0000","last_peered":"2024-03-27T06:41:31.484491+0000","last_clean":"2024-03-27T06:41:31.484491+0000","last_became_active":"2024-03-27T06:41:29.408252+0000","last_became_peered":"2024-03-27T06:41:29.408252+0000","last_unstale":"2024-03-27T06:41:31.484491+0000","last_undegraded":"2024-03-27T06:41:31.484491+0000","last_fullsized":"2024-03-27T06:41:31.484491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:14:13.609113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499934+0000","last_change":"2024-03-27T06:41:29.409221+0000","last_active":"2024-03-27T06:41:30.499934+0000","last_peered":"2024-03-27T06:41:30.499934+0000","last_clean":"2024-03-27T06:41:30.499934+0000","last_became_active":"2024-03-27T06:41:29.408598+0000","last_became_peered":"2024-03-27T06:41:29.408598+0000","last_unstale":"2024-03-27T06:41:30.499934+0000","last_undegraded":"2024-03-27T06:41:30.499934+0000","last_fullsized":"2024-03-27T06:41:30.499934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:42:16.251149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.392104+0000","last_change":"2024-03-27T06:41:29.406143+0000","last_active":"2024-03-27T06:41:31.392104+0000","last_peered":"2024-03-27T06:41:31.392104+0000","last_clean":"2024-03-27T06:41:31.392104+0000","last_became_active":"2024-03-27T06:41:29.405754+0000","last_became_peered":"2024-03-27T06:41:29.405754+0000","last_unstale":"2024-03-27T06:41:31.392104+0000","last_undegraded":"2024-03-27T06:41:31.392104+0000","last_fullsized":"2024-03-27T06:41:31.392104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:42:47.787099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486367+0000","last_change":"2024-03-27T06:41:29.402471+0000","last_active":"2024-03-27T06:41:31.486367+0000","last_peered":"2024-03-27T06:41:31.486367+0000","last_clean":"2024-03-27T06:41:31.486367+0000","last_became_active":"2024-03-27T06:41:29.402113+0000","last_became_peered":"2024-03-27T06:41:29.402113+0000","last_unstale":"2024-03-27T06:41:31.486367+0000","last_undegraded":"2024-03-27T06:41:31.486367+0000","last_fullsized":"2024-03-27T06:41:31.486367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:34:16.408058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.498076+0000","last_change":"2024-03-27T06:41:28.397815+0000","last_active":"2024-03-27T06:41:32.498076+0000","last_peered":"2024-03-27T06:41:32.498076+0000","last_clean":"2024-03-27T06:41:32.498076+0000","last_became_active":"2024-03-27T06:41:28.397503+0000","last_became_peered":"2024-03-27T06:41:28.397503+0000","last_unstale":"2024-03-27T06:41:32.498076+0000","last_undegraded":"2024-03-27T06:41:32.498076+0000","last_fullsized":"2024-03-27T06:41:32.498076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:15:53.751809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386143+0000","last_change":"2024-03-27T06:41:29.419528+0000","last_active":"2024-03-27T06:41:30.386143+0000","last_peered":"2024-03-27T06:41:30.386143+0000","last_clean":"2024-03-27T06:41:30.386143+0000","last_became_active":"2024-03-27T06:41:29.417091+0000","last_became_peered":"2024-03-27T06:41:29.417091+0000","last_unstale":"2024-03-27T06:41:30.386143+0000","last_undegraded":"2024-03-27T06:41:30.386143+0000","last_fullsized":"2024-03-27T06:41:30.386143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:13:14.629211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386273+0000","last_change":"2024-03-27T06:41:28.400468+0000","last_active":"2024-03-27T06:41:30.386273+0000","last_peered":"2024-03-27T06:41:30.386273+0000","last_clean":"2024-03-27T06:41:30.386273+0000","last_became_active":"2024-03-27T06:41:28.400190+0000","last_became_peered":"2024-03-27T06:41:28.400190+0000","last_unstale":"2024-03-27T06:41:30.386273+0000","last_undegraded":"2024-03-27T06:41:30.386273+0000","last_fullsized":"2024-03-27T06:41:30.386273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:24:17.059874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.487462+0000","last_change":"2024-03-27T06:41:29.409364+0000","last_active":"2024-03-27T06:41:31.487462+0000","last_peered":"2024-03-27T06:41:31.487462+0000","last_clean":"2024-03-27T06:41:31.487462+0000","last_became_active":"2024-03-27T06:41:29.408946+0000","last_became_peered":"2024-03-27T06:41:29.408946+0000","last_unstale":"2024-03-27T06:41:31.487462+0000","last_undegraded":"2024-03-27T06:41:31.487462+0000","last_fullsized":"2024-03-27T06:41:31.487462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:46:17.132071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'11","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:34.518361+0000","last_change":"2024-03-27T06:41:28.401327+0000","last_active":"2024-03-27T06:41:34.518361+0000","last_peered":"2024-03-27T06:41:34.518361+0000","last_clean":"2024-03-27T06:41:34.518361+0000","last_became_active":"2024-03-27T06:41:28.400524+0000","last_became_peered":"2024-03-27T06:41:28.400524+0000","last_unstale":"2024-03-27T06:41:34.518361+0000","last_undegraded":"2024-03-27T06:41:34.518361+0000","last_fullsized":"2024-03-27T06:41:34.518361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:11:52.608690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485863+0000","last_change":"2024-03-27T06:41:29.403042+0000","last_active":"2024-03-27T06:41:31.485863+0000","last_peered":"2024-03-27T06:41:31.485863+0000","last_clean":"2024-03-27T06:41:31.485863+0000","last_became_active":"2024-03-27T06:41:29.402516+0000","last_became_peered":"2024-03-27T06:41:29.402516+0000","last_unstale":"2024-03-27T06:41:31.485863+0000","last_undegraded":"2024-03-27T06:41:31.485863+0000","last_fullsized":"2024-03-27T06:41:31.485863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:20:55.777498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:34.517278+0000","last_change":"2024-03-27T06:41:28.395155+0000","last_active":"2024-03-27T06:41:34.517278+0000","last_peered":"2024-03-27T06:41:34.517278+0000","last_clean":"2024-03-27T06:41:34.517278+0000","last_became_active":"2024-03-27T06:41:28.394760+0000","last_became_peered":"2024-03-27T06:41:28.394760+0000","last_unstale":"2024-03-27T06:41:34.517278+0000","last_undegraded":"2024-03-27T06:41:34.517278+0000","last_fullsized":"2024-03-27T06:41:34.517278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:15:28.182581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.387384+0000","last_change":"2024-03-27T06:41:29.409747+0000","last_active":"2024-03-27T06:41:30.387384+0000","last_peered":"2024-03-27T06:41:30.387384+0000","last_clean":"2024-03-27T06:41:30.387384+0000","last_became_active":"2024-03-27T06:41:29.409425+0000","last_became_peered":"2024-03-27T06:41:29.409425+0000","last_unstale":"2024-03-27T06:41:30.387384+0000","last_undegraded":"2024-03-27T06:41:30.387384+0000","last_fullsized":"2024-03-27T06:41:30.387384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:36.181279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.489480+0000","last_change":"2024-03-27T06:41:28.394726+0000","last_active":"2024-03-27T06:41:31.489480+0000","last_peered":"2024-03-27T06:41:31.489480+0000","last_clean":"2024-03-27T06:41:31.489480+0000","last_became_active":"2024-03-27T06:41:28.394378+0000","last_became_peered":"2024-03-27T06:41:28.394378+0000","last_unstale":"2024-03-27T06:41:31.489480+0000","last_undegraded":"2024-03-27T06:41:31.489480+0000","last_fullsized":"2024-03-27T06:41:31.489480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:30:22.194211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.995883+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:41:30.995883+0000","last_peered":"2024-03-27T06:41:30.995883+0000","last_clean":"2024-03-27T06:41:30.995883+0000","last_became_active":"2024-03-27T06:41:29.408826+0000","last_became_peered":"2024-03-27T06:41:29.408826+0000","last_unstale":"2024-03-27T06:41:30.995883+0000","last_undegraded":"2024-03-27T06:41:30.995883+0000","last_fullsized":"2024-03-27T06:41:30.995883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:50:53.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.496181+0000","last_change":"2024-03-27T06:41:28.393655+0000","last_active":"2024-03-27T06:41:32.496181+0000","last_peered":"2024-03-27T06:41:32.496181+0000","last_clean":"2024-03-27T06:41:32.496181+0000","last_became_active":"2024-03-27T06:41:28.393245+0000","last_became_peered":"2024-03-27T06:41:28.393245+0000","last_unstale":"2024-03-27T06:41:32.496181+0000","last_undegraded":"2024-03-27T06:41:32.496181+0000","last_fullsized":"2024-03-27T06:41:32.496181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:58:21.169444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.387105+0000","last_change":"2024-03-27T06:41:29.417730+0000","last_active":"2024-03-27T06:41:30.387105+0000","last_peered":"2024-03-27T06:41:30.387105+0000","last_clean":"2024-03-27T06:41:30.387105+0000","last_became_active":"2024-03-27T06:41:29.417319+0000","last_became_peered":"2024-03-27T06:41:29.417319+0000","last_unstale":"2024-03-27T06:41:30.387105+0000","last_undegraded":"2024-03-27T06:41:30.387105+0000","last_fullsized":"2024-03-27T06:41:30.387105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:25.224220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.497816+0000","last_change":"2024-03-27T06:41:28.402259+0000","last_active":"2024-03-27T06:41:32.497816+0000","last_peered":"2024-03-27T06:41:32.497816+0000","last_clean":"2024-03-27T06:41:32.497816+0000","last_became_active":"2024-03-27T06:41:28.401922+0000","last_became_peered":"2024-03-27T06:41:28.401922+0000","last_unstale":"2024-03-27T06:41:32.497816+0000","last_undegraded":"2024-03-27T06:41:32.497816+0000","last_fullsized":"2024-03-27T06:41:32.497816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:57:33.245183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.385798+0000","last_change":"2024-03-27T06:41:29.409034+0000","last_active":"2024-03-27T06:41:30.385798+0000","last_peered":"2024-03-27T06:41:30.385798+0000","last_clean":"2024-03-27T06:41:30.385798+0000","last_became_active":"2024-03-27T06:41:29.408666+0000","last_became_peered":"2024-03-27T06:41:29.408666+0000","last_unstale":"2024-03-27T06:41:30.385798+0000","last_undegraded":"2024-03-27T06:41:30.385798+0000","last_fullsized":"2024-03-27T06:41:30.385798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:09:55.532059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386766+0000","last_change":"2024-03-27T06:41:28.393802+0000","last_active":"2024-03-27T06:41:30.386766+0000","last_peered":"2024-03-27T06:41:30.386766+0000","last_clean":"2024-03-27T06:41:30.386766+0000","last_became_active":"2024-03-27T06:41:28.393503+0000","last_became_peered":"2024-03-27T06:41:28.393503+0000","last_unstale":"2024-03-27T06:41:30.386766+0000","last_undegraded":"2024-03-27T06:41:30.386766+0000","last_fullsized":"2024-03-27T06:41:30.386766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:17.765709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.391948+0000","last_change":"2024-03-27T06:41:29.416557+0000","last_active":"2024-03-27T06:41:31.391948+0000","last_peered":"2024-03-27T06:41:31.391948+0000","last_clean":"2024-03-27T06:41:31.391948+0000","last_became_active":"2024-03-27T06:41:29.416223+0000","last_became_peered":"2024-03-27T06:41:29.416223+0000","last_unstale":"2024-03-27T06:41:31.391948+0000","last_undegraded":"2024-03-27T06:41:31.391948+0000","last_fullsized":"2024-03-27T06:41:31.391948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:54:20.929325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.489773+0000","last_change":"2024-03-27T06:41:28.396014+0000","last_active":"2024-03-27T06:41:31.489773+0000","last_peered":"2024-03-27T06:41:31.489773+0000","last_clean":"2024-03-27T06:41:31.489773+0000","last_became_active":"2024-03-27T06:41:28.395699+0000","last_became_peered":"2024-03-27T06:41:28.395699+0000","last_unstale":"2024-03-27T06:41:31.489773+0000","last_undegraded":"2024-03-27T06:41:31.489773+0000","last_fullsized":"2024-03-27T06:41:31.489773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:00:59.679843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485861+0000","last_change":"2024-03-27T06:41:29.401668+0000","last_active":"2024-03-27T06:41:31.485861+0000","last_peered":"2024-03-27T06:41:31.485861+0000","last_clean":"2024-03-27T06:41:31.485861+0000","last_became_active":"2024-03-27T06:41:29.401328+0000","last_became_peered":"2024-03-27T06:41:29.401328+0000","last_unstale":"2024-03-27T06:41:31.485861+0000","last_undegraded":"2024-03-27T06:41:31.485861+0000","last_fullsized":"2024-03-27T06:41:31.485861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:32:54.273194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:34.517174+0000","last_change":"2024-03-27T06:41:28.399900+0000","last_active":"2024-03-27T06:41:34.517174+0000","last_peered":"2024-03-27T06:41:34.517174+0000","last_clean":"2024-03-27T06:41:34.517174+0000","last_became_active":"2024-03-27T06:41:28.399560+0000","last_became_peered":"2024-03-27T06:41:28.399560+0000","last_unstale":"2024-03-27T06:41:34.517174+0000","last_undegraded":"2024-03-27T06:41:34.517174+0000","last_fullsized":"2024-03-27T06:41:34.517174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:05:59.054473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.487279+0000","last_change":"2024-03-27T06:41:29.413919+0000","last_active":"2024-03-27T06:41:31.487279+0000","last_peered":"2024-03-27T06:41:31.487279+0000","last_clean":"2024-03-27T06:41:31.487279+0000","last_became_active":"2024-03-27T06:41:29.413566+0000","last_became_peered":"2024-03-27T06:41:29.413566+0000","last_unstale":"2024-03-27T06:41:31.487279+0000","last_undegraded":"2024-03-27T06:41:31.487279+0000","last_fullsized":"2024-03-27T06:41:31.487279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:45:03.705372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.387381+0000","last_change":"2024-03-27T06:41:28.391661+0000","last_active":"2024-03-27T06:41:30.387381+0000","last_peered":"2024-03-27T06:41:30.387381+0000","last_clean":"2024-03-27T06:41:30.387381+0000","last_became_active":"2024-03-27T06:41:28.391230+0000","last_became_peered":"2024-03-27T06:41:28.391230+0000","last_unstale":"2024-03-27T06:41:30.387381+0000","last_undegraded":"2024-03-27T06:41:30.387381+0000","last_fullsized":"2024-03-27T06:41:30.387381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:41:22.789267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486135+0000","last_change":"2024-03-27T06:41:29.405727+0000","last_active":"2024-03-27T06:41:31.486135+0000","last_peered":"2024-03-27T06:41:31.486135+0000","last_clean":"2024-03-27T06:41:31.486135+0000","last_became_active":"2024-03-27T06:41:29.405413+0000","last_became_peered":"2024-03-27T06:41:29.405413+0000","last_unstale":"2024-03-27T06:41:31.486135+0000","last_undegraded":"2024-03-27T06:41:31.486135+0000","last_fullsized":"2024-03-27T06:41:31.486135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:35:21.575729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.489532+0000","last_change":"2024-03-27T06:41:28.391155+0000","last_active":"2024-03-27T06:41:31.489532+0000","last_peered":"2024-03-27T06:41:31.489532+0000","last_clean":"2024-03-27T06:41:31.489532+0000","last_became_active":"2024-03-27T06:41:28.390828+0000","last_became_peered":"2024-03-27T06:41:28.390828+0000","last_unstale":"2024-03-27T06:41:31.489532+0000","last_undegraded":"2024-03-27T06:41:31.489532+0000","last_fullsized":"2024-03-27T06:41:31.489532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:40.192560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.996162+0000","last_change":"2024-03-27T06:41:29.413283+0000","last_active":"2024-03-27T06:41:30.996162+0000","last_peered":"2024-03-27T06:41:30.996162+0000","last_clean":"2024-03-27T06:41:30.996162+0000","last_became_active":"2024-03-27T06:41:29.412247+0000","last_became_peered":"2024-03-27T06:41:29.412247+0000","last_unstale":"2024-03-27T06:41:30.996162+0000","last_undegraded":"2024-03-27T06:41:30.996162+0000","last_fullsized":"2024-03-27T06:41:30.996162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:50:15.336003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.489806+0000","last_change":"2024-03-27T06:41:28.395089+0000","last_active":"2024-03-27T06:41:31.489806+0000","last_peered":"2024-03-27T06:41:31.489806+0000","last_clean":"2024-03-27T06:41:31.489806+0000","last_became_active":"2024-03-27T06:41:28.394145+0000","last_became_peered":"2024-03-27T06:41:28.394145+0000","last_unstale":"2024-03-27T06:41:31.489806+0000","last_undegraded":"2024-03-27T06:41:31.489806+0000","last_fullsized":"2024-03-27T06:41:31.489806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:40.858404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484490+0000","last_change":"2024-03-27T06:41:29.416955+0000","last_active":"2024-03-27T06:41:31.484490+0000","last_peered":"2024-03-27T06:41:31.484490+0000","last_clean":"2024-03-27T06:41:31.484490+0000","last_became_active":"2024-03-27T06:41:29.416616+0000","last_became_peered":"2024-03-27T06:41:29.416616+0000","last_unstale":"2024-03-27T06:41:31.484490+0000","last_undegraded":"2024-03-27T06:41:31.484490+0000","last_fullsized":"2024-03-27T06:41:31.484490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:38:18.791602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.488985+0000","last_change":"2024-03-27T06:41:28.393876+0000","last_active":"2024-03-27T06:41:31.488985+0000","last_peered":"2024-03-27T06:41:31.488985+0000","last_clean":"2024-03-27T06:41:31.488985+0000","last_became_active":"2024-03-27T06:41:28.393557+0000","last_became_peered":"2024-03-27T06:41:28.393557+0000","last_unstale":"2024-03-27T06:41:31.488985+0000","last_undegraded":"2024-03-27T06:41:31.488985+0000","last_fullsized":"2024-03-27T06:41:31.488985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:30.831039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447116+0000","last_change":"2024-03-27T06:41:29.407421+0000","last_active":"2024-03-27T06:41:31.447116+0000","last_peered":"2024-03-27T06:41:31.447116+0000","last_clean":"2024-03-27T06:41:31.447116+0000","last_became_active":"2024-03-27T06:41:29.407084+0000","last_became_peered":"2024-03-27T06:41:29.407084+0000","last_unstale":"2024-03-27T06:41:31.447116+0000","last_undegraded":"2024-03-27T06:41:31.447116+0000","last_fullsized":"2024-03-27T06:41:31.447116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:50:10.145598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386985+0000","last_change":"2024-03-27T06:41:28.393849+0000","last_active":"2024-03-27T06:41:30.386985+0000","last_peered":"2024-03-27T06:41:30.386985+0000","last_clean":"2024-03-27T06:41:30.386985+0000","last_became_active":"2024-03-27T06:41:28.393529+0000","last_became_peered":"2024-03-27T06:41:28.393529+0000","last_unstale":"2024-03-27T06:41:30.386985+0000","last_undegraded":"2024-03-27T06:41:30.386985+0000","last_fullsized":"2024-03-27T06:41:30.386985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:24:23.251723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.446841+0000","last_change":"2024-03-27T06:41:29.411273+0000","last_active":"2024-03-27T06:41:31.446841+0000","last_peered":"2024-03-27T06:41:31.446841+0000","last_clean":"2024-03-27T06:41:31.446841+0000","last_became_active":"2024-03-27T06:41:29.410936+0000","last_became_peered":"2024-03-27T06:41:29.410936+0000","last_unstale":"2024-03-27T06:41:31.446841+0000","last_undegraded":"2024-03-27T06:41:31.446841+0000","last_fullsized":"2024-03-27T06:41:31.446841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:41.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:33.501794+0000","last_change":"2024-03-27T06:41:28.401406+0000","last_active":"2024-03-27T06:41:33.501794+0000","last_peered":"2024-03-27T06:41:33.501794+0000","last_clean":"2024-03-27T06:41:33.501794+0000","last_became_active":"2024-03-27T06:41:28.401049+0000","last_became_peered":"2024-03-27T06:41:28.401049+0000","last_unstale":"2024-03-27T06:41:33.501794+0000","last_undegraded":"2024-03-27T06:41:33.501794+0000","last_fullsized":"2024-03-27T06:41:33.501794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:21:27.398474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386819+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:41:30.386819+0000","last_peered":"2024-03-27T06:41:30.386819+0000","last_clean":"2024-03-27T06:41:30.386819+0000","last_became_active":"2024-03-27T06:41:29.408815+0000","last_became_peered":"2024-03-27T06:41:29.408815+0000","last_unstale":"2024-03-27T06:41:30.386819+0000","last_undegraded":"2024-03-27T06:41:30.386819+0000","last_fullsized":"2024-03-27T06:41:30.386819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:45.547747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.996481+0000","last_change":"2024-03-27T06:41:28.398249+0000","last_active":"2024-03-27T06:41:30.996481+0000","last_peered":"2024-03-27T06:41:30.996481+0000","last_clean":"2024-03-27T06:41:30.996481+0000","last_became_active":"2024-03-27T06:41:28.397176+0000","last_became_peered":"2024-03-27T06:41:28.397176+0000","last_unstale":"2024-03-27T06:41:30.996481+0000","last_undegraded":"2024-03-27T06:41:30.996481+0000","last_fullsized":"2024-03-27T06:41:30.996481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:32.614558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499874+0000","last_change":"2024-03-27T06:41:29.412928+0000","last_active":"2024-03-27T06:41:30.499874+0000","last_peered":"2024-03-27T06:41:30.499874+0000","last_clean":"2024-03-27T06:41:30.499874+0000","last_became_active":"2024-03-27T06:41:29.412576+0000","last_became_peered":"2024-03-27T06:41:29.412576+0000","last_unstale":"2024-03-27T06:41:30.499874+0000","last_undegraded":"2024-03-27T06:41:30.499874+0000","last_fullsized":"2024-03-27T06:41:30.499874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:01:22.389424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.391917+0000","last_change":"2024-03-27T06:41:28.397135+0000","last_active":"2024-03-27T06:41:31.391917+0000","last_peered":"2024-03-27T06:41:31.391917+0000","last_clean":"2024-03-27T06:41:31.391917+0000","last_became_active":"2024-03-27T06:41:28.396483+0000","last_became_peered":"2024-03-27T06:41:28.396483+0000","last_unstale":"2024-03-27T06:41:31.391917+0000","last_undegraded":"2024-03-27T06:41:31.391917+0000","last_fullsized":"2024-03-27T06:41:31.391917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:05:02.038024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485102+0000","last_change":"2024-03-27T06:41:29.418629+0000","last_active":"2024-03-27T06:41:31.485102+0000","last_peered":"2024-03-27T06:41:31.485102+0000","last_clean":"2024-03-27T06:41:31.485102+0000","last_became_active":"2024-03-27T06:41:29.418278+0000","last_became_peered":"2024-03-27T06:41:29.418278+0000","last_unstale":"2024-03-27T06:41:31.485102+0000","last_undegraded":"2024-03-27T06:41:31.485102+0000","last_fullsized":"2024-03-27T06:41:31.485102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:33:54.461349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:34.518510+0000","last_change":"2024-03-27T06:41:28.400611+0000","last_active":"2024-03-27T06:41:34.518510+0000","last_peered":"2024-03-27T06:41:34.518510+0000","last_clean":"2024-03-27T06:41:34.518510+0000","last_became_active":"2024-03-27T06:41:28.400211+0000","last_became_peered":"2024-03-27T06:41:28.400211+0000","last_unstale":"2024-03-27T06:41:34.518510+0000","last_undegraded":"2024-03-27T06:41:34.518510+0000","last_fullsized":"2024-03-27T06:41:34.518510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:46:35.491071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484494+0000","last_change":"2024-03-27T06:41:29.413246+0000","last_active":"2024-03-27T06:41:31.484494+0000","last_peered":"2024-03-27T06:41:31.484494+0000","last_clean":"2024-03-27T06:41:31.484494+0000","last_became_active":"2024-03-27T06:41:29.412946+0000","last_became_peered":"2024-03-27T06:41:29.412946+0000","last_unstale":"2024-03-27T06:41:31.484494+0000","last_undegraded":"2024-03-27T06:41:31.484494+0000","last_fullsized":"2024-03-27T06:41:31.484494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:41:51.441233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386959+0000","last_change":"2024-03-27T06:41:29.402160+0000","last_active":"2024-03-27T06:41:30.386959+0000","last_peered":"2024-03-27T06:41:30.386959+0000","last_clean":"2024-03-27T06:41:30.386959+0000","last_became_active":"2024-03-27T06:41:29.401789+0000","last_became_peered":"2024-03-27T06:41:29.401789+0000","last_unstale":"2024-03-27T06:41:30.386959+0000","last_undegraded":"2024-03-27T06:41:30.386959+0000","last_fullsized":"2024-03-27T06:41:30.386959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:31:27.821966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.497735+0000","last_change":"2024-03-27T06:41:28.395978+0000","last_active":"2024-03-27T06:41:32.497735+0000","last_peered":"2024-03-27T06:41:32.497735+0000","last_clean":"2024-03-27T06:41:32.497735+0000","last_became_active":"2024-03-27T06:41:28.395650+0000","last_became_peered":"2024-03-27T06:41:28.395650+0000","last_unstale":"2024-03-27T06:41:32.497735+0000","last_undegraded":"2024-03-27T06:41:32.497735+0000","last_fullsized":"2024-03-27T06:41:32.497735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:01:15.841344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386229+0000","last_change":"2024-03-27T06:41:29.423303+0000","last_active":"2024-03-27T06:41:30.386229+0000","last_peered":"2024-03-27T06:41:30.386229+0000","last_clean":"2024-03-27T06:41:30.386229+0000","last_became_active":"2024-03-27T06:41:29.422977+0000","last_became_peered":"2024-03-27T06:41:29.422977+0000","last_unstale":"2024-03-27T06:41:30.386229+0000","last_undegraded":"2024-03-27T06:41:30.386229+0000","last_fullsized":"2024-03-27T06:41:30.386229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:14:01.877801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.488210+0000","last_change":"2024-03-27T06:41:28.395141+0000","last_active":"2024-03-27T06:41:31.488210+0000","last_peered":"2024-03-27T06:41:31.488210+0000","last_clean":"2024-03-27T06:41:31.488210+0000","last_became_active":"2024-03-27T06:41:28.394826+0000","last_became_peered":"2024-03-27T06:41:28.394826+0000","last_unstale":"2024-03-27T06:41:31.488210+0000","last_undegraded":"2024-03-27T06:41:31.488210+0000","last_fullsized":"2024-03-27T06:41:31.488210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:29:49.997090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.446870+0000","last_change":"2024-03-27T06:40:16.460939+0000","last_active":"2024-03-27T06:41:31.446870+0000","last_peered":"2024-03-27T06:41:31.446870+0000","last_clean":"2024-03-27T06:41:31.446870+0000","last_became_active":"2024-03-27T06:40:16.460453+0000","last_became_peered":"2024-03-27T06:40:16.460453+0000","last_unstale":"2024-03-27T06:41:31.446870+0000","last_undegraded":"2024-03-27T06:41:31.446870+0000","last_fullsized":"2024-03-27T06:41:31.446870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_clean_scrub_stamp":"2024-03-27T06:36:48.325777+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:28.241688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484847+0000","last_change":"2024-03-27T06:41:29.404203+0000","last_active":"2024-03-27T06:41:31.484847+0000","last_peered":"2024-03-27T06:41:31.484847+0000","last_clean":"2024-03-27T06:41:31.484847+0000","last_became_active":"2024-03-27T06:41:29.403829+0000","last_became_peered":"2024-03-27T06:41:29.403829+0000","last_unstale":"2024-03-27T06:41:31.484847+0000","last_undegraded":"2024-03-27T06:41:31.484847+0000","last_fullsized":"2024-03-27T06:41:31.484847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:48:25.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.497940+0000","last_change":"2024-03-27T06:41:28.399205+0000","last_active":"2024-03-27T06:41:32.497940+0000","last_peered":"2024-03-27T06:41:32.497940+0000","last_clean":"2024-03-27T06:41:32.497940+0000","last_became_active":"2024-03-27T06:41:28.398889+0000","last_became_peered":"2024-03-27T06:41:28.398889+0000","last_unstale":"2024-03-27T06:41:32.497940+0000","last_undegraded":"2024-03-27T06:41:32.497940+0000","last_fullsized":"2024-03-27T06:41:32.497940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:28:08.769187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:33.503207+0000","last_change":"2024-03-27T06:41:28.401511+0000","last_active":"2024-03-27T06:41:33.503207+0000","last_peered":"2024-03-27T06:41:33.503207+0000","last_clean":"2024-03-27T06:41:33.503207+0000","last_became_active":"2024-03-27T06:41:28.401067+0000","last_became_peered":"2024-03-27T06:41:28.401067+0000","last_unstale":"2024-03-27T06:41:33.503207+0000","last_undegraded":"2024-03-27T06:41:33.503207+0000","last_fullsized":"2024-03-27T06:41:33.503207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:14:27.088706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484112+0000","last_change":"2024-03-27T06:41:29.395466+0000","last_active":"2024-03-27T06:41:31.484112+0000","last_peered":"2024-03-27T06:41:31.484112+0000","last_clean":"2024-03-27T06:41:31.484112+0000","last_became_active":"2024-03-27T06:41:29.395094+0000","last_became_peered":"2024-03-27T06:41:29.395094+0000","last_unstale":"2024-03-27T06:41:31.484112+0000","last_undegraded":"2024-03-27T06:41:31.484112+0000","last_fullsized":"2024-03-27T06:41:31.484112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:29:01.746809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.496816+0000","last_change":"2024-03-27T06:41:28.400420+0000","last_active":"2024-03-27T06:41:32.496816+0000","last_peered":"2024-03-27T06:41:32.496816+0000","last_clean":"2024-03-27T06:41:32.496816+0000","last_became_active":"2024-03-27T06:41:28.400115+0000","last_became_peered":"2024-03-27T06:41:28.400115+0000","last_unstale":"2024-03-27T06:41:32.496816+0000","last_undegraded":"2024-03-27T06:41:32.496816+0000","last_fullsized":"2024-03-27T06:41:32.496816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:37:40.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386121+0000","last_change":"2024-03-27T06:41:29.411428+0000","last_active":"2024-03-27T06:41:30.386121+0000","last_peered":"2024-03-27T06:41:30.386121+0000","last_clean":"2024-03-27T06:41:30.386121+0000","last_became_active":"2024-03-27T06:41:29.411075+0000","last_became_peered":"2024-03-27T06:41:29.411075+0000","last_unstale":"2024-03-27T06:41:30.386121+0000","last_undegraded":"2024-03-27T06:41:30.386121+0000","last_fullsized":"2024-03-27T06:41:30.386121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:55.764459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.487843+0000","last_change":"2024-03-27T06:41:28.399799+0000","last_active":"2024-03-27T06:41:31.487843+0000","last_peered":"2024-03-27T06:41:31.487843+0000","last_clean":"2024-03-27T06:41:31.487843+0000","last_became_active":"2024-03-27T06:41:28.399385+0000","last_became_peered":"2024-03-27T06:41:28.399385+0000","last_unstale":"2024-03-27T06:41:31.487843+0000","last_undegraded":"2024-03-27T06:41:31.487843+0000","last_fullsized":"2024-03-27T06:41:31.487843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:16:18.005519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.487856+0000","last_change":"2024-03-27T06:41:29.418991+0000","last_active":"2024-03-27T06:41:31.487856+0000","last_peered":"2024-03-27T06:41:31.487856+0000","last_clean":"2024-03-27T06:41:31.487856+0000","last_became_active":"2024-03-27T06:41:29.418527+0000","last_became_peered":"2024-03-27T06:41:29.418527+0000","last_unstale":"2024-03-27T06:41:31.487856+0000","last_undegraded":"2024-03-27T06:41:31.487856+0000","last_fullsized":"2024-03-27T06:41:31.487856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:49:55.913276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.488408+0000","last_change":"2024-03-27T06:41:28.397771+0000","last_active":"2024-03-27T06:41:31.488408+0000","last_peered":"2024-03-27T06:41:31.488408+0000","last_clean":"2024-03-27T06:41:31.488408+0000","last_became_active":"2024-03-27T06:41:28.397470+0000","last_became_peered":"2024-03-27T06:41:28.397470+0000","last_unstale":"2024-03-27T06:41:31.488408+0000","last_undegraded":"2024-03-27T06:41:31.488408+0000","last_fullsized":"2024-03-27T06:41:31.488408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:37:21.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.995858+0000","last_change":"2024-03-27T06:41:29.412280+0000","last_active":"2024-03-27T06:41:30.995858+0000","last_peered":"2024-03-27T06:41:30.995858+0000","last_clean":"2024-03-27T06:41:30.995858+0000","last_became_active":"2024-03-27T06:41:29.411336+0000","last_became_peered":"2024-03-27T06:41:29.411336+0000","last_unstale":"2024-03-27T06:41:30.995858+0000","last_undegraded":"2024-03-27T06:41:30.995858+0000","last_fullsized":"2024-03-27T06:41:30.995858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:13:03.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.488405+0000","last_change":"2024-03-27T06:41:28.403529+0000","last_active":"2024-03-27T06:41:31.488405+0000","last_peered":"2024-03-27T06:41:31.488405+0000","last_clean":"2024-03-27T06:41:31.488405+0000","last_became_active":"2024-03-27T06:41:28.403155+0000","last_became_peered":"2024-03-27T06:41:28.403155+0000","last_unstale":"2024-03-27T06:41:31.488405+0000","last_undegraded":"2024-03-27T06:41:31.488405+0000","last_fullsized":"2024-03-27T06:41:31.488405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:23:53.971374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499816+0000","last_change":"2024-03-27T06:41:29.404929+0000","last_active":"2024-03-27T06:41:30.499816+0000","last_peered":"2024-03-27T06:41:30.499816+0000","last_clean":"2024-03-27T06:41:30.499816+0000","last_became_active":"2024-03-27T06:41:29.404602+0000","last_became_peered":"2024-03-27T06:41:29.404602+0000","last_unstale":"2024-03-27T06:41:30.499816+0000","last_undegraded":"2024-03-27T06:41:30.499816+0000","last_fullsized":"2024-03-27T06:41:30.499816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:19:07.379046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486063+0000","last_change":"2024-03-27T06:41:28.398602+0000","last_active":"2024-03-27T06:41:31.486063+0000","last_peered":"2024-03-27T06:41:31.486063+0000","last_clean":"2024-03-27T06:41:31.486063+0000","last_became_active":"2024-03-27T06:41:28.398068+0000","last_became_peered":"2024-03-27T06:41:28.398068+0000","last_unstale":"2024-03-27T06:41:31.486063+0000","last_undegraded":"2024-03-27T06:41:31.486063+0000","last_fullsized":"2024-03-27T06:41:31.486063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:10:53.312044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386101+0000","last_change":"2024-03-27T06:41:29.420245+0000","last_active":"2024-03-27T06:41:30.386101+0000","last_peered":"2024-03-27T06:41:30.386101+0000","last_clean":"2024-03-27T06:41:30.386101+0000","last_became_active":"2024-03-27T06:41:29.419912+0000","last_became_peered":"2024-03-27T06:41:29.419912+0000","last_unstale":"2024-03-27T06:41:30.386101+0000","last_undegraded":"2024-03-27T06:41:30.386101+0000","last_fullsized":"2024-03-27T06:41:30.386101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:46:58.203083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485070+0000","last_change":"2024-03-27T06:41:28.398595+0000","last_active":"2024-03-27T06:41:31.485070+0000","last_peered":"2024-03-27T06:41:31.485070+0000","last_clean":"2024-03-27T06:41:31.485070+0000","last_became_active":"2024-03-27T06:41:28.398036+0000","last_became_peered":"2024-03-27T06:41:28.398036+0000","last_unstale":"2024-03-27T06:41:31.485070+0000","last_undegraded":"2024-03-27T06:41:31.485070+0000","last_fullsized":"2024-03-27T06:41:31.485070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:47:15.769760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447120+0000","last_change":"2024-03-27T06:41:29.406576+0000","last_active":"2024-03-27T06:41:31.447120+0000","last_peered":"2024-03-27T06:41:31.447120+0000","last_clean":"2024-03-27T06:41:31.447120+0000","last_became_active":"2024-03-27T06:41:29.406222+0000","last_became_peered":"2024-03-27T06:41:29.406222+0000","last_unstale":"2024-03-27T06:41:31.447120+0000","last_undegraded":"2024-03-27T06:41:31.447120+0000","last_fullsized":"2024-03-27T06:41:31.447120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:05:12.446154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486702+0000","last_change":"2024-03-27T06:41:28.399361+0000","last_active":"2024-03-27T06:41:31.486702+0000","last_peered":"2024-03-27T06:41:31.486702+0000","last_clean":"2024-03-27T06:41:31.486702+0000","last_became_active":"2024-03-27T06:41:28.398021+0000","last_became_peered":"2024-03-27T06:41:28.398021+0000","last_unstale":"2024-03-27T06:41:31.486702+0000","last_undegraded":"2024-03-27T06:41:31.486702+0000","last_fullsized":"2024-03-27T06:41:31.486702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:51:34.106340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447288+0000","last_change":"2024-03-27T06:41:29.405696+0000","last_active":"2024-03-27T06:41:31.447288+0000","last_peered":"2024-03-27T06:41:31.447288+0000","last_clean":"2024-03-27T06:41:31.447288+0000","last_became_active":"2024-03-27T06:41:29.405273+0000","last_became_peered":"2024-03-27T06:41:29.405273+0000","last_unstale":"2024-03-27T06:41:31.447288+0000","last_undegraded":"2024-03-27T06:41:31.447288+0000","last_fullsized":"2024-03-27T06:41:31.447288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:02:58.778464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.387074+0000","last_change":"2024-03-27T06:41:28.391406+0000","last_active":"2024-03-27T06:41:30.387074+0000","last_peered":"2024-03-27T06:41:30.387074+0000","last_clean":"2024-03-27T06:41:30.387074+0000","last_became_active":"2024-03-27T06:41:28.390950+0000","last_became_peered":"2024-03-27T06:41:28.390950+0000","last_unstale":"2024-03-27T06:41:30.387074+0000","last_undegraded":"2024-03-27T06:41:30.387074+0000","last_fullsized":"2024-03-27T06:41:30.387074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:10:48.347139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447059+0000","last_change":"2024-03-27T06:41:29.410422+0000","last_active":"2024-03-27T06:41:31.447059+0000","last_peered":"2024-03-27T06:41:31.447059+0000","last_clean":"2024-03-27T06:41:31.447059+0000","last_became_active":"2024-03-27T06:41:29.409738+0000","last_became_peered":"2024-03-27T06:41:29.409738+0000","last_unstale":"2024-03-27T06:41:31.447059+0000","last_undegraded":"2024-03-27T06:41:31.447059+0000","last_fullsized":"2024-03-27T06:41:31.447059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:38:17.546504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484646+0000","last_change":"2024-03-27T06:41:28.398328+0000","last_active":"2024-03-27T06:41:31.484646+0000","last_peered":"2024-03-27T06:41:31.484646+0000","last_clean":"2024-03-27T06:41:31.484646+0000","last_became_active":"2024-03-27T06:41:28.397972+0000","last_became_peered":"2024-03-27T06:41:28.397972+0000","last_unstale":"2024-03-27T06:41:31.484646+0000","last_undegraded":"2024-03-27T06:41:31.484646+0000","last_fullsized":"2024-03-27T06:41:31.484646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:11:48.501071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447370+0000","last_change":"2024-03-27T06:41:29.408809+0000","last_active":"2024-03-27T06:41:31.447370+0000","last_peered":"2024-03-27T06:41:31.447370+0000","last_clean":"2024-03-27T06:41:31.447370+0000","last_became_active":"2024-03-27T06:41:29.408447+0000","last_became_peered":"2024-03-27T06:41:29.408447+0000","last_unstale":"2024-03-27T06:41:31.447370+0000","last_undegraded":"2024-03-27T06:41:31.447370+0000","last_fullsized":"2024-03-27T06:41:31.447370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:04:55.269490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499751+0000","last_change":"2024-03-27T06:41:29.403265+0000","last_active":"2024-03-27T06:41:30.499751+0000","last_peered":"2024-03-27T06:41:30.499751+0000","last_clean":"2024-03-27T06:41:30.499751+0000","last_became_active":"2024-03-27T06:41:29.402891+0000","last_became_peered":"2024-03-27T06:41:29.402891+0000","last_unstale":"2024-03-27T06:41:30.499751+0000","last_undegraded":"2024-03-27T06:41:30.499751+0000","last_fullsized":"2024-03-27T06:41:30.499751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:08:26.557314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486322+0000","last_change":"2024-03-27T06:41:29.409896+0000","last_active":"2024-03-27T06:41:31.486322+0000","last_peered":"2024-03-27T06:41:31.486322+0000","last_clean":"2024-03-27T06:41:31.486322+0000","last_became_active":"2024-03-27T06:41:29.408279+0000","last_became_peered":"2024-03-27T06:41:29.408279+0000","last_unstale":"2024-03-27T06:41:31.486322+0000","last_undegraded":"2024-03-27T06:41:31.486322+0000","last_fullsized":"2024-03-27T06:41:31.486322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:49:03.257543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486189+0000","last_change":"2024-03-27T06:41:29.412526+0000","last_active":"2024-03-27T06:41:31.486189+0000","last_peered":"2024-03-27T06:41:31.486189+0000","last_clean":"2024-03-27T06:41:31.486189+0000","last_became_active":"2024-03-27T06:41:29.412180+0000","last_became_peered":"2024-03-27T06:41:29.412180+0000","last_unstale":"2024-03-27T06:41:31.486189+0000","last_undegraded":"2024-03-27T06:41:31.486189+0000","last_fullsized":"2024-03-27T06:41:31.486189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:47.461457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486932+0000","last_change":"2024-03-27T06:41:29.406512+0000","last_active":"2024-03-27T06:41:31.486932+0000","last_peered":"2024-03-27T06:41:31.486932+0000","last_clean":"2024-03-27T06:41:31.486932+0000","last_became_active":"2024-03-27T06:41:29.406183+0000","last_became_peered":"2024-03-27T06:41:29.406183+0000","last_unstale":"2024-03-27T06:41:31.486932+0000","last_undegraded":"2024-03-27T06:41:31.486932+0000","last_fullsized":"2024-03-27T06:41:31.486932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:08:14.393102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484146+0000","last_change":"2024-03-27T06:41:29.402442+0000","last_active":"2024-03-27T06:41:31.484146+0000","last_peered":"2024-03-27T06:41:31.484146+0000","last_clean":"2024-03-27T06:41:31.484146+0000","last_became_active":"2024-03-27T06:41:29.402078+0000","last_became_peered":"2024-03-27T06:41:29.402078+0000","last_unstale":"2024-03-27T06:41:31.484146+0000","last_undegraded":"2024-03-27T06:41:31.484146+0000","last_fullsized":"2024-03-27T06:41:31.484146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:07:35.344881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499799+0000","last_change":"2024-03-27T06:41:29.418563+0000","last_active":"2024-03-27T06:41:30.499799+0000","last_peered":"2024-03-27T06:41:30.499799+0000","last_clean":"2024-03-27T06:41:30.499799+0000","last_became_active":"2024-03-27T06:41:29.418209+0000","last_became_peered":"2024-03-27T06:41:29.418209+0000","last_unstale":"2024-03-27T06:41:30.499799+0000","last_undegraded":"2024-03-27T06:41:30.499799+0000","last_fullsized":"2024-03-27T06:41:30.499799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:53:58.536148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486875+0000","last_change":"2024-03-27T06:41:29.400414+0000","last_active":"2024-03-27T06:41:31.486875+0000","last_peered":"2024-03-27T06:41:31.486875+0000","last_clean":"2024-03-27T06:41:31.486875+0000","last_became_active":"2024-03-27T06:41:29.397865+0000","last_became_peered":"2024-03-27T06:41:29.397865+0000","last_unstale":"2024-03-27T06:41:31.486875+0000","last_undegraded":"2024-03-27T06:41:31.486875+0000","last_fullsized":"2024-03-27T06:41:31.486875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:35:28.405347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.385947+0000","last_change":"2024-03-27T06:41:29.416740+0000","last_active":"2024-03-27T06:41:30.385947+0000","last_peered":"2024-03-27T06:41:30.385947+0000","last_clean":"2024-03-27T06:41:30.385947+0000","last_became_active":"2024-03-27T06:41:29.416417+0000","last_became_peered":"2024-03-27T06:41:29.416417+0000","last_unstale":"2024-03-27T06:41:30.385947+0000","last_undegraded":"2024-03-27T06:41:30.385947+0000","last_fullsized":"2024-03-27T06:41:30.385947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:08:34.186527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386990+0000","last_change":"2024-03-27T06:41:29.418797+0000","last_active":"2024-03-27T06:41:30.386990+0000","last_peered":"2024-03-27T06:41:30.386990+0000","last_clean":"2024-03-27T06:41:30.386990+0000","last_became_active":"2024-03-27T06:41:29.418472+0000","last_became_peered":"2024-03-27T06:41:29.418472+0000","last_unstale":"2024-03-27T06:41:30.386990+0000","last_undegraded":"2024-03-27T06:41:30.386990+0000","last_fullsized":"2024-03-27T06:41:30.386990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:30:55.540353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.385785+0000","last_change":"2024-03-27T06:41:29.416124+0000","last_active":"2024-03-27T06:41:30.385785+0000","last_peered":"2024-03-27T06:41:30.385785+0000","last_clean":"2024-03-27T06:41:30.385785+0000","last_became_active":"2024-03-27T06:41:29.415665+0000","last_became_peered":"2024-03-27T06:41:29.415665+0000","last_unstale":"2024-03-27T06:41:30.385785+0000","last_undegraded":"2024-03-27T06:41:30.385785+0000","last_fullsized":"2024-03-27T06:41:30.385785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:46:04.160705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484729+0000","last_change":"2024-03-27T06:41:29.417165+0000","last_active":"2024-03-27T06:41:31.484729+0000","last_peered":"2024-03-27T06:41:31.484729+0000","last_clean":"2024-03-27T06:41:31.484729+0000","last_became_active":"2024-03-27T06:41:29.416856+0000","last_became_peered":"2024-03-27T06:41:29.416856+0000","last_unstale":"2024-03-27T06:41:31.484729+0000","last_undegraded":"2024-03-27T06:41:31.484729+0000","last_fullsized":"2024-03-27T06:41:31.484729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:55:20.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486129+0000","last_change":"2024-03-27T06:41:29.412377+0000","last_active":"2024-03-27T06:41:31.486129+0000","last_peered":"2024-03-27T06:41:31.486129+0000","last_clean":"2024-03-27T06:41:31.486129+0000","last_became_active":"2024-03-27T06:41:29.411761+0000","last_became_peered":"2024-03-27T06:41:29.411761+0000","last_unstale":"2024-03-27T06:41:31.486129+0000","last_undegraded":"2024-03-27T06:41:31.486129+0000","last_fullsized":"2024-03-27T06:41:31.486129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:16:56.844342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499690+0000","last_change":"2024-03-27T06:41:29.413660+0000","last_active":"2024-03-27T06:41:30.499690+0000","last_peered":"2024-03-27T06:41:30.499690+0000","last_clean":"2024-03-27T06:41:30.499690+0000","last_became_active":"2024-03-27T06:41:29.413201+0000","last_became_peered":"2024-03-27T06:41:29.413201+0000","last_unstale":"2024-03-27T06:41:30.499690+0000","last_undegraded":"2024-03-27T06:41:30.499690+0000","last_fullsized":"2024-03-27T06:41:30.499690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:14:46.446595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":278864,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5278,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":17182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350988,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":1035130,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776151,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":442356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":443072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":1035186,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":441684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":443050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":443840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":441676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":1033416,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T06:41:36.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T06:41:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.451+0000 7f835ad9f640 1 -- 172.21.15.16:0/2220622283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83541075f0 msgr2=0x7f8354109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.451+0000 7f835ad9f640 1 --2- 172.21.15.16:0/2220622283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83541075f0 0x7f8354109ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f834c009ca0 tx=0x7f834c02c0f0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.451+0000 7f835ad9f640 1 -- 172.21.15.16:0/2220622283 shutdown_connections 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.451+0000 7f835ad9f640 1 --2- 172.21.15.16:0/2220622283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83541075f0 0x7f8354109ab0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.451+0000 7f835ad9f640 1 --2- 172.21.15.16:0/2220622283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354100060 0x7f8354107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.451+0000 7f835ad9f640 1 --2- 172.21.15.16:0/2220622283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83540ff700 0x7f83540ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.451+0000 7f835ad9f640 1 -- 172.21.15.16:0/2220622283 >> 172.21.15.16:0/2220622283 conn(0x7f83540fb130 msgr2=0x7f83540fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.451+0000 7f835ad9f640 1 -- 172.21.15.16:0/2220622283 shutdown_connections 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.451+0000 7f835ad9f640 1 -- 172.21.15.16:0/2220622283 wait complete. 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835ad9f640 1 Processor -- start 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835ad9f640 1 -- start start 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835ad9f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83540ff700 0x7f8354197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835ad9f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354100060 0x7f8354198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835ad9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83541075f0 0x7f835419f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835ad9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f835410d730 con 0x7f83541075f0 2024-03-27T06:41:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835ad9f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f835410d5b0 con 0x7f83540ff700 2024-03-27T06:41:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835ad9f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f835410d8b0 con 0x7f8354100060 2024-03-27T06:41:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835a59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83541075f0 0x7f835419f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835a59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83541075f0 0x7f835419f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50812/0 (socket says 172.21.15.16:50812) 2024-03-27T06:41:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f8359d9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83540ff700 0x7f8354197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:36.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835a59e640 1 -- 172.21.15.16:0/2857115340 learned_addr learned my addr 172.21.15.16:0/2857115340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835959c640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354100060 0x7f8354198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835a59e640 1 -- 172.21.15.16:0/2857115340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354100060 msgr2=0x7f8354198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835a59e640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354100060 0x7f8354198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835a59e640 1 -- 172.21.15.16:0/2857115340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83540ff700 msgr2=0x7f8354197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835a59e640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83540ff700 0x7f8354197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.452+0000 7f835a59e640 1 -- 172.21.15.16:0/2857115340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835419f920 con 0x7f83541075f0 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.453+0000 7f835a59e640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83541075f0 0x7f835419f280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f834c02c700 tx=0x7f834c0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.453+0000 7f834affd640 1 -- 172.21.15.16:0/2857115340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c039040 con 0x7f83541075f0 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.453+0000 7f834affd640 1 -- 172.21.15.16:0/2857115340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f834c044070 con 0x7f83541075f0 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.453+0000 7f835ad9f640 1 -- 172.21.15.16:0/2857115340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835419fbb0 con 0x7f83541075f0 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.453+0000 7f834affd640 1 -- 172.21.15.16:0/2857115340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c0043c0 con 0x7f83541075f0 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.453+0000 7f835ad9f640 1 -- 172.21.15.16:0/2857115340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83541a00a0 con 0x7f83541075f0 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.454+0000 7f835ad9f640 1 -- 172.21.15.16:0/2857115340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8354105e70 con 0x7f83541075f0 2024-03-27T06:41:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.455+0000 7f834affd640 1 -- 172.21.15.16:0/2857115340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f834c0045a0 con 0x7f83541075f0 2024-03-27T06:41:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.455+0000 7f834affd640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83400a36a0 0x7f83400a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.455+0000 7f8359d9d640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83400a36a0 0x7f83400a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.456+0000 7f834affd640 1 -- 172.21.15.16:0/2857115340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f834c0f2260 con 0x7f83541075f0 2024-03-27T06:41:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.456+0000 7f8359d9d640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83400a36a0 0x7f83400a5b60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f83541991e0 tx=0x7f8350005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:36.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.460+0000 7f834affd640 1 -- 172.21.15.16:0/2857115340 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f834c0bef90 con 0x7f83541075f0 2024-03-27T06:41:36.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:36 smithi016 ceph-mon[31908]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:36.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:36 smithi016 ceph-mon[31908]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:36.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:36 smithi016 ceph-mon[31908]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:36.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:36 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/747067189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:36.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:36 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/634948958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.643+0000 7f835ad9f640 1 -- 172.21.15.16:0/2857115340 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8354061f90 con 0x7f83400a36a0 2024-03-27T06:41:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.651+0000 7f834affd640 1 -- 172.21.15.16:0/2857115340 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258915 (secure 0 0 0) 0x7f8354061f90 con 0x7f83400a36a0 2024-03-27T06:41:36.648 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:36.649 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 -- 172.21.15.16:0/2857115340 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83400a36a0 msgr2=0x7f83400a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83400a36a0 0x7f83400a5b60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f83541991e0 tx=0x7f8350005f90 comp rx=0 tx=0).stop 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 -- 172.21.15.16:0/2857115340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83541075f0 msgr2=0x7f835419f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83541075f0 0x7f835419f280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f834c02c700 tx=0x7f834c0346d0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 -- 172.21.15.16:0/2857115340 shutdown_connections 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f83400a36a0 0x7f83400a5b60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83541075f0 0x7f835419f280 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354100060 0x7f8354198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 --2- 172.21.15.16:0/2857115340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83540ff700 0x7f8354197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 -- 172.21.15.16:0/2857115340 >> 172.21.15.16:0/2857115340 conn(0x7f83540fb130 msgr2=0x7f8354103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 -- 172.21.15.16:0/2857115340 shutdown_connections 2024-03-27T06:41:36.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:36.654+0000 7f835ad9f640 1 -- 172.21.15.16:0/2857115340 wait complete. 2024-03-27T06:41:36.659 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-03-27T06:41:35.562261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595646,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":320,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":251,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364524,"kb_used_data":11172,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577300,"statfs":{"total":1151940427776,"available":1151567155200,"internally_reserved":0,"allocated":11440128,"data_stored":7085760,"data_compressed":106419,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5278,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.180868"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499787+0000","last_change":"2024-03-27T06:41:29.408953+0000","last_active":"2024-03-27T06:41:30.499787+0000","last_peered":"2024-03-27T06:41:30.499787+0000","last_clean":"2024-03-27T06:41:30.499787+0000","last_became_active":"2024-03-27T06:41:29.408143+0000","last_became_peered":"2024-03-27T06:41:29.408143+0000","last_unstale":"2024-03-27T06:41:30.499787+0000","last_undegraded":"2024-03-27T06:41:30.499787+0000","last_fullsized":"2024-03-27T06:41:30.499787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:22.486951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486607+0000","last_change":"2024-03-27T06:41:29.405080+0000","last_active":"2024-03-27T06:41:31.486607+0000","last_peered":"2024-03-27T06:41:31.486607+0000","last_clean":"2024-03-27T06:41:31.486607+0000","last_became_active":"2024-03-27T06:41:29.404743+0000","last_became_peered":"2024-03-27T06:41:29.404743+0000","last_unstale":"2024-03-27T06:41:31.486607+0000","last_undegraded":"2024-03-27T06:41:31.486607+0000","last_fullsized":"2024-03-27T06:41:31.486607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:24:30.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486810+0000","last_change":"2024-03-27T06:41:29.409346+0000","last_active":"2024-03-27T06:41:31.486810+0000","last_peered":"2024-03-27T06:41:31.486810+0000","last_clean":"2024-03-27T06:41:31.486810+0000","last_became_active":"2024-03-27T06:41:29.408837+0000","last_became_peered":"2024-03-27T06:41:29.408837+0000","last_unstale":"2024-03-27T06:41:31.486810+0000","last_undegraded":"2024-03-27T06:41:31.486810+0000","last_fullsized":"2024-03-27T06:41:31.486810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:11:12.274869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386987+0000","last_change":"2024-03-27T06:41:29.412115+0000","last_active":"2024-03-27T06:41:30.386987+0000","last_peered":"2024-03-27T06:41:30.386987+0000","last_clean":"2024-03-27T06:41:30.386987+0000","last_became_active":"2024-03-27T06:41:29.411702+0000","last_became_peered":"2024-03-27T06:41:29.411702+0000","last_unstale":"2024-03-27T06:41:30.386987+0000","last_undegraded":"2024-03-27T06:41:30.386987+0000","last_fullsized":"2024-03-27T06:41:30.386987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:04:42.347229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.996006+0000","last_change":"2024-03-27T06:41:29.409888+0000","last_active":"2024-03-27T06:41:30.996006+0000","last_peered":"2024-03-27T06:41:30.996006+0000","last_clean":"2024-03-27T06:41:30.996006+0000","last_became_active":"2024-03-27T06:41:29.409558+0000","last_became_peered":"2024-03-27T06:41:29.409558+0000","last_unstale":"2024-03-27T06:41:30.996006+0000","last_undegraded":"2024-03-27T06:41:30.996006+0000","last_fullsized":"2024-03-27T06:41:30.996006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:38:51.452793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485103+0000","last_change":"2024-03-27T06:41:29.404088+0000","last_active":"2024-03-27T06:41:31.485103+0000","last_peered":"2024-03-27T06:41:31.485103+0000","last_clean":"2024-03-27T06:41:31.485103+0000","last_became_active":"2024-03-27T06:41:29.403698+0000","last_became_peered":"2024-03-27T06:41:29.403698+0000","last_unstale":"2024-03-27T06:41:31.485103+0000","last_undegraded":"2024-03-27T06:41:31.485103+0000","last_fullsized":"2024-03-27T06:41:31.485103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:14:31.296523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.446893+0000","last_change":"2024-03-27T06:41:29.413157+0000","last_active":"2024-03-27T06:41:31.446893+0000","last_peered":"2024-03-27T06:41:31.446893+0000","last_clean":"2024-03-27T06:41:31.446893+0000","last_became_active":"2024-03-27T06:41:29.412669+0000","last_became_peered":"2024-03-27T06:41:29.412669+0000","last_unstale":"2024-03-27T06:41:31.446893+0000","last_undegraded":"2024-03-27T06:41:31.446893+0000","last_fullsized":"2024-03-27T06:41:31.446893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:28:20.823770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.385741+0000","last_change":"2024-03-27T06:41:29.417207+0000","last_active":"2024-03-27T06:41:30.385741+0000","last_peered":"2024-03-27T06:41:30.385741+0000","last_clean":"2024-03-27T06:41:30.385741+0000","last_became_active":"2024-03-27T06:41:29.415556+0000","last_became_peered":"2024-03-27T06:41:29.415556+0000","last_unstale":"2024-03-27T06:41:30.385741+0000","last_undegraded":"2024-03-27T06:41:30.385741+0000","last_fullsized":"2024-03-27T06:41:30.385741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:26:04.040296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486112+0000","last_change":"2024-03-27T06:41:29.400032+0000","last_active":"2024-03-27T06:41:31.486112+0000","last_peered":"2024-03-27T06:41:31.486112+0000","last_clean":"2024-03-27T06:41:31.486112+0000","last_became_active":"2024-03-27T06:41:29.399274+0000","last_became_peered":"2024-03-27T06:41:29.399274+0000","last_unstale":"2024-03-27T06:41:31.486112+0000","last_undegraded":"2024-03-27T06:41:31.486112+0000","last_fullsized":"2024-03-27T06:41:31.486112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:55.306309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485371+0000","last_change":"2024-03-27T06:41:29.412471+0000","last_active":"2024-03-27T06:41:31.485371+0000","last_peered":"2024-03-27T06:41:31.485371+0000","last_clean":"2024-03-27T06:41:31.485371+0000","last_became_active":"2024-03-27T06:41:29.412134+0000","last_became_peered":"2024-03-27T06:41:29.412134+0000","last_unstale":"2024-03-27T06:41:31.485371+0000","last_undegraded":"2024-03-27T06:41:31.485371+0000","last_fullsized":"2024-03-27T06:41:31.485371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:35:57.146944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447274+0000","last_change":"2024-03-27T06:41:29.407915+0000","last_active":"2024-03-27T06:41:31.447274+0000","last_peered":"2024-03-27T06:41:31.447274+0000","last_clean":"2024-03-27T06:41:31.447274+0000","last_became_active":"2024-03-27T06:41:29.407563+0000","last_became_peered":"2024-03-27T06:41:29.407563+0000","last_unstale":"2024-03-27T06:41:31.447274+0000","last_undegraded":"2024-03-27T06:41:31.447274+0000","last_fullsized":"2024-03-27T06:41:31.447274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:09:05.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386890+0000","last_change":"2024-03-27T06:41:29.413374+0000","last_active":"2024-03-27T06:41:30.386890+0000","last_peered":"2024-03-27T06:41:30.386890+0000","last_clean":"2024-03-27T06:41:30.386890+0000","last_became_active":"2024-03-27T06:41:29.413010+0000","last_became_peered":"2024-03-27T06:41:29.413010+0000","last_unstale":"2024-03-27T06:41:30.386890+0000","last_undegraded":"2024-03-27T06:41:30.386890+0000","last_fullsized":"2024-03-27T06:41:30.386890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:48:52.329611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486307+0000","last_change":"2024-03-27T06:41:29.408640+0000","last_active":"2024-03-27T06:41:31.486307+0000","last_peered":"2024-03-27T06:41:31.486307+0000","last_clean":"2024-03-27T06:41:31.486307+0000","last_became_active":"2024-03-27T06:41:29.408268+0000","last_became_peered":"2024-03-27T06:41:29.408268+0000","last_unstale":"2024-03-27T06:41:31.486307+0000","last_undegraded":"2024-03-27T06:41:31.486307+0000","last_fullsized":"2024-03-27T06:41:31.486307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:51:59.803596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486475+0000","last_change":"2024-03-27T06:41:29.420436+0000","last_active":"2024-03-27T06:41:31.486475+0000","last_peered":"2024-03-27T06:41:31.486475+0000","last_clean":"2024-03-27T06:41:31.486475+0000","last_became_active":"2024-03-27T06:41:29.419426+0000","last_became_peered":"2024-03-27T06:41:29.419426+0000","last_unstale":"2024-03-27T06:41:31.486475+0000","last_undegraded":"2024-03-27T06:41:31.486475+0000","last_fullsized":"2024-03-27T06:41:31.486475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:38:15.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486445+0000","last_change":"2024-03-27T06:41:29.415673+0000","last_active":"2024-03-27T06:41:31.486445+0000","last_peered":"2024-03-27T06:41:31.486445+0000","last_clean":"2024-03-27T06:41:31.486445+0000","last_became_active":"2024-03-27T06:41:29.414741+0000","last_became_peered":"2024-03-27T06:41:29.414741+0000","last_unstale":"2024-03-27T06:41:31.486445+0000","last_undegraded":"2024-03-27T06:41:31.486445+0000","last_fullsized":"2024-03-27T06:41:31.486445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:38:43.183517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484491+0000","last_change":"2024-03-27T06:41:29.408609+0000","last_active":"2024-03-27T06:41:31.484491+0000","last_peered":"2024-03-27T06:41:31.484491+0000","last_clean":"2024-03-27T06:41:31.484491+0000","last_became_active":"2024-03-27T06:41:29.408252+0000","last_became_peered":"2024-03-27T06:41:29.408252+0000","last_unstale":"2024-03-27T06:41:31.484491+0000","last_undegraded":"2024-03-27T06:41:31.484491+0000","last_fullsized":"2024-03-27T06:41:31.484491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:14:13.609113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499934+0000","last_change":"2024-03-27T06:41:29.409221+0000","last_active":"2024-03-27T06:41:30.499934+0000","last_peered":"2024-03-27T06:41:30.499934+0000","last_clean":"2024-03-27T06:41:30.499934+0000","last_became_active":"2024-03-27T06:41:29.408598+0000","last_became_peered":"2024-03-27T06:41:29.408598+0000","last_unstale":"2024-03-27T06:41:30.499934+0000","last_undegraded":"2024-03-27T06:41:30.499934+0000","last_fullsized":"2024-03-27T06:41:30.499934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:42:16.251149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.392104+0000","last_change":"2024-03-27T06:41:29.406143+0000","last_active":"2024-03-27T06:41:31.392104+0000","last_peered":"2024-03-27T06:41:31.392104+0000","last_clean":"2024-03-27T06:41:31.392104+0000","last_became_active":"2024-03-27T06:41:29.405754+0000","last_became_peered":"2024-03-27T06:41:29.405754+0000","last_unstale":"2024-03-27T06:41:31.392104+0000","last_undegraded":"2024-03-27T06:41:31.392104+0000","last_fullsized":"2024-03-27T06:41:31.392104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:42:47.787099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486367+0000","last_change":"2024-03-27T06:41:29.402471+0000","last_active":"2024-03-27T06:41:31.486367+0000","last_peered":"2024-03-27T06:41:31.486367+0000","last_clean":"2024-03-27T06:41:31.486367+0000","last_became_active":"2024-03-27T06:41:29.402113+0000","last_became_peered":"2024-03-27T06:41:29.402113+0000","last_unstale":"2024-03-27T06:41:31.486367+0000","last_undegraded":"2024-03-27T06:41:31.486367+0000","last_fullsized":"2024-03-27T06:41:31.486367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:34:16.408058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.498076+0000","last_change":"2024-03-27T06:41:28.397815+0000","last_active":"2024-03-27T06:41:32.498076+0000","last_peered":"2024-03-27T06:41:32.498076+0000","last_clean":"2024-03-27T06:41:32.498076+0000","last_became_active":"2024-03-27T06:41:28.397503+0000","last_became_peered":"2024-03-27T06:41:28.397503+0000","last_unstale":"2024-03-27T06:41:32.498076+0000","last_undegraded":"2024-03-27T06:41:32.498076+0000","last_fullsized":"2024-03-27T06:41:32.498076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:15:53.751809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386143+0000","last_change":"2024-03-27T06:41:29.419528+0000","last_active":"2024-03-27T06:41:30.386143+0000","last_peered":"2024-03-27T06:41:30.386143+0000","last_clean":"2024-03-27T06:41:30.386143+0000","last_became_active":"2024-03-27T06:41:29.417091+0000","last_became_peered":"2024-03-27T06:41:29.417091+0000","last_unstale":"2024-03-27T06:41:30.386143+0000","last_undegraded":"2024-03-27T06:41:30.386143+0000","last_fullsized":"2024-03-27T06:41:30.386143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:13:14.629211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386273+0000","last_change":"2024-03-27T06:41:28.400468+0000","last_active":"2024-03-27T06:41:30.386273+0000","last_peered":"2024-03-27T06:41:30.386273+0000","last_clean":"2024-03-27T06:41:30.386273+0000","last_became_active":"2024-03-27T06:41:28.400190+0000","last_became_peered":"2024-03-27T06:41:28.400190+0000","last_unstale":"2024-03-27T06:41:30.386273+0000","last_undegraded":"2024-03-27T06:41:30.386273+0000","last_fullsized":"2024-03-27T06:41:30.386273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:24:17.059874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.487462+0000","last_change":"2024-03-27T06:41:29.409364+0000","last_active":"2024-03-27T06:41:31.487462+0000","last_peered":"2024-03-27T06:41:31.487462+0000","last_clean":"2024-03-27T06:41:31.487462+0000","last_became_active":"2024-03-27T06:41:29.408946+0000","last_became_peered":"2024-03-27T06:41:29.408946+0000","last_unstale":"2024-03-27T06:41:31.487462+0000","last_undegraded":"2024-03-27T06:41:31.487462+0000","last_fullsized":"2024-03-27T06:41:31.487462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:46:17.132071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'11","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:34.518361+0000","last_change":"2024-03-27T06:41:28.401327+0000","last_active":"2024-03-27T06:41:34.518361+0000","last_peered":"2024-03-27T06:41:34.518361+0000","last_clean":"2024-03-27T06:41:34.518361+0000","last_became_active":"2024-03-27T06:41:28.400524+0000","last_became_peered":"2024-03-27T06:41:28.400524+0000","last_unstale":"2024-03-27T06:41:34.518361+0000","last_undegraded":"2024-03-27T06:41:34.518361+0000","last_fullsized":"2024-03-27T06:41:34.518361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:11:52.608690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485863+0000","last_change":"2024-03-27T06:41:29.403042+0000","last_active":"2024-03-27T06:41:31.485863+0000","last_peered":"2024-03-27T06:41:31.485863+0000","last_clean":"2024-03-27T06:41:31.485863+0000","last_became_active":"2024-03-27T06:41:29.402516+0000","last_became_peered":"2024-03-27T06:41:29.402516+0000","last_unstale":"2024-03-27T06:41:31.485863+0000","last_undegraded":"2024-03-27T06:41:31.485863+0000","last_fullsized":"2024-03-27T06:41:31.485863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:20:55.777498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:34.517278+0000","last_change":"2024-03-27T06:41:28.395155+0000","last_active":"2024-03-27T06:41:34.517278+0000","last_peered":"2024-03-27T06:41:34.517278+0000","last_clean":"2024-03-27T06:41:34.517278+0000","last_became_active":"2024-03-27T06:41:28.394760+0000","last_became_peered":"2024-03-27T06:41:28.394760+0000","last_unstale":"2024-03-27T06:41:34.517278+0000","last_undegraded":"2024-03-27T06:41:34.517278+0000","last_fullsized":"2024-03-27T06:41:34.517278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:15:28.182581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.387384+0000","last_change":"2024-03-27T06:41:29.409747+0000","last_active":"2024-03-27T06:41:30.387384+0000","last_peered":"2024-03-27T06:41:30.387384+0000","last_clean":"2024-03-27T06:41:30.387384+0000","last_became_active":"2024-03-27T06:41:29.409425+0000","last_became_peered":"2024-03-27T06:41:29.409425+0000","last_unstale":"2024-03-27T06:41:30.387384+0000","last_undegraded":"2024-03-27T06:41:30.387384+0000","last_fullsized":"2024-03-27T06:41:30.387384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:36.181279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.489480+0000","last_change":"2024-03-27T06:41:28.394726+0000","last_active":"2024-03-27T06:41:31.489480+0000","last_peered":"2024-03-27T06:41:31.489480+0000","last_clean":"2024-03-27T06:41:31.489480+0000","last_became_active":"2024-03-27T06:41:28.394378+0000","last_became_peered":"2024-03-27T06:41:28.394378+0000","last_unstale":"2024-03-27T06:41:31.489480+0000","last_undegraded":"2024-03-27T06:41:31.489480+0000","last_fullsized":"2024-03-27T06:41:31.489480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:30:22.194211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.995883+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:41:30.995883+0000","last_peered":"2024-03-27T06:41:30.995883+0000","last_clean":"2024-03-27T06:41:30.995883+0000","last_became_active":"2024-03-27T06:41:29.408826+0000","last_became_peered":"2024-03-27T06:41:29.408826+0000","last_unstale":"2024-03-27T06:41:30.995883+0000","last_undegraded":"2024-03-27T06:41:30.995883+0000","last_fullsized":"2024-03-27T06:41:30.995883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:50:53.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.496181+0000","last_change":"2024-03-27T06:41:28.393655+0000","last_active":"2024-03-27T06:41:32.496181+0000","last_peered":"2024-03-27T06:41:32.496181+0000","last_clean":"2024-03-27T06:41:32.496181+0000","last_became_active":"2024-03-27T06:41:28.393245+0000","last_became_peered":"2024-03-27T06:41:28.393245+0000","last_unstale":"2024-03-27T06:41:32.496181+0000","last_undegraded":"2024-03-27T06:41:32.496181+0000","last_fullsized":"2024-03-27T06:41:32.496181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:58:21.169444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.387105+0000","last_change":"2024-03-27T06:41:29.417730+0000","last_active":"2024-03-27T06:41:30.387105+0000","last_peered":"2024-03-27T06:41:30.387105+0000","last_clean":"2024-03-27T06:41:30.387105+0000","last_became_active":"2024-03-27T06:41:29.417319+0000","last_became_peered":"2024-03-27T06:41:29.417319+0000","last_unstale":"2024-03-27T06:41:30.387105+0000","last_undegraded":"2024-03-27T06:41:30.387105+0000","last_fullsized":"2024-03-27T06:41:30.387105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:25.224220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.497816+0000","last_change":"2024-03-27T06:41:28.402259+0000","last_active":"2024-03-27T06:41:32.497816+0000","last_peered":"2024-03-27T06:41:32.497816+0000","last_clean":"2024-03-27T06:41:32.497816+0000","last_became_active":"2024-03-27T06:41:28.401922+0000","last_became_peered":"2024-03-27T06:41:28.401922+0000","last_unstale":"2024-03-27T06:41:32.497816+0000","last_undegraded":"2024-03-27T06:41:32.497816+0000","last_fullsized":"2024-03-27T06:41:32.497816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:57:33.245183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.385798+0000","last_change":"2024-03-27T06:41:29.409034+0000","last_active":"2024-03-27T06:41:30.385798+0000","last_peered":"2024-03-27T06:41:30.385798+0000","last_clean":"2024-03-27T06:41:30.385798+0000","last_became_active":"2024-03-27T06:41:29.408666+0000","last_became_peered":"2024-03-27T06:41:29.408666+0000","last_unstale":"2024-03-27T06:41:30.385798+0000","last_undegraded":"2024-03-27T06:41:30.385798+0000","last_fullsized":"2024-03-27T06:41:30.385798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:09:55.532059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386766+0000","last_change":"2024-03-27T06:41:28.393802+0000","last_active":"2024-03-27T06:41:30.386766+0000","last_peered":"2024-03-27T06:41:30.386766+0000","last_clean":"2024-03-27T06:41:30.386766+0000","last_became_active":"2024-03-27T06:41:28.393503+0000","last_became_peered":"2024-03-27T06:41:28.393503+0000","last_unstale":"2024-03-27T06:41:30.386766+0000","last_undegraded":"2024-03-27T06:41:30.386766+0000","last_fullsized":"2024-03-27T06:41:30.386766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:17.765709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.391948+0000","last_change":"2024-03-27T06:41:29.416557+0000","last_active":"2024-03-27T06:41:31.391948+0000","last_peered":"2024-03-27T06:41:31.391948+0000","last_clean":"2024-03-27T06:41:31.391948+0000","last_became_active":"2024-03-27T06:41:29.416223+0000","last_became_peered":"2024-03-27T06:41:29.416223+0000","last_unstale":"2024-03-27T06:41:31.391948+0000","last_undegraded":"2024-03-27T06:41:31.391948+0000","last_fullsized":"2024-03-27T06:41:31.391948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:54:20.929325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.489773+0000","last_change":"2024-03-27T06:41:28.396014+0000","last_active":"2024-03-27T06:41:31.489773+0000","last_peered":"2024-03-27T06:41:31.489773+0000","last_clean":"2024-03-27T06:41:31.489773+0000","last_became_active":"2024-03-27T06:41:28.395699+0000","last_became_peered":"2024-03-27T06:41:28.395699+0000","last_unstale":"2024-03-27T06:41:31.489773+0000","last_undegraded":"2024-03-27T06:41:31.489773+0000","last_fullsized":"2024-03-27T06:41:31.489773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:00:59.679843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485861+0000","last_change":"2024-03-27T06:41:29.401668+0000","last_active":"2024-03-27T06:41:31.485861+0000","last_peered":"2024-03-27T06:41:31.485861+0000","last_clean":"2024-03-27T06:41:31.485861+0000","last_became_active":"2024-03-27T06:41:29.401328+0000","last_became_peered":"2024-03-27T06:41:29.401328+0000","last_unstale":"2024-03-27T06:41:31.485861+0000","last_undegraded":"2024-03-27T06:41:31.485861+0000","last_fullsized":"2024-03-27T06:41:31.485861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:32:54.273194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:34.517174+0000","last_change":"2024-03-27T06:41:28.399900+0000","last_active":"2024-03-27T06:41:34.517174+0000","last_peered":"2024-03-27T06:41:34.517174+0000","last_clean":"2024-03-27T06:41:34.517174+0000","last_became_active":"2024-03-27T06:41:28.399560+0000","last_became_peered":"2024-03-27T06:41:28.399560+0000","last_unstale":"2024-03-27T06:41:34.517174+0000","last_undegraded":"2024-03-27T06:41:34.517174+0000","last_fullsized":"2024-03-27T06:41:34.517174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:05:59.054473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.487279+0000","last_change":"2024-03-27T06:41:29.413919+0000","last_active":"2024-03-27T06:41:31.487279+0000","last_peered":"2024-03-27T06:41:31.487279+0000","last_clean":"2024-03-27T06:41:31.487279+0000","last_became_active":"2024-03-27T06:41:29.413566+0000","last_became_peered":"2024-03-27T06:41:29.413566+0000","last_unstale":"2024-03-27T06:41:31.487279+0000","last_undegraded":"2024-03-27T06:41:31.487279+0000","last_fullsized":"2024-03-27T06:41:31.487279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:45:03.705372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.387381+0000","last_change":"2024-03-27T06:41:28.391661+0000","last_active":"2024-03-27T06:41:30.387381+0000","last_peered":"2024-03-27T06:41:30.387381+0000","last_clean":"2024-03-27T06:41:30.387381+0000","last_became_active":"2024-03-27T06:41:28.391230+0000","last_became_peered":"2024-03-27T06:41:28.391230+0000","last_unstale":"2024-03-27T06:41:30.387381+0000","last_undegraded":"2024-03-27T06:41:30.387381+0000","last_fullsized":"2024-03-27T06:41:30.387381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:41:22.789267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486135+0000","last_change":"2024-03-27T06:41:29.405727+0000","last_active":"2024-03-27T06:41:31.486135+0000","last_peered":"2024-03-27T06:41:31.486135+0000","last_clean":"2024-03-27T06:41:31.486135+0000","last_became_active":"2024-03-27T06:41:29.405413+0000","last_became_peered":"2024-03-27T06:41:29.405413+0000","last_unstale":"2024-03-27T06:41:31.486135+0000","last_undegraded":"2024-03-27T06:41:31.486135+0000","last_fullsized":"2024-03-27T06:41:31.486135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:35:21.575729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.489532+0000","last_change":"2024-03-27T06:41:28.391155+0000","last_active":"2024-03-27T06:41:31.489532+0000","last_peered":"2024-03-27T06:41:31.489532+0000","last_clean":"2024-03-27T06:41:31.489532+0000","last_became_active":"2024-03-27T06:41:28.390828+0000","last_became_peered":"2024-03-27T06:41:28.390828+0000","last_unstale":"2024-03-27T06:41:31.489532+0000","last_undegraded":"2024-03-27T06:41:31.489532+0000","last_fullsized":"2024-03-27T06:41:31.489532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:40.192560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.996162+0000","last_change":"2024-03-27T06:41:29.413283+0000","last_active":"2024-03-27T06:41:30.996162+0000","last_peered":"2024-03-27T06:41:30.996162+0000","last_clean":"2024-03-27T06:41:30.996162+0000","last_became_active":"2024-03-27T06:41:29.412247+0000","last_became_peered":"2024-03-27T06:41:29.412247+0000","last_unstale":"2024-03-27T06:41:30.996162+0000","last_undegraded":"2024-03-27T06:41:30.996162+0000","last_fullsized":"2024-03-27T06:41:30.996162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:50:15.336003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.489806+0000","last_change":"2024-03-27T06:41:28.395089+0000","last_active":"2024-03-27T06:41:31.489806+0000","last_peered":"2024-03-27T06:41:31.489806+0000","last_clean":"2024-03-27T06:41:31.489806+0000","last_became_active":"2024-03-27T06:41:28.394145+0000","last_became_peered":"2024-03-27T06:41:28.394145+0000","last_unstale":"2024-03-27T06:41:31.489806+0000","last_undegraded":"2024-03-27T06:41:31.489806+0000","last_fullsized":"2024-03-27T06:41:31.489806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:40.858404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484490+0000","last_change":"2024-03-27T06:41:29.416955+0000","last_active":"2024-03-27T06:41:31.484490+0000","last_peered":"2024-03-27T06:41:31.484490+0000","last_clean":"2024-03-27T06:41:31.484490+0000","last_became_active":"2024-03-27T06:41:29.416616+0000","last_became_peered":"2024-03-27T06:41:29.416616+0000","last_unstale":"2024-03-27T06:41:31.484490+0000","last_undegraded":"2024-03-27T06:41:31.484490+0000","last_fullsized":"2024-03-27T06:41:31.484490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:38:18.791602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.488985+0000","last_change":"2024-03-27T06:41:28.393876+0000","last_active":"2024-03-27T06:41:31.488985+0000","last_peered":"2024-03-27T06:41:31.488985+0000","last_clean":"2024-03-27T06:41:31.488985+0000","last_became_active":"2024-03-27T06:41:28.393557+0000","last_became_peered":"2024-03-27T06:41:28.393557+0000","last_unstale":"2024-03-27T06:41:31.488985+0000","last_undegraded":"2024-03-27T06:41:31.488985+0000","last_fullsized":"2024-03-27T06:41:31.488985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:30.831039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447116+0000","last_change":"2024-03-27T06:41:29.407421+0000","last_active":"2024-03-27T06:41:31.447116+0000","last_peered":"2024-03-27T06:41:31.447116+0000","last_clean":"2024-03-27T06:41:31.447116+0000","last_became_active":"2024-03-27T06:41:29.407084+0000","last_became_peered":"2024-03-27T06:41:29.407084+0000","last_unstale":"2024-03-27T06:41:31.447116+0000","last_undegraded":"2024-03-27T06:41:31.447116+0000","last_fullsized":"2024-03-27T06:41:31.447116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T06:50:10.145598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386985+0000","last_change":"2024-03-27T06:41:28.393849+0000","last_active":"2024-03-27T06:41:30.386985+0000","last_peered":"2024-03-27T06:41:30.386985+0000","last_clean":"2024-03-27T06:41:30.386985+0000","last_became_active":"2024-03-27T06:41:28.393529+0000","last_became_peered":"2024-03-27T06:41:28.393529+0000","last_unstale":"2024-03-27T06:41:30.386985+0000","last_undegraded":"2024-03-27T06:41:30.386985+0000","last_fullsized":"2024-03-27T06:41:30.386985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:24:23.251723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.446841+0000","last_change":"2024-03-27T06:41:29.411273+0000","last_active":"2024-03-27T06:41:31.446841+0000","last_peered":"2024-03-27T06:41:31.446841+0000","last_clean":"2024-03-27T06:41:31.446841+0000","last_became_active":"2024-03-27T06:41:29.410936+0000","last_became_peered":"2024-03-27T06:41:29.410936+0000","last_unstale":"2024-03-27T06:41:31.446841+0000","last_undegraded":"2024-03-27T06:41:31.446841+0000","last_fullsized":"2024-03-27T06:41:31.446841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:41.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:33.501794+0000","last_change":"2024-03-27T06:41:28.401406+0000","last_active":"2024-03-27T06:41:33.501794+0000","last_peered":"2024-03-27T06:41:33.501794+0000","last_clean":"2024-03-27T06:41:33.501794+0000","last_became_active":"2024-03-27T06:41:28.401049+0000","last_became_peered":"2024-03-27T06:41:28.401049+0000","last_unstale":"2024-03-27T06:41:33.501794+0000","last_undegraded":"2024-03-27T06:41:33.501794+0000","last_fullsized":"2024-03-27T06:41:33.501794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:21:27.398474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386819+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:41:30.386819+0000","last_peered":"2024-03-27T06:41:30.386819+0000","last_clean":"2024-03-27T06:41:30.386819+0000","last_became_active":"2024-03-27T06:41:29.408815+0000","last_became_peered":"2024-03-27T06:41:29.408815+0000","last_unstale":"2024-03-27T06:41:30.386819+0000","last_undegraded":"2024-03-27T06:41:30.386819+0000","last_fullsized":"2024-03-27T06:41:30.386819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:45.547747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.996481+0000","last_change":"2024-03-27T06:41:28.398249+0000","last_active":"2024-03-27T06:41:30.996481+0000","last_peered":"2024-03-27T06:41:30.996481+0000","last_clean":"2024-03-27T06:41:30.996481+0000","last_became_active":"2024-03-27T06:41:28.397176+0000","last_became_peered":"2024-03-27T06:41:28.397176+0000","last_unstale":"2024-03-27T06:41:30.996481+0000","last_undegraded":"2024-03-27T06:41:30.996481+0000","last_fullsized":"2024-03-27T06:41:30.996481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:32.614558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499874+0000","last_change":"2024-03-27T06:41:29.412928+0000","last_active":"2024-03-27T06:41:30.499874+0000","last_peered":"2024-03-27T06:41:30.499874+0000","last_clean":"2024-03-27T06:41:30.499874+0000","last_became_active":"2024-03-27T06:41:29.412576+0000","last_became_peered":"2024-03-27T06:41:29.412576+0000","last_unstale":"2024-03-27T06:41:30.499874+0000","last_undegraded":"2024-03-27T06:41:30.499874+0000","last_fullsized":"2024-03-27T06:41:30.499874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:01:22.389424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.391917+0000","last_change":"2024-03-27T06:41:28.397135+0000","last_active":"2024-03-27T06:41:31.391917+0000","last_peered":"2024-03-27T06:41:31.391917+0000","last_clean":"2024-03-27T06:41:31.391917+0000","last_became_active":"2024-03-27T06:41:28.396483+0000","last_became_peered":"2024-03-27T06:41:28.396483+0000","last_unstale":"2024-03-27T06:41:31.391917+0000","last_undegraded":"2024-03-27T06:41:31.391917+0000","last_fullsized":"2024-03-27T06:41:31.391917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:05:02.038024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485102+0000","last_change":"2024-03-27T06:41:29.418629+0000","last_active":"2024-03-27T06:41:31.485102+0000","last_peered":"2024-03-27T06:41:31.485102+0000","last_clean":"2024-03-27T06:41:31.485102+0000","last_became_active":"2024-03-27T06:41:29.418278+0000","last_became_peered":"2024-03-27T06:41:29.418278+0000","last_unstale":"2024-03-27T06:41:31.485102+0000","last_undegraded":"2024-03-27T06:41:31.485102+0000","last_fullsized":"2024-03-27T06:41:31.485102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:33:54.461349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:34.518510+0000","last_change":"2024-03-27T06:41:28.400611+0000","last_active":"2024-03-27T06:41:34.518510+0000","last_peered":"2024-03-27T06:41:34.518510+0000","last_clean":"2024-03-27T06:41:34.518510+0000","last_became_active":"2024-03-27T06:41:28.400211+0000","last_became_peered":"2024-03-27T06:41:28.400211+0000","last_unstale":"2024-03-27T06:41:34.518510+0000","last_undegraded":"2024-03-27T06:41:34.518510+0000","last_fullsized":"2024-03-27T06:41:34.518510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:46:35.491071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484494+0000","last_change":"2024-03-27T06:41:29.413246+0000","last_active":"2024-03-27T06:41:31.484494+0000","last_peered":"2024-03-27T06:41:31.484494+0000","last_clean":"2024-03-27T06:41:31.484494+0000","last_became_active":"2024-03-27T06:41:29.412946+0000","last_became_peered":"2024-03-27T06:41:29.412946+0000","last_unstale":"2024-03-27T06:41:31.484494+0000","last_undegraded":"2024-03-27T06:41:31.484494+0000","last_fullsized":"2024-03-27T06:41:31.484494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:41:51.441233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386959+0000","last_change":"2024-03-27T06:41:29.402160+0000","last_active":"2024-03-27T06:41:30.386959+0000","last_peered":"2024-03-27T06:41:30.386959+0000","last_clean":"2024-03-27T06:41:30.386959+0000","last_became_active":"2024-03-27T06:41:29.401789+0000","last_became_peered":"2024-03-27T06:41:29.401789+0000","last_unstale":"2024-03-27T06:41:30.386959+0000","last_undegraded":"2024-03-27T06:41:30.386959+0000","last_fullsized":"2024-03-27T06:41:30.386959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:31:27.821966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.497735+0000","last_change":"2024-03-27T06:41:28.395978+0000","last_active":"2024-03-27T06:41:32.497735+0000","last_peered":"2024-03-27T06:41:32.497735+0000","last_clean":"2024-03-27T06:41:32.497735+0000","last_became_active":"2024-03-27T06:41:28.395650+0000","last_became_peered":"2024-03-27T06:41:28.395650+0000","last_unstale":"2024-03-27T06:41:32.497735+0000","last_undegraded":"2024-03-27T06:41:32.497735+0000","last_fullsized":"2024-03-27T06:41:32.497735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:01:15.841344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386229+0000","last_change":"2024-03-27T06:41:29.423303+0000","last_active":"2024-03-27T06:41:30.386229+0000","last_peered":"2024-03-27T06:41:30.386229+0000","last_clean":"2024-03-27T06:41:30.386229+0000","last_became_active":"2024-03-27T06:41:29.422977+0000","last_became_peered":"2024-03-27T06:41:29.422977+0000","last_unstale":"2024-03-27T06:41:30.386229+0000","last_undegraded":"2024-03-27T06:41:30.386229+0000","last_fullsized":"2024-03-27T06:41:30.386229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:14:01.877801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.488210+0000","last_change":"2024-03-27T06:41:28.395141+0000","last_active":"2024-03-27T06:41:31.488210+0000","last_peered":"2024-03-27T06:41:31.488210+0000","last_clean":"2024-03-27T06:41:31.488210+0000","last_became_active":"2024-03-27T06:41:28.394826+0000","last_became_peered":"2024-03-27T06:41:28.394826+0000","last_unstale":"2024-03-27T06:41:31.488210+0000","last_undegraded":"2024-03-27T06:41:31.488210+0000","last_fullsized":"2024-03-27T06:41:31.488210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:29:49.997090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.446870+0000","last_change":"2024-03-27T06:40:16.460939+0000","last_active":"2024-03-27T06:41:31.446870+0000","last_peered":"2024-03-27T06:41:31.446870+0000","last_clean":"2024-03-27T06:41:31.446870+0000","last_became_active":"2024-03-27T06:40:16.460453+0000","last_became_peered":"2024-03-27T06:40:16.460453+0000","last_unstale":"2024-03-27T06:41:31.446870+0000","last_undegraded":"2024-03-27T06:41:31.446870+0000","last_fullsized":"2024-03-27T06:41:31.446870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_clean_scrub_stamp":"2024-03-27T06:36:48.325777+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:28.241688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484847+0000","last_change":"2024-03-27T06:41:29.404203+0000","last_active":"2024-03-27T06:41:31.484847+0000","last_peered":"2024-03-27T06:41:31.484847+0000","last_clean":"2024-03-27T06:41:31.484847+0000","last_became_active":"2024-03-27T06:41:29.403829+0000","last_became_peered":"2024-03-27T06:41:29.403829+0000","last_unstale":"2024-03-27T06:41:31.484847+0000","last_undegraded":"2024-03-27T06:41:31.484847+0000","last_fullsized":"2024-03-27T06:41:31.484847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:48:25.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.497940+0000","last_change":"2024-03-27T06:41:28.399205+0000","last_active":"2024-03-27T06:41:32.497940+0000","last_peered":"2024-03-27T06:41:32.497940+0000","last_clean":"2024-03-27T06:41:32.497940+0000","last_became_active":"2024-03-27T06:41:28.398889+0000","last_became_peered":"2024-03-27T06:41:28.398889+0000","last_unstale":"2024-03-27T06:41:32.497940+0000","last_undegraded":"2024-03-27T06:41:32.497940+0000","last_fullsized":"2024-03-27T06:41:32.497940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:28:08.769187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:33.503207+0000","last_change":"2024-03-27T06:41:28.401511+0000","last_active":"2024-03-27T06:41:33.503207+0000","last_peered":"2024-03-27T06:41:33.503207+0000","last_clean":"2024-03-27T06:41:33.503207+0000","last_became_active":"2024-03-27T06:41:28.401067+0000","last_became_peered":"2024-03-27T06:41:28.401067+0000","last_unstale":"2024-03-27T06:41:33.503207+0000","last_undegraded":"2024-03-27T06:41:33.503207+0000","last_fullsized":"2024-03-27T06:41:33.503207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:14:27.088706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484112+0000","last_change":"2024-03-27T06:41:29.395466+0000","last_active":"2024-03-27T06:41:31.484112+0000","last_peered":"2024-03-27T06:41:31.484112+0000","last_clean":"2024-03-27T06:41:31.484112+0000","last_became_active":"2024-03-27T06:41:29.395094+0000","last_became_peered":"2024-03-27T06:41:29.395094+0000","last_unstale":"2024-03-27T06:41:31.484112+0000","last_undegraded":"2024-03-27T06:41:31.484112+0000","last_fullsized":"2024-03-27T06:41:31.484112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:29:01.746809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:32.496816+0000","last_change":"2024-03-27T06:41:28.400420+0000","last_active":"2024-03-27T06:41:32.496816+0000","last_peered":"2024-03-27T06:41:32.496816+0000","last_clean":"2024-03-27T06:41:32.496816+0000","last_became_active":"2024-03-27T06:41:28.400115+0000","last_became_peered":"2024-03-27T06:41:28.400115+0000","last_unstale":"2024-03-27T06:41:32.496816+0000","last_undegraded":"2024-03-27T06:41:32.496816+0000","last_fullsized":"2024-03-27T06:41:32.496816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:37:40.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386121+0000","last_change":"2024-03-27T06:41:29.411428+0000","last_active":"2024-03-27T06:41:30.386121+0000","last_peered":"2024-03-27T06:41:30.386121+0000","last_clean":"2024-03-27T06:41:30.386121+0000","last_became_active":"2024-03-27T06:41:29.411075+0000","last_became_peered":"2024-03-27T06:41:29.411075+0000","last_unstale":"2024-03-27T06:41:30.386121+0000","last_undegraded":"2024-03-27T06:41:30.386121+0000","last_fullsized":"2024-03-27T06:41:30.386121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:55.764459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.487843+0000","last_change":"2024-03-27T06:41:28.399799+0000","last_active":"2024-03-27T06:41:31.487843+0000","last_peered":"2024-03-27T06:41:31.487843+0000","last_clean":"2024-03-27T06:41:31.487843+0000","last_became_active":"2024-03-27T06:41:28.399385+0000","last_became_peered":"2024-03-27T06:41:28.399385+0000","last_unstale":"2024-03-27T06:41:31.487843+0000","last_undegraded":"2024-03-27T06:41:31.487843+0000","last_fullsized":"2024-03-27T06:41:31.487843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:16:18.005519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.487856+0000","last_change":"2024-03-27T06:41:29.418991+0000","last_active":"2024-03-27T06:41:31.487856+0000","last_peered":"2024-03-27T06:41:31.487856+0000","last_clean":"2024-03-27T06:41:31.487856+0000","last_became_active":"2024-03-27T06:41:29.418527+0000","last_became_peered":"2024-03-27T06:41:29.418527+0000","last_unstale":"2024-03-27T06:41:31.487856+0000","last_undegraded":"2024-03-27T06:41:31.487856+0000","last_fullsized":"2024-03-27T06:41:31.487856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:49:55.913276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.488408+0000","last_change":"2024-03-27T06:41:28.397771+0000","last_active":"2024-03-27T06:41:31.488408+0000","last_peered":"2024-03-27T06:41:31.488408+0000","last_clean":"2024-03-27T06:41:31.488408+0000","last_became_active":"2024-03-27T06:41:28.397470+0000","last_became_peered":"2024-03-27T06:41:28.397470+0000","last_unstale":"2024-03-27T06:41:31.488408+0000","last_undegraded":"2024-03-27T06:41:31.488408+0000","last_fullsized":"2024-03-27T06:41:31.488408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:37:21.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.995858+0000","last_change":"2024-03-27T06:41:29.412280+0000","last_active":"2024-03-27T06:41:30.995858+0000","last_peered":"2024-03-27T06:41:30.995858+0000","last_clean":"2024-03-27T06:41:30.995858+0000","last_became_active":"2024-03-27T06:41:29.411336+0000","last_became_peered":"2024-03-27T06:41:29.411336+0000","last_unstale":"2024-03-27T06:41:30.995858+0000","last_undegraded":"2024-03-27T06:41:30.995858+0000","last_fullsized":"2024-03-27T06:41:30.995858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:13:03.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.488405+0000","last_change":"2024-03-27T06:41:28.403529+0000","last_active":"2024-03-27T06:41:31.488405+0000","last_peered":"2024-03-27T06:41:31.488405+0000","last_clean":"2024-03-27T06:41:31.488405+0000","last_became_active":"2024-03-27T06:41:28.403155+0000","last_became_peered":"2024-03-27T06:41:28.403155+0000","last_unstale":"2024-03-27T06:41:31.488405+0000","last_undegraded":"2024-03-27T06:41:31.488405+0000","last_fullsized":"2024-03-27T06:41:31.488405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:23:53.971374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499816+0000","last_change":"2024-03-27T06:41:29.404929+0000","last_active":"2024-03-27T06:41:30.499816+0000","last_peered":"2024-03-27T06:41:30.499816+0000","last_clean":"2024-03-27T06:41:30.499816+0000","last_became_active":"2024-03-27T06:41:29.404602+0000","last_became_peered":"2024-03-27T06:41:29.404602+0000","last_unstale":"2024-03-27T06:41:30.499816+0000","last_undegraded":"2024-03-27T06:41:30.499816+0000","last_fullsized":"2024-03-27T06:41:30.499816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:19:07.379046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486063+0000","last_change":"2024-03-27T06:41:28.398602+0000","last_active":"2024-03-27T06:41:31.486063+0000","last_peered":"2024-03-27T06:41:31.486063+0000","last_clean":"2024-03-27T06:41:31.486063+0000","last_became_active":"2024-03-27T06:41:28.398068+0000","last_became_peered":"2024-03-27T06:41:28.398068+0000","last_unstale":"2024-03-27T06:41:31.486063+0000","last_undegraded":"2024-03-27T06:41:31.486063+0000","last_fullsized":"2024-03-27T06:41:31.486063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:10:53.312044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386101+0000","last_change":"2024-03-27T06:41:29.420245+0000","last_active":"2024-03-27T06:41:30.386101+0000","last_peered":"2024-03-27T06:41:30.386101+0000","last_clean":"2024-03-27T06:41:30.386101+0000","last_became_active":"2024-03-27T06:41:29.419912+0000","last_became_peered":"2024-03-27T06:41:29.419912+0000","last_unstale":"2024-03-27T06:41:30.386101+0000","last_undegraded":"2024-03-27T06:41:30.386101+0000","last_fullsized":"2024-03-27T06:41:30.386101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:46:58.203083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.485070+0000","last_change":"2024-03-27T06:41:28.398595+0000","last_active":"2024-03-27T06:41:31.485070+0000","last_peered":"2024-03-27T06:41:31.485070+0000","last_clean":"2024-03-27T06:41:31.485070+0000","last_became_active":"2024-03-27T06:41:28.398036+0000","last_became_peered":"2024-03-27T06:41:28.398036+0000","last_unstale":"2024-03-27T06:41:31.485070+0000","last_undegraded":"2024-03-27T06:41:31.485070+0000","last_fullsized":"2024-03-27T06:41:31.485070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:47:15.769760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447120+0000","last_change":"2024-03-27T06:41:29.406576+0000","last_active":"2024-03-27T06:41:31.447120+0000","last_peered":"2024-03-27T06:41:31.447120+0000","last_clean":"2024-03-27T06:41:31.447120+0000","last_became_active":"2024-03-27T06:41:29.406222+0000","last_became_peered":"2024-03-27T06:41:29.406222+0000","last_unstale":"2024-03-27T06:41:31.447120+0000","last_undegraded":"2024-03-27T06:41:31.447120+0000","last_fullsized":"2024-03-27T06:41:31.447120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:05:12.446154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486702+0000","last_change":"2024-03-27T06:41:28.399361+0000","last_active":"2024-03-27T06:41:31.486702+0000","last_peered":"2024-03-27T06:41:31.486702+0000","last_clean":"2024-03-27T06:41:31.486702+0000","last_became_active":"2024-03-27T06:41:28.398021+0000","last_became_peered":"2024-03-27T06:41:28.398021+0000","last_unstale":"2024-03-27T06:41:31.486702+0000","last_undegraded":"2024-03-27T06:41:31.486702+0000","last_fullsized":"2024-03-27T06:41:31.486702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:51:34.106340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447288+0000","last_change":"2024-03-27T06:41:29.405696+0000","last_active":"2024-03-27T06:41:31.447288+0000","last_peered":"2024-03-27T06:41:31.447288+0000","last_clean":"2024-03-27T06:41:31.447288+0000","last_became_active":"2024-03-27T06:41:29.405273+0000","last_became_peered":"2024-03-27T06:41:29.405273+0000","last_unstale":"2024-03-27T06:41:31.447288+0000","last_undegraded":"2024-03-27T06:41:31.447288+0000","last_fullsized":"2024-03-27T06:41:31.447288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:02:58.778464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.387074+0000","last_change":"2024-03-27T06:41:28.391406+0000","last_active":"2024-03-27T06:41:30.387074+0000","last_peered":"2024-03-27T06:41:30.387074+0000","last_clean":"2024-03-27T06:41:30.387074+0000","last_became_active":"2024-03-27T06:41:28.390950+0000","last_became_peered":"2024-03-27T06:41:28.390950+0000","last_unstale":"2024-03-27T06:41:30.387074+0000","last_undegraded":"2024-03-27T06:41:30.387074+0000","last_fullsized":"2024-03-27T06:41:30.387074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:10:48.347139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447059+0000","last_change":"2024-03-27T06:41:29.410422+0000","last_active":"2024-03-27T06:41:31.447059+0000","last_peered":"2024-03-27T06:41:31.447059+0000","last_clean":"2024-03-27T06:41:31.447059+0000","last_became_active":"2024-03-27T06:41:29.409738+0000","last_became_peered":"2024-03-27T06:41:29.409738+0000","last_unstale":"2024-03-27T06:41:31.447059+0000","last_undegraded":"2024-03-27T06:41:31.447059+0000","last_fullsized":"2024-03-27T06:41:31.447059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:38:17.546504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484646+0000","last_change":"2024-03-27T06:41:28.398328+0000","last_active":"2024-03-27T06:41:31.484646+0000","last_peered":"2024-03-27T06:41:31.484646+0000","last_clean":"2024-03-27T06:41:31.484646+0000","last_became_active":"2024-03-27T06:41:28.397972+0000","last_became_peered":"2024-03-27T06:41:28.397972+0000","last_unstale":"2024-03-27T06:41:31.484646+0000","last_undegraded":"2024-03-27T06:41:31.484646+0000","last_fullsized":"2024-03-27T06:41:31.484646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:11:48.501071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.447370+0000","last_change":"2024-03-27T06:41:29.408809+0000","last_active":"2024-03-27T06:41:31.447370+0000","last_peered":"2024-03-27T06:41:31.447370+0000","last_clean":"2024-03-27T06:41:31.447370+0000","last_became_active":"2024-03-27T06:41:29.408447+0000","last_became_peered":"2024-03-27T06:41:29.408447+0000","last_unstale":"2024-03-27T06:41:31.447370+0000","last_undegraded":"2024-03-27T06:41:31.447370+0000","last_fullsized":"2024-03-27T06:41:31.447370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:04:55.269490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499751+0000","last_change":"2024-03-27T06:41:29.403265+0000","last_active":"2024-03-27T06:41:30.499751+0000","last_peered":"2024-03-27T06:41:30.499751+0000","last_clean":"2024-03-27T06:41:30.499751+0000","last_became_active":"2024-03-27T06:41:29.402891+0000","last_became_peered":"2024-03-27T06:41:29.402891+0000","last_unstale":"2024-03-27T06:41:30.499751+0000","last_undegraded":"2024-03-27T06:41:30.499751+0000","last_fullsized":"2024-03-27T06:41:30.499751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:08:26.557314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486322+0000","last_change":"2024-03-27T06:41:29.409896+0000","last_active":"2024-03-27T06:41:31.486322+0000","last_peered":"2024-03-27T06:41:31.486322+0000","last_clean":"2024-03-27T06:41:31.486322+0000","last_became_active":"2024-03-27T06:41:29.408279+0000","last_became_peered":"2024-03-27T06:41:29.408279+0000","last_unstale":"2024-03-27T06:41:31.486322+0000","last_undegraded":"2024-03-27T06:41:31.486322+0000","last_fullsized":"2024-03-27T06:41:31.486322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:49:03.257543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486189+0000","last_change":"2024-03-27T06:41:29.412526+0000","last_active":"2024-03-27T06:41:31.486189+0000","last_peered":"2024-03-27T06:41:31.486189+0000","last_clean":"2024-03-27T06:41:31.486189+0000","last_became_active":"2024-03-27T06:41:29.412180+0000","last_became_peered":"2024-03-27T06:41:29.412180+0000","last_unstale":"2024-03-27T06:41:31.486189+0000","last_undegraded":"2024-03-27T06:41:31.486189+0000","last_fullsized":"2024-03-27T06:41:31.486189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:47.461457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486932+0000","last_change":"2024-03-27T06:41:29.406512+0000","last_active":"2024-03-27T06:41:31.486932+0000","last_peered":"2024-03-27T06:41:31.486932+0000","last_clean":"2024-03-27T06:41:31.486932+0000","last_became_active":"2024-03-27T06:41:29.406183+0000","last_became_peered":"2024-03-27T06:41:29.406183+0000","last_unstale":"2024-03-27T06:41:31.486932+0000","last_undegraded":"2024-03-27T06:41:31.486932+0000","last_fullsized":"2024-03-27T06:41:31.486932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:08:14.393102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484146+0000","last_change":"2024-03-27T06:41:29.402442+0000","last_active":"2024-03-27T06:41:31.484146+0000","last_peered":"2024-03-27T06:41:31.484146+0000","last_clean":"2024-03-27T06:41:31.484146+0000","last_became_active":"2024-03-27T06:41:29.402078+0000","last_became_peered":"2024-03-27T06:41:29.402078+0000","last_unstale":"2024-03-27T06:41:31.484146+0000","last_undegraded":"2024-03-27T06:41:31.484146+0000","last_fullsized":"2024-03-27T06:41:31.484146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:07:35.344881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499799+0000","last_change":"2024-03-27T06:41:29.418563+0000","last_active":"2024-03-27T06:41:30.499799+0000","last_peered":"2024-03-27T06:41:30.499799+0000","last_clean":"2024-03-27T06:41:30.499799+0000","last_became_active":"2024-03-27T06:41:29.418209+0000","last_became_peered":"2024-03-27T06:41:29.418209+0000","last_unstale":"2024-03-27T06:41:30.499799+0000","last_undegraded":"2024-03-27T06:41:30.499799+0000","last_fullsized":"2024-03-27T06:41:30.499799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:53:58.536148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486875+0000","last_change":"2024-03-27T06:41:29.400414+0000","last_active":"2024-03-27T06:41:31.486875+0000","last_peered":"2024-03-27T06:41:31.486875+0000","last_clean":"2024-03-27T06:41:31.486875+0000","last_became_active":"2024-03-27T06:41:29.397865+0000","last_became_peered":"2024-03-27T06:41:29.397865+0000","last_unstale":"2024-03-27T06:41:31.486875+0000","last_undegraded":"2024-03-27T06:41:31.486875+0000","last_fullsized":"2024-03-27T06:41:31.486875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:35:28.405347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.385947+0000","last_change":"2024-03-27T06:41:29.416740+0000","last_active":"2024-03-27T06:41:30.385947+0000","last_peered":"2024-03-27T06:41:30.385947+0000","last_clean":"2024-03-27T06:41:30.385947+0000","last_became_active":"2024-03-27T06:41:29.416417+0000","last_became_peered":"2024-03-27T06:41:29.416417+0000","last_unstale":"2024-03-27T06:41:30.385947+0000","last_undegraded":"2024-03-27T06:41:30.385947+0000","last_fullsized":"2024-03-27T06:41:30.385947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:08:34.186527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.386990+0000","last_change":"2024-03-27T06:41:29.418797+0000","last_active":"2024-03-27T06:41:30.386990+0000","last_peered":"2024-03-27T06:41:30.386990+0000","last_clean":"2024-03-27T06:41:30.386990+0000","last_became_active":"2024-03-27T06:41:29.418472+0000","last_became_peered":"2024-03-27T06:41:29.418472+0000","last_unstale":"2024-03-27T06:41:30.386990+0000","last_undegraded":"2024-03-27T06:41:30.386990+0000","last_fullsized":"2024-03-27T06:41:30.386990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:30:55.540353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.385785+0000","last_change":"2024-03-27T06:41:29.416124+0000","last_active":"2024-03-27T06:41:30.385785+0000","last_peered":"2024-03-27T06:41:30.385785+0000","last_clean":"2024-03-27T06:41:30.385785+0000","last_became_active":"2024-03-27T06:41:29.415665+0000","last_became_peered":"2024-03-27T06:41:29.415665+0000","last_unstale":"2024-03-27T06:41:30.385785+0000","last_undegraded":"2024-03-27T06:41:30.385785+0000","last_fullsized":"2024-03-27T06:41:30.385785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:46:04.160705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.484729+0000","last_change":"2024-03-27T06:41:29.417165+0000","last_active":"2024-03-27T06:41:31.484729+0000","last_peered":"2024-03-27T06:41:31.484729+0000","last_clean":"2024-03-27T06:41:31.484729+0000","last_became_active":"2024-03-27T06:41:29.416856+0000","last_became_peered":"2024-03-27T06:41:29.416856+0000","last_unstale":"2024-03-27T06:41:31.484729+0000","last_undegraded":"2024-03-27T06:41:31.484729+0000","last_fullsized":"2024-03-27T06:41:31.484729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:55:20.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:31.486129+0000","last_change":"2024-03-27T06:41:29.412377+0000","last_active":"2024-03-27T06:41:31.486129+0000","last_peered":"2024-03-27T06:41:31.486129+0000","last_clean":"2024-03-27T06:41:31.486129+0000","last_became_active":"2024-03-27T06:41:29.411761+0000","last_became_peered":"2024-03-27T06:41:29.411761+0000","last_unstale":"2024-03-27T06:41:31.486129+0000","last_undegraded":"2024-03-27T06:41:31.486129+0000","last_fullsized":"2024-03-27T06:41:31.486129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:16:56.844342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-27T06:41:30.499690+0000","last_change":"2024-03-27T06:41:29.413660+0000","last_active":"2024-03-27T06:41:30.499690+0000","last_peered":"2024-03-27T06:41:30.499690+0000","last_clean":"2024-03-27T06:41:30.499690+0000","last_became_active":"2024-03-27T06:41:29.413201+0000","last_became_peered":"2024-03-27T06:41:29.413201+0000","last_unstale":"2024-03-27T06:41:30.499690+0000","last_undegraded":"2024-03-27T06:41:30.499690+0000","last_fullsized":"2024-03-27T06:41:30.499690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:14:46.446595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":278864,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5278,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":17182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350988,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":1035130,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776151,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":442356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":443072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":1035186,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":441684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":443050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":443840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":441676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":1033416,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T06:41:36.663 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-27T06:41:36.664 INFO:tasks.cephadm:Setting up client nodes... 2024-03-27T06:41:36.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-27T06:41:36.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:36 smithi170 ceph-mon[36438]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:36.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:36 smithi170 ceph-mon[36438]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:36.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:36 smithi170 ceph-mon[36438]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:36.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:36 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/747067189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:36.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:36 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/634948958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:36.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:36 smithi156 ceph-mon[36261]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:36.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:36 smithi156 ceph-mon[36261]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:36.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:36 smithi156 ceph-mon[36261]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:41:36.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:36 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/747067189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:41:36.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:36 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/634948958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:37.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:37 smithi170 ceph-mon[36438]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s wr, 19 op/s 2024-03-27T06:41:37.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:37 smithi170 ceph-mon[36438]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:41:37.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:37 smithi016 ceph-mon[31908]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s wr, 19 op/s 2024-03-27T06:41:37.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:37 smithi016 ceph-mon[31908]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:41:37.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:37 smithi156 ceph-mon[36261]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s wr, 19 op/s 2024-03-27T06:41:37.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:37 smithi156 ceph-mon[36261]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:41:38.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:38 smithi016 ceph-mon[31908]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:41:38.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.767+0000 7f937bc4e700 1 -- 172.21.15.16:0/1806481421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9374104250 msgr2=0x7f937410ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:38.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.767+0000 7f937bc4e700 1 --2- 172.21.15.16:0/1806481421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9374104250 0x7f937410ab00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f936c00b0d0 tx=0x7f936c00b3e0 comp rx=0 tx=0).stop 2024-03-27T06:41:38.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.767+0000 7f937bc4e700 1 -- 172.21.15.16:0/1806481421 shutdown_connections 2024-03-27T06:41:38.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.767+0000 7f937bc4e700 1 --2- 172.21.15.16:0/1806481421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9374104250 0x7f937410ab00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:38.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.767+0000 7f937bc4e700 1 --2- 172.21.15.16:0/1806481421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93741026b0 0x7f9374102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:38.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.767+0000 7f937bc4e700 1 --2- 172.21.15.16:0/1806481421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93741038b0 0x7f9374103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.767+0000 7f937bc4e700 1 -- 172.21.15.16:0/1806481421 >> 172.21.15.16:0/1806481421 conn(0x7f93740fdc90 msgr2=0x7f93741000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.767+0000 7f937bc4e700 1 -- 172.21.15.16:0/1806481421 shutdown_connections 2024-03-27T06:41:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.767+0000 7f937bc4e700 1 -- 172.21.15.16:0/1806481421 wait complete. 2024-03-27T06:41:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f937bc4e700 1 Processor -- start 2024-03-27T06:41:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f937bc4e700 1 -- start start 2024-03-27T06:41:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f937bc4e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93741026b0 0x7f937419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f937bc4e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93741038b0 0x7f937419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:38.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f937bc4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9374104250 0x7f93741a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:38.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f937bc4e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f937410e3e0 con 0x7f9374104250 2024-03-27T06:41:38.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f937bc4e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f937410e260 con 0x7f93741026b0 2024-03-27T06:41:38.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f937bc4e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f937410e560 con 0x7f93741038b0 2024-03-27T06:41:38.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f937a1eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9374104250 0x7f93741a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:38.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f93791e9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93741038b0 0x7f937419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:38.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f93799ea700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93741026b0 0x7f937419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:38.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f93799ea700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93741026b0 0x7f937419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:52868/0 (socket says 172.21.15.16:52868) 2024-03-27T06:41:38.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f93791e9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93741038b0 0x7f937419af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:48338/0 (socket says 172.21.15.16:48338) 2024-03-27T06:41:38.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f937a1eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9374104250 0x7f93741a1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50828/0 (socket says 172.21.15.16:50828) 2024-03-27T06:41:38.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f93799ea700 1 -- 172.21.15.16:0/1332330063 learned_addr learned my addr 172.21.15.16:0/1332330063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:38.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f93799ea700 1 -- 172.21.15.16:0/1332330063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93741038b0 msgr2=0x7f937419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:38.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f93799ea700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93741038b0 0x7f937419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:38.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f93799ea700 1 -- 172.21.15.16:0/1332330063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9374104250 msgr2=0x7f93741a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:38.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f93799ea700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9374104250 0x7f93741a1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:38.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f93799ea700 1 -- 172.21.15.16:0/1332330063 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93741a2660 con 0x7f93741026b0 2024-03-27T06:41:38.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f93791e9700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93741038b0 0x7f937419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:41:38.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.768+0000 7f937a1eb700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9374104250 0x7f93741a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:41:38.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.769+0000 7f93799ea700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93741026b0 0x7f937419aa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f937000e970 tx=0x7f937000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.769+0000 7f936affd700 1 -- 172.21.15.16:0/1332330063 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370004220 con 0x7f93741026b0 2024-03-27T06:41:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.769+0000 7f936affd700 1 -- 172.21.15.16:0/1332330063 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f937000cce0 con 0x7f93741026b0 2024-03-27T06:41:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.769+0000 7f937bc4e700 1 -- 172.21.15.16:0/1332330063 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93741a2950 con 0x7f93741026b0 2024-03-27T06:41:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.769+0000 7f936affd700 1 -- 172.21.15.16:0/1332330063 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370010430 con 0x7f93741026b0 2024-03-27T06:41:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.769+0000 7f937bc4e700 1 -- 172.21.15.16:0/1332330063 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93741a2e90 con 0x7f93741026b0 2024-03-27T06:41:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.770+0000 7f937bc4e700 1 -- 172.21.15.16:0/1332330063 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9374050b50 con 0x7f93741026b0 2024-03-27T06:41:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.771+0000 7f936affd700 1 -- 172.21.15.16:0/1332330063 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f937000c7e0 con 0x7f93741026b0 2024-03-27T06:41:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.772+0000 7f936affd700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93600a37e0 0x7f93600a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.772+0000 7f93791e9700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93600a37e0 0x7f93600a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.772+0000 7f936affd700 1 -- 172.21.15.16:0/1332330063 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f93700cfff0 con 0x7f93741026b0 2024-03-27T06:41:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.772+0000 7f93791e9700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93600a37e0 0x7f93600a5ca0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9364011490 tx=0x7f9364009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:38.775+0000 7f936affd700 1 -- 172.21.15.16:0/1332330063 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f937001f340 con 0x7f93741026b0 2024-03-27T06:41:38.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:38 smithi170 ceph-mon[36438]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:41:38.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:38 smithi156 ceph-mon[36261]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:41:39.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.070+0000 7f937bc4e700 1 -- 172.21.15.16:0/1332330063 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f93741a3430 con 0x7f93741026b0 2024-03-27T06:41:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.075+0000 7f936affd700 1 -- 172.21.15.16:0/1332330063 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f93741a3430 con 0x7f93741026b0 2024-03-27T06:41:39.070 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-03-27T06:41:39.070 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCjvwNmGbZIBBAAv5U9ECB+H7QvCtqqz4HJmg== 2024-03-27T06:41:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 -- 172.21.15.16:0/1332330063 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93600a37e0 msgr2=0x7f93600a5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93600a37e0 0x7f93600a5ca0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9364011490 tx=0x7f9364009260 comp rx=0 tx=0).stop 2024-03-27T06:41:39.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 -- 172.21.15.16:0/1332330063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93741026b0 msgr2=0x7f937419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93741026b0 0x7f937419aa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f937000e970 tx=0x7f937000ec80 comp rx=0 tx=0).stop 2024-03-27T06:41:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 -- 172.21.15.16:0/1332330063 shutdown_connections 2024-03-27T06:41:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9374104250 0x7f93741a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93741026b0 0x7f937419aa00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f93600a37e0 0x7f93600a5ca0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 --2- 172.21.15.16:0/1332330063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93741038b0 0x7f937419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:39.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 -- 172.21.15.16:0/1332330063 >> 172.21.15.16:0/1332330063 conn(0x7f93740fdc90 msgr2=0x7f9374074870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:39.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 -- 172.21.15.16:0/1332330063 shutdown_connections 2024-03-27T06:41:39.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:39.077+0000 7f937bc4e700 1 -- 172.21.15.16:0/1332330063 wait complete. 2024-03-27T06:41:39.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:39 smithi016 ceph-mon[31908]: pgmap v295: 97 pgs: 97 active+clean; 585 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7.1 KiB/s wr, 27 op/s 2024-03-27T06:41:39.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:39 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1332330063' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-27T06:41:39.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:39 smithi016 ceph-mon[31908]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-27T06:41:39.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:39 smithi016 ceph-mon[31908]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-27T06:41:39.784 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:41:39.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-27T06:41:39.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-27T06:41:39.827 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-27T06:41:39.827 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-27T06:41:39.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-27T06:41:39.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:39 smithi170 ceph-mon[36438]: pgmap v295: 97 pgs: 97 active+clean; 585 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7.1 KiB/s wr, 27 op/s 2024-03-27T06:41:39.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:39 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1332330063' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-27T06:41:39.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:39 smithi170 ceph-mon[36438]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-27T06:41:39.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:39 smithi170 ceph-mon[36438]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-27T06:41:39.972 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:41:39 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b[66776]: 2024-03-27T06:41:39.494+0000 7ff65d639700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-27T06:41:39.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:39 smithi156 ceph-mon[36261]: pgmap v295: 97 pgs: 97 active+clean; 585 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7.1 KiB/s wr, 27 op/s 2024-03-27T06:41:39.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:39 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1332330063' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-27T06:41:39.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:39 smithi156 ceph-mon[36261]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-27T06:41:39.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:39 smithi156 ceph-mon[36261]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-27T06:41:40.110 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:41.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:41 smithi016 ceph-mon[31908]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-03-27T06:41:41.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:41 smithi170 ceph-mon[36438]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-03-27T06:41:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.862+0000 7f4636894700 1 -- 172.21.15.16:0/1093537895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4630113300 msgr2=0x7f46301156f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.862+0000 7f4636894700 1 --2- 172.21.15.16:0/1093537895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4630113300 0x7f46301156f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f462800b0a0 tx=0x7f462800b3b0 comp rx=0 tx=0).stop 2024-03-27T06:41:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.862+0000 7f4636894700 1 -- 172.21.15.16:0/1093537895 shutdown_connections 2024-03-27T06:41:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.862+0000 7f4636894700 1 --2- 172.21.15.16:0/1093537895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4630113300 0x7f46301156f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.862+0000 7f4636894700 1 --2- 172.21.15.16:0/1093537895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4630076660 0x7f4630074870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.862+0000 7f4636894700 1 --2- 172.21.15.16:0/1093537895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4630074db0 0x7f4630075230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.862+0000 7f4636894700 1 -- 172.21.15.16:0/1093537895 >> 172.21.15.16:0/1093537895 conn(0x7f4630079760 msgr2=0x7f4630079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.862+0000 7f4636894700 1 -- 172.21.15.16:0/1093537895 shutdown_connections 2024-03-27T06:41:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.862+0000 7f4636894700 1 -- 172.21.15.16:0/1093537895 wait complete. 2024-03-27T06:41:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.862+0000 7f4636894700 1 Processor -- start 2024-03-27T06:41:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f4636894700 1 -- start start 2024-03-27T06:41:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f4636894700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4630074db0 0x7f463019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f4636894700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4630076660 0x7f463019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f4636894700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4630113300 0x7f46301a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f4636894700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4630119090 con 0x7f4630074db0 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f4636894700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f4630118f10 con 0x7f4630076660 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f4636894700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4630119210 con 0x7f4630113300 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f462ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4630074db0 0x7f463019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f462ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4630074db0 0x7f463019f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50854/0 (socket says 172.21.15.16:50854) 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f462ffff700 1 -- 172.21.15.16:0/4091984802 learned_addr learned my addr 172.21.15.16:0/4091984802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f462f7fe700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4630076660 0x7f463019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f4634e31700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4630113300 0x7f46301a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f462ffff700 1 -- 172.21.15.16:0/4091984802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4630113300 msgr2=0x7f46301a3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f462ffff700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4630113300 0x7f46301a3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f462ffff700 1 -- 172.21.15.16:0/4091984802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4630076660 msgr2=0x7f463019f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f462ffff700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4630076660 0x7f463019f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f462ffff700 1 -- 172.21.15.16:0/4091984802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46301a43f0 con 0x7f4630074db0 2024-03-27T06:41:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f4634e31700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4630113300 0x7f46301a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:41:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.863+0000 7f462f7fe700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4630076660 0x7f463019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:41:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.864+0000 7f462ffff700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4630074db0 0x7f463019f3e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f462000c9b0 tx=0x7f462000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.864+0000 7f462d7fa700 1 -- 172.21.15.16:0/4091984802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620007b00 con 0x7f4630074db0 2024-03-27T06:41:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.864+0000 7f4636894700 1 -- 172.21.15.16:0/4091984802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46301a46e0 con 0x7f4630074db0 2024-03-27T06:41:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.864+0000 7f462d7fa700 1 -- 172.21.15.16:0/4091984802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f462000ce50 con 0x7f4630074db0 2024-03-27T06:41:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.864+0000 7f462d7fa700 1 -- 172.21.15.16:0/4091984802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46200186f0 con 0x7f4630074db0 2024-03-27T06:41:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.864+0000 7f4636894700 1 -- 172.21.15.16:0/4091984802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46301abf60 con 0x7f4630074db0 2024-03-27T06:41:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.865+0000 7f4636894700 1 -- 172.21.15.16:0/4091984802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4630069300 con 0x7f4630074db0 2024-03-27T06:41:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.866+0000 7f462d7fa700 1 -- 172.21.15.16:0/4091984802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f4620007ca0 con 0x7f4630074db0 2024-03-27T06:41:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.866+0000 7f462d7fa700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f461c0a3730 0x7f461c0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.867+0000 7f462f7fe700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f461c0a3730 0x7f461c0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.867+0000 7f462d7fa700 1 -- 172.21.15.16:0/4091984802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f46200cfe00 con 0x7f4630074db0 2024-03-27T06:41:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.867+0000 7f462f7fe700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f461c0a3730 0x7f461c0a5bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4630079dd0 tx=0x7f4618009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:41.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:41.870+0000 7f462d7fa700 1 -- 172.21.15.16:0/4091984802 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4620014030 con 0x7f4630074db0 2024-03-27T06:41:41.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:41 smithi156 ceph-mon[36261]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 26 op/s 2024-03-27T06:41:42.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.134+0000 7f4636894700 1 -- 172.21.15.16:0/4091984802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f46301ac190 con 0x7f4630074db0 2024-03-27T06:41:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.139+0000 7f462d7fa700 1 -- 172.21.15.16:0/4091984802 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264391 (secure 0 0 0) 0x7f462009cab0 con 0x7f4630074db0 2024-03-27T06:41:42.137 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:42.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.141+0000 7f4636894700 1 -- 172.21.15.16:0/4091984802 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f461c0a3730 msgr2=0x7f461c0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:42.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.141+0000 7f4636894700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f461c0a3730 0x7f461c0a5bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4630079dd0 tx=0x7f4618009260 comp rx=0 tx=0).stop 2024-03-27T06:41:42.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.141+0000 7f4636894700 1 -- 172.21.15.16:0/4091984802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4630074db0 msgr2=0x7f463019f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:42.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.141+0000 7f4636894700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4630074db0 0x7f463019f3e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f462000c9b0 tx=0x7f462000ccc0 comp rx=0 tx=0).stop 2024-03-27T06:41:42.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.142+0000 7f4636894700 1 -- 172.21.15.16:0/4091984802 shutdown_connections 2024-03-27T06:41:42.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.142+0000 7f4636894700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f461c0a3730 0x7f461c0a5bf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:42.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.142+0000 7f4636894700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4630113300 0x7f46301a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:42.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.142+0000 7f4636894700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4630074db0 0x7f463019f3e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:42.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.142+0000 7f4636894700 1 --2- 172.21.15.16:0/4091984802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4630076660 0x7f463019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:42.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.142+0000 7f4636894700 1 -- 172.21.15.16:0/4091984802 >> 172.21.15.16:0/4091984802 conn(0x7f4630079760 msgr2=0x7f4630111e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:42.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.142+0000 7f4636894700 1 -- 172.21.15.16:0/4091984802 shutdown_connections 2024-03-27T06:41:42.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:42.142+0000 7f4636894700 1 -- 172.21.15.16:0/4091984802 wait complete. 2024-03-27T06:41:42.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:42 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/4091984802' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-27T06:41:42.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:42 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/4091984802' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-27T06:41:42.850 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:42 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/4091984802' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-27T06:41:42.881 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4198806087},{"type":"v1","addr":"172.21.15.16:6801","nonce":4198806087}]},"active_addr":"172.21.15.16:6801/4198806087","active_change":"2024-03-27T06:34:19.478963+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3519122217}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3655179049}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":361067788}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":212881673}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":67883869}]}]} 2024-03-27T06:41:42.892 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-27T06:41:42.892 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-27T06:41:42.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-27T06:41:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:43.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:43 smithi016 ceph-mon[31908]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.6 KiB/s wr, 25 op/s 2024-03-27T06:41:43.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:43 smithi170 ceph-mon[36438]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.6 KiB/s wr, 25 op/s 2024-03-27T06:41:43.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:43 smithi156 ceph-mon[36261]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.6 KiB/s wr, 25 op/s 2024-03-27T06:41:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.760+0000 7f7848b9c700 1 -- 172.21.15.16:0/600712999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7844101f30 msgr2=0x7f784410e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.760+0000 7f7848b9c700 1 --2- 172.21.15.16:0/600712999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7844101f30 0x7f784410e380 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f783c00b0d0 tx=0x7f783c00b3e0 comp rx=0 tx=0).stop 2024-03-27T06:41:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.760+0000 7f7848b9c700 1 -- 172.21.15.16:0/600712999 shutdown_connections 2024-03-27T06:41:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.760+0000 7f7848b9c700 1 --2- 172.21.15.16:0/600712999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7844101f30 0x7f784410e380 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.760+0000 7f7848b9c700 1 --2- 172.21.15.16:0/600712999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78441075b0 0x7f7844107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.760+0000 7f7848b9c700 1 --2- 172.21.15.16:0/600712999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7844101570 0x7f78441019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.760+0000 7f7848b9c700 1 -- 172.21.15.16:0/600712999 >> 172.21.15.16:0/600712999 conn(0x7f7844077590 msgr2=0x7f78440779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.760+0000 7f7848b9c700 1 -- 172.21.15.16:0/600712999 shutdown_connections 2024-03-27T06:41:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.760+0000 7f7848b9c700 1 -- 172.21.15.16:0/600712999 wait complete. 2024-03-27T06:41:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.761+0000 7f7848b9c700 1 Processor -- start 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.761+0000 7f7848b9c700 1 -- start start 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7848b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7844101570 0x7f784419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7848b9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7844101f30 0x7f784419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7848b9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78441075b0 0x7f7844197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7848b9c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7844111950 con 0x7f7844101f30 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7848b9c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f78441117d0 con 0x7f7844101570 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7848b9c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7844111ad0 con 0x7f78441075b0 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7841d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7844101f30 0x7f784419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7841d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7844101f30 0x7f784419de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50870/0 (socket says 172.21.15.16:50870) 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7841d9b700 1 -- 172.21.15.16:0/343391306 learned_addr learned my addr 172.21.15.16:0/343391306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f784259c700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7844101570 0x7f784419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7842d9d700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78441075b0 0x7f7844197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7841d9b700 1 -- 172.21.15.16:0/343391306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78441075b0 msgr2=0x7f7844197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7841d9b700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78441075b0 0x7f7844197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:44.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7841d9b700 1 -- 172.21.15.16:0/343391306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7844101570 msgr2=0x7f784419d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:44.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7841d9b700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7844101570 0x7f784419d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:44.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7841d9b700 1 -- 172.21.15.16:0/343391306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7844198250 con 0x7f7844101f30 2024-03-27T06:41:44.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7841d9b700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7844101f30 0x7f784419de70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f783400eb30 tx=0x7f783400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:44.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f783b7fe700 1 -- 172.21.15.16:0/343391306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783400cbd0 con 0x7f7844101f30 2024-03-27T06:41:44.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f7848b9c700 1 -- 172.21.15.16:0/343391306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7844198540 con 0x7f7844101f30 2024-03-27T06:41:44.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.762+0000 7f783b7fe700 1 -- 172.21.15.16:0/343391306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7834004520 con 0x7f7844101f30 2024-03-27T06:41:44.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.763+0000 7f783b7fe700 1 -- 172.21.15.16:0/343391306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834010430 con 0x7f7844101f30 2024-03-27T06:41:44.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.763+0000 7f7848b9c700 1 -- 172.21.15.16:0/343391306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78441a4800 con 0x7f7844101f30 2024-03-27T06:41:44.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.764+0000 7f7848b9c700 1 -- 172.21.15.16:0/343391306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7844069300 con 0x7f7844101f30 2024-03-27T06:41:44.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.765+0000 7f783b7fe700 1 -- 172.21.15.16:0/343391306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f7834003fd0 con 0x7f7844101f30 2024-03-27T06:41:44.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.765+0000 7f783b7fe700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78300a3730 0x7f78300a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:44.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.765+0000 7f784259c700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78300a3730 0x7f78300a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:44.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.765+0000 7f783b7fe700 1 -- 172.21.15.16:0/343391306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f78340d0050 con 0x7f7844101f30 2024-03-27T06:41:44.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.766+0000 7f784259c700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78300a3730 0x7f78300a5bf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f782c009c60 tx=0x7f782c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:44.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.772+0000 7f783b7fe700 1 -- 172.21.15.16:0/343391306 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f783409cc70 con 0x7f7844101f30 2024-03-27T06:41:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.977+0000 7f7848b9c700 1 -- 172.21.15.16:0/343391306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78441a4a90 con 0x7f7844101f30 2024-03-27T06:41:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.978+0000 7f783b7fe700 1 -- 172.21.15.16:0/343391306 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22173 (secure 0 0 0) 0x7f783400cd70 con 0x7f7844101f30 2024-03-27T06:41:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:44.974 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:41:30.382838+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:41:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.980+0000 7f7848b9c700 1 -- 172.21.15.16:0/343391306 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78300a3730 msgr2=0x7f78300a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.980+0000 7f7848b9c700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78300a3730 0x7f78300a5bf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f782c009c60 tx=0x7f782c009260 comp rx=0 tx=0).stop 2024-03-27T06:41:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.981+0000 7f7848b9c700 1 -- 172.21.15.16:0/343391306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7844101f30 msgr2=0x7f784419de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.981+0000 7f7848b9c700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7844101f30 0x7f784419de70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f783400eb30 tx=0x7f783400ee40 comp rx=0 tx=0).stop 2024-03-27T06:41:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.981+0000 7f7848b9c700 1 -- 172.21.15.16:0/343391306 shutdown_connections 2024-03-27T06:41:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.981+0000 7f7848b9c700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78300a3730 0x7f78300a5bf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.981+0000 7f7848b9c700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78441075b0 0x7f7844197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.981+0000 7f7848b9c700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7844101570 0x7f784419d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.981+0000 7f7848b9c700 1 --2- 172.21.15.16:0/343391306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7844101f30 0x7f784419de70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.981+0000 7f7848b9c700 1 -- 172.21.15.16:0/343391306 >> 172.21.15.16:0/343391306 conn(0x7f7844077590 msgr2=0x7f78440ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.981+0000 7f7848b9c700 1 -- 172.21.15.16:0/343391306 shutdown_connections 2024-03-27T06:41:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:44.981+0000 7f7848b9c700 1 -- 172.21.15.16:0/343391306 wait complete. 2024-03-27T06:41:45.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:45 smithi016 ceph-mon[31908]: pgmap v298: 97 pgs: 97 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.9 KiB/s wr, 26 op/s 2024-03-27T06:41:45.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:45 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/343391306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:45.804 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-27T06:41:45.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-27T06:41:45.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:45 smithi170 ceph-mon[36438]: pgmap v298: 97 pgs: 97 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.9 KiB/s wr, 26 op/s 2024-03-27T06:41:45.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:45 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/343391306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:45 smithi156 ceph-mon[36261]: pgmap v298: 97 pgs: 97 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.9 KiB/s wr, 26 op/s 2024-03-27T06:41:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:45 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/343391306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:46.050 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:47.788 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:47 smithi016 ceph-mon[31908]: pgmap v299: 97 pgs: 97 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-03-27T06:41:47.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:47 smithi170 ceph-mon[36438]: pgmap v299: 97 pgs: 97 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-03-27T06:41:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.898+0000 7f9df07b4700 1 -- 172.21.15.16:0/1337648443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8108670 msgr2=0x7f9de8108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.898+0000 7f9df07b4700 1 --2- 172.21.15.16:0/1337648443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8108670 0x7f9de8108a50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800b0d0 tx=0x7f9dd800b3e0 comp rx=0 tx=0).stop 2024-03-27T06:41:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.898+0000 7f9df07b4700 1 -- 172.21.15.16:0/1337648443 shutdown_connections 2024-03-27T06:41:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.898+0000 7f9df07b4700 1 --2- 172.21.15.16:0/1337648443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9de8103050 0x7f9de810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.898+0000 7f9df07b4700 1 --2- 172.21.15.16:0/1337648443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8108670 0x7f9de8108a50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.898+0000 7f9df07b4700 1 --2- 172.21.15.16:0/1337648443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9de8102690 0x7f9de8102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.898+0000 7f9df07b4700 1 -- 172.21.15.16:0/1337648443 >> 172.21.15.16:0/1337648443 conn(0x7f9de8079770 msgr2=0x7f9de8079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.898+0000 7f9df07b4700 1 -- 172.21.15.16:0/1337648443 shutdown_connections 2024-03-27T06:41:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.898+0000 7f9df07b4700 1 -- 172.21.15.16:0/1337648443 wait complete. 2024-03-27T06:41:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9df07b4700 1 Processor -- start 2024-03-27T06:41:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9df07b4700 1 -- start start 2024-03-27T06:41:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9df07b4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9de8102690 0x7f9de8198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9df07b4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9de8103050 0x7f9de8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9df07b4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8108670 0x7f9de819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9df07b4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8112a30 con 0x7f9de8108670 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9df07b4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f9de81128b0 con 0x7f9de8102690 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9df07b4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8112bb0 con 0x7f9de8103050 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9deed51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8108670 0x7f9de819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9deed51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8108670 0x7f9de819d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50266/0 (socket says 172.21.15.16:50266) 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9deed51700 1 -- 172.21.15.16:0/2674667035 learned_addr learned my addr 172.21.15.16:0/2674667035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9dee550700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9de8102690 0x7f9de8198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9dedd4f700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9de8103050 0x7f9de8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9deed51700 1 -- 172.21.15.16:0/2674667035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9de8103050 msgr2=0x7f9de8199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9deed51700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9de8103050 0x7f9de8199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9deed51700 1 -- 172.21.15.16:0/2674667035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9de8102690 msgr2=0x7f9de8198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9deed51700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9de8102690 0x7f9de8198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9deed51700 1 -- 172.21.15.16:0/2674667035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de819df30 con 0x7f9de8108670 2024-03-27T06:41:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.899+0000 7f9dee550700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9de8102690 0x7f9de8198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:41:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.900+0000 7f9deed51700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8108670 0x7f9de819d7f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9de000e940 tx=0x7f9de000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.900+0000 7f9de77fe700 1 -- 172.21.15.16:0/2674667035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0004a30 con 0x7f9de8108670 2024-03-27T06:41:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.900+0000 7f9de77fe700 1 -- 172.21.15.16:0/2674667035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9de0005520 con 0x7f9de8108670 2024-03-27T06:41:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.900+0000 7f9de77fe700 1 -- 172.21.15.16:0/2674667035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de000ce80 con 0x7f9de8108670 2024-03-27T06:41:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.900+0000 7f9df07b4700 1 -- 172.21.15.16:0/2674667035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de819e220 con 0x7f9de8108670 2024-03-27T06:41:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.900+0000 7f9df07b4700 1 -- 172.21.15.16:0/2674667035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de81a5aa0 con 0x7f9de8108670 2024-03-27T06:41:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.901+0000 7f9df07b4700 1 -- 172.21.15.16:0/2674667035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de8069300 con 0x7f9de8108670 2024-03-27T06:41:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.902+0000 7f9de77fe700 1 -- 172.21.15.16:0/2674667035 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f9de0005020 con 0x7f9de8108670 2024-03-27T06:41:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.902+0000 7f9de77fe700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9dd00a3730 0x7f9dd00a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.903+0000 7f9de77fe700 1 -- 172.21.15.16:0/2674667035 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9de00cfe70 con 0x7f9de8108670 2024-03-27T06:41:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.903+0000 7f9dee550700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9dd00a3730 0x7f9dd00a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.903+0000 7f9dee550700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9dd00a3730 0x7f9dd00a5bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009e60 tx=0x7f9dd800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:47.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:47.907+0000 7f9de77fe700 1 -- 172.21.15.16:0/2674667035 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f9de009ca90 con 0x7f9de8108670 2024-03-27T06:41:47.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:47 smithi156 ceph-mon[36261]: pgmap v299: 97 pgs: 97 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-03-27T06:41:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.112+0000 7f9df07b4700 1 -- 172.21.15.16:0/2674667035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9de802dd30 con 0x7f9de8108670 2024-03-27T06:41:48.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.114+0000 7f9de77fe700 1 -- 172.21.15.16:0/2674667035 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22173 (secure 0 0 0) 0x7f9de009c390 con 0x7f9de8108670 2024-03-27T06:41:48.110 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:41:48.110 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:41:30.382838+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:41:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 -- 172.21.15.16:0/2674667035 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9dd00a3730 msgr2=0x7f9dd00a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9dd00a3730 0x7f9dd00a5bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009e60 tx=0x7f9dd800b040 comp rx=0 tx=0).stop 2024-03-27T06:41:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 -- 172.21.15.16:0/2674667035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8108670 msgr2=0x7f9de819d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8108670 0x7f9de819d7f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9de000e940 tx=0x7f9de000ec50 comp rx=0 tx=0).stop 2024-03-27T06:41:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 -- 172.21.15.16:0/2674667035 shutdown_connections 2024-03-27T06:41:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9de8108670 0x7f9de819d7f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9de8102690 0x7f9de8198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f9dd00a3730 0x7f9dd00a5bf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 --2- 172.21.15.16:0/2674667035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9de8103050 0x7f9de8199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 -- 172.21.15.16:0/2674667035 >> 172.21.15.16:0/2674667035 conn(0x7f9de8079770 msgr2=0x7f9de8101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 -- 172.21.15.16:0/2674667035 shutdown_connections 2024-03-27T06:41:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:48.116+0000 7f9df07b4700 1 -- 172.21.15.16:0/2674667035 wait complete. 2024-03-27T06:41:48.825 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:48 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2674667035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:48.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:48 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2674667035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:48.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-27T06:41:48.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-27T06:41:48.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-27T06:41:48.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-27T06:41:48.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-27T06:41:48.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-27T06:41:48.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-27T06:41:48.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-27T06:41:48.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-27T06:41:48.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-27T06:41:48.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-27T06:41:48.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-27T06:41:48.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:48 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2674667035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:41:49.521 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:49.521 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:49 smithi016 ceph-mon[31908]: pgmap v300: 97 pgs: 97 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 19 op/s 2024-03-27T06:41:49.522 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:49.568 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:49.574 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:49.639 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:49.647 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:49.647 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:49.647 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:49 smithi170 ceph-mon[36438]: pgmap v300: 97 pgs: 97 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 19 op/s 2024-03-27T06:41:49.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:49 smithi156 ceph-mon[36261]: pgmap v300: 97 pgs: 97 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 19 op/s 2024-03-27T06:41:50.977 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:41:51.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:51 smithi170 ceph-mon[36438]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-03-27T06:41:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:51 smithi016 ceph-mon[31908]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-03-27T06:41:51.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:51 smithi156 ceph-mon[36261]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-03-27T06:41:53.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:53 smithi170 ceph-mon[36438]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-03-27T06:41:53.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:53 smithi016 ceph-mon[31908]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-03-27T06:41:53.971 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:53 smithi156 ceph-mon[36261]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-03-27T06:41:54.825 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:54 smithi016 ceph-mon[31908]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-27T06:41:54.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:54 smithi170 ceph-mon[36438]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-27T06:41:54.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:54 smithi156 ceph-mon[36261]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-27T06:41:56.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:56 smithi156 ceph-mon[36261]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-27T06:41:57.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:56 smithi170 ceph-mon[36438]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-27T06:41:57.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:56 smithi016 ceph-mon[31908]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-27T06:41:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.254+0000 7f59e2724700 1 -- 172.21.15.16:0/2309745143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc101530 msgr2=0x7f59dc1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.254+0000 7f59e2724700 1 --2- 172.21.15.16:0/2309745143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc101530 0x7f59dc1019b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009990 tx=0x7f59d0009ca0 comp rx=0 tx=0).stop 2024-03-27T06:41:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.254+0000 7f59e2724700 1 -- 172.21.15.16:0/2309745143 shutdown_connections 2024-03-27T06:41:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.254+0000 7f59e2724700 1 --2- 172.21.15.16:0/2309745143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59dc101ef0 0x7f59dc10e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.254+0000 7f59e2724700 1 --2- 172.21.15.16:0/2309745143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59dc107640 0x7f59dc107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.254+0000 7f59e2724700 1 --2- 172.21.15.16:0/2309745143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc101530 0x7f59dc1019b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.254+0000 7f59e2724700 1 -- 172.21.15.16:0/2309745143 >> 172.21.15.16:0/2309745143 conn(0x7f59dc0784b0 msgr2=0x7f59dc0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.254+0000 7f59e2724700 1 -- 172.21.15.16:0/2309745143 shutdown_connections 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.254+0000 7f59e2724700 1 -- 172.21.15.16:0/2309745143 wait complete. 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.255+0000 7f59e2724700 1 Processor -- start 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.255+0000 7f59e2724700 1 -- start start 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59e2724700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59dc101530 0x7f59dc19b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59e2724700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc101ef0 0x7f59dc19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59e2724700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59dc107640 0x7f59dc19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59e2724700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc111930 con 0x7f59dc101ef0 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59e2724700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc1117b0 con 0x7f59dc101530 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59e2724700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc111ab0 con 0x7f59dc107640 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc101ef0 0x7f59dc19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59e0cc1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59dc107640 0x7f59dc19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59dbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59dc101530 0x7f59dc19b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc101ef0 0x7f59dc19b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33156/0 (socket says 172.21.15.16:33156) 2024-03-27T06:41:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59db7fe700 1 -- 172.21.15.16:0/3025535758 learned_addr learned my addr 172.21.15.16:0/3025535758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:41:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59dbfff700 1 -- 172.21.15.16:0/3025535758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59dc107640 msgr2=0x7f59dc19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59dbfff700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59dc107640 0x7f59dc19f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59dbfff700 1 -- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc101ef0 msgr2=0x7f59dc19b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59dbfff700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc101ef0 0x7f59dc19b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59dbfff700 1 -- 172.21.15.16:0/3025535758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc1a0030 con 0x7f59dc101530 2024-03-27T06:41:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59db7fe700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc101ef0 0x7f59dc19b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:41:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.256+0000 7f59e0cc1700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59dc107640 0x7f59dc19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:41:57.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.257+0000 7f59dbfff700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59dc101530 0x7f59dc19b020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00e9f0 tx=0x7f59cc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.257+0000 7f59d97fa700 1 -- 172.21.15.16:0/3025535758 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00c9c0 con 0x7f59dc101530 2024-03-27T06:41:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.257+0000 7f59d97fa700 1 -- 172.21.15.16:0/3025535758 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f59cc00cb60 con 0x7f59dc101530 2024-03-27T06:41:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.257+0000 7f59e2724700 1 -- 172.21.15.16:0/3025535758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc1a0320 con 0x7f59dc101530 2024-03-27T06:41:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.257+0000 7f59d97fa700 1 -- 172.21.15.16:0/3025535758 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc018680 con 0x7f59dc101530 2024-03-27T06:41:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.257+0000 7f59e2724700 1 -- 172.21.15.16:0/3025535758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc1a7ba0 con 0x7f59dc101530 2024-03-27T06:41:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.258+0000 7f59e2724700 1 -- 172.21.15.16:0/3025535758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59dc06b700 con 0x7f59dc101530 2024-03-27T06:41:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.259+0000 7f59d97fa700 1 -- 172.21.15.16:0/3025535758 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f59cc018a30 con 0x7f59dc101530 2024-03-27T06:41:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.260+0000 7f59d97fa700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f59c40a37b0 0x7f59c40a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.260+0000 7f59db7fe700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f59c40a37b0 0x7f59c40a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.260+0000 7f59d97fa700 1 -- 172.21.15.16:0/3025535758 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f59cc0d08c0 con 0x7f59dc101530 2024-03-27T06:41:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.260+0000 7f59d97fa700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606] conn(0x7f59c40acf90 0x7f59c40af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:41:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.260+0000 7f59d97fa700 1 -- 172.21.15.16:0/3025535758 --> [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59cc01bec0 con 0x7f59c40acf90 2024-03-27T06:41:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.260+0000 7f59d97fa700 1 -- 172.21.15.16:0/3025535758 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f59cc0d0c70 con 0x7f59dc101530 2024-03-27T06:41:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.260+0000 7f59e0cc1700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606] conn(0x7f59c40acf90 0x7f59c40af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:41:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.261+0000 7f59e0cc1700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606] conn(0x7f59c40acf90 0x7f59c40af3b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.261+0000 7f59db7fe700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f59c40a37b0 0x7f59c40a5c70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f59dc10e810 tx=0x7f59d0009590 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:41:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.262+0000 7f59d97fa700 1 -- 172.21.15.16:0/3025535758 <== osd.3 v2:172.21.15.16:6810/875136606 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f59cc01bec0 con 0x7f59c40acf90 2024-03-27T06:41:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.283+0000 7f59e2724700 1 -- 172.21.15.16:0/3025535758 --> [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59dc06a590 con 0x7f59c40acf90 2024-03-27T06:41:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.283+0000 7f59d97fa700 1 -- 172.21.15.16:0/3025535758 <== osd.3 v2:172.21.15.16:6810/875136606 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f59dc06a590 con 0x7f59c40acf90 2024-03-27T06:41:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.284+0000 7f59e2724700 1 -- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606] conn(0x7f59c40acf90 msgr2=0x7f59c40af3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.284+0000 7f59e2724700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606] conn(0x7f59c40acf90 0x7f59c40af3b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:41:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.284+0000 7f59e2724700 1 -- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f59c40a37b0 msgr2=0x7f59c40a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.284+0000 7f59e2724700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f59c40a37b0 0x7f59c40a5c70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f59dc10e810 tx=0x7f59d0009590 comp rx=0 tx=0).stop 2024-03-27T06:41:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.284+0000 7f59e2724700 1 -- 172.21.15.16:0/3025535758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59dc101530 msgr2=0x7f59dc19b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:41:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.284+0000 7f59e2724700 1 --2- 172.21.15.16:0/3025535758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59dc101530 0x7f59dc19b020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00e9f0 tx=0x7f59cc00ed00 comp rx=0 tx=0).stop 2024-03-27T06:41:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.284+0000 7f59e0cc1700 1 -- 172.21.15.16:0/3025535758 reap_dead start 2024-03-27T06:41:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.284+0000 7f59e2724700 1 -- 172.21.15.16:0/3025535758 shutdown_connections 2024-03-27T06:41:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.284+0000 7f59e2724700 1 -- 172.21.15.16:0/3025535758 >> 172.21.15.16:0/3025535758 conn(0x7f59dc0784b0 msgr2=0x7f59dc06ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:41:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.284+0000 7f59e2724700 1 -- 172.21.15.16:0/3025535758 shutdown_connections 2024-03-27T06:41:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:41:57.284+0000 7f59e2724700 1 -- 172.21.15.16:0/3025535758 wait complete. 2024-03-27T06:41:58.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:41:58 smithi156 ceph-mon[36261]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:41:59.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:41:58 smithi016 ceph-mon[31908]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:41:59.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:41:58 smithi170 ceph-mon[36438]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:42:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.623+0000 7f6349b72700 1 -- 172.21.15.16:0/1024849173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634406b540 msgr2=0x7f634410f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.623+0000 7f6349b72700 1 --2- 172.21.15.16:0/1024849173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634406b540 0x7f634410f4a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f633c00b0d0 tx=0x7f633c00b3e0 comp rx=0 tx=0).stop 2024-03-27T06:42:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.623+0000 7f6349b72700 1 -- 172.21.15.16:0/1024849173 shutdown_connections 2024-03-27T06:42:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.623+0000 7f6349b72700 1 --2- 172.21.15.16:0/1024849173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634406b540 0x7f634410f4a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.623+0000 7f6349b72700 1 --2- 172.21.15.16:0/1024849173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63441085f0 0x7f63441089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.623+0000 7f6349b72700 1 --2- 172.21.15.16:0/1024849173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f634406ab80 0x7f634406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.623+0000 7f6349b72700 1 -- 172.21.15.16:0/1024849173 >> 172.21.15.16:0/1024849173 conn(0x7f6344077590 msgr2=0x7f63440779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.623+0000 7f6349b72700 1 -- 172.21.15.16:0/1024849173 shutdown_connections 2024-03-27T06:42:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.623+0000 7f6349b72700 1 -- 172.21.15.16:0/1024849173 wait complete. 2024-03-27T06:42:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.624+0000 7f6349b72700 1 Processor -- start 2024-03-27T06:42:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.624+0000 7f6349b72700 1 -- start start 2024-03-27T06:42:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6349b72700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f634406ab80 0x7f6344101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6349b72700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634406b540 0x7f6344101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6349b72700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63441085f0 0x7f63441020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6349b72700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6344112aa0 con 0x7f634406b540 2024-03-27T06:42:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6349b72700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f6344112920 con 0x7f63441085f0 2024-03-27T06:42:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6349b72700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6344112c20 con 0x7f634406ab80 2024-03-27T06:42:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6342ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634406b540 0x7f6344101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6342ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634406b540 0x7f6344101b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33176/0 (socket says 172.21.15.16:33176) 2024-03-27T06:42:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6342ffd700 1 -- 172.21.15.16:0/2645481918 learned_addr learned my addr 172.21.15.16:0/2645481918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f63437fe700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f634406ab80 0x7f6344101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6343fff700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63441085f0 0x7f63441020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6342ffd700 1 -- 172.21.15.16:0/2645481918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f634406ab80 msgr2=0x7f6344101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6342ffd700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f634406ab80 0x7f6344101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6342ffd700 1 -- 172.21.15.16:0/2645481918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63441085f0 msgr2=0x7f63441020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6342ffd700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63441085f0 0x7f63441020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6342ffd700 1 -- 172.21.15.16:0/2645481918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63441a7340 con 0x7f634406b540 2024-03-27T06:42:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.625+0000 7f6342ffd700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634406b540 0x7f6344101b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e9a0 tx=0x7f632c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.626+0000 7f6340ff9700 1 -- 172.21.15.16:0/2645481918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c00ca10 con 0x7f634406b540 2024-03-27T06:42:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.626+0000 7f6349b72700 1 -- 172.21.15.16:0/2645481918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63441a7630 con 0x7f634406b540 2024-03-27T06:42:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.626+0000 7f6340ff9700 1 -- 172.21.15.16:0/2645481918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f632c00cbb0 con 0x7f634406b540 2024-03-27T06:42:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.626+0000 7f6340ff9700 1 -- 172.21.15.16:0/2645481918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c018710 con 0x7f634406b540 2024-03-27T06:42:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.626+0000 7f6349b72700 1 -- 172.21.15.16:0/2645481918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63441a7bc0 con 0x7f634406b540 2024-03-27T06:42:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.627+0000 7f6349b72700 1 -- 172.21.15.16:0/2645481918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6344062d50 con 0x7f634406b540 2024-03-27T06:42:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.629+0000 7f6340ff9700 1 -- 172.21.15.16:0/2645481918 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f632c010430 con 0x7f634406b540 2024-03-27T06:42:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.629+0000 7f6340ff9700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f63300a37b0 0x7f63300a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.630+0000 7f6340ff9700 1 -- 172.21.15.16:0/2645481918 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f632c0d0610 con 0x7f634406b540 2024-03-27T06:42:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.630+0000 7f63437fe700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f63300a37b0 0x7f63300a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.630+0000 7f6340ff9700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882] conn(0x7f63300acf90 0x7f63300af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.630+0000 7f6340ff9700 1 -- 172.21.15.16:0/2645481918 --> [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63300afa60 con 0x7f63300acf90 2024-03-27T06:42:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.630+0000 7f6340ff9700 1 -- 172.21.15.16:0/2645481918 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f632c0d0af0 con 0x7f634406b540 2024-03-27T06:42:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.630+0000 7f6343fff700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882] conn(0x7f63300acf90 0x7f63300af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.630+0000 7f63437fe700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f63300a37b0 0x7f63300a5c70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6334009be0 tx=0x7f6334009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.630+0000 7f6343fff700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882] conn(0x7f63300acf90 0x7f63300af3b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.631+0000 7f6340ff9700 1 -- 172.21.15.16:0/2645481918 <== osd.5 v2:172.21.15.170:6808/493340882 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63300afa60 con 0x7f63300acf90 2024-03-27T06:42:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.649+0000 7f6349b72700 1 -- 172.21.15.16:0/2645481918 --> [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6344063150 con 0x7f63300acf90 2024-03-27T06:42:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.649+0000 7f6340ff9700 1 -- 172.21.15.16:0/2645481918 <== osd.5 v2:172.21.15.170:6808/493340882 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6344063150 con 0x7f63300acf90 2024-03-27T06:42:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.650+0000 7f6349b72700 1 -- 172.21.15.16:0/2645481918 >> [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882] conn(0x7f63300acf90 msgr2=0x7f63300af3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.650+0000 7f6349b72700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882] conn(0x7f63300acf90 0x7f63300af3b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.650+0000 7f6349b72700 1 -- 172.21.15.16:0/2645481918 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f63300a37b0 msgr2=0x7f63300a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.650+0000 7f6349b72700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f63300a37b0 0x7f63300a5c70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6334009be0 tx=0x7f6334009260 comp rx=0 tx=0).stop 2024-03-27T06:42:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.650+0000 7f6349b72700 1 -- 172.21.15.16:0/2645481918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634406b540 msgr2=0x7f6344101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.650+0000 7f6349b72700 1 --2- 172.21.15.16:0/2645481918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634406b540 0x7f6344101b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e9a0 tx=0x7f632c00ecb0 comp rx=0 tx=0).stop 2024-03-27T06:42:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.650+0000 7f6343fff700 1 -- 172.21.15.16:0/2645481918 reap_dead start 2024-03-27T06:42:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.650+0000 7f6349b72700 1 -- 172.21.15.16:0/2645481918 shutdown_connections 2024-03-27T06:42:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.650+0000 7f6349b72700 1 -- 172.21.15.16:0/2645481918 >> 172.21.15.16:0/2645481918 conn(0x7f6344077590 msgr2=0x7f63441000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.650+0000 7f6349b72700 1 -- 172.21.15.16:0/2645481918 shutdown_connections 2024-03-27T06:42:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:00.650+0000 7f6349b72700 1 -- 172.21.15.16:0/2645481918 wait complete. 2024-03-27T06:42:00.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:00 smithi156 ceph-mon[36261]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:42:01.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:00 smithi170 ceph-mon[36438]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:42:01.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:00 smithi016 ceph-mon[31908]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:42:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.907+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/2667159071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b1806acb0 msgr2=0x7f0b1810f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.907+0000 7f0b1f2cd700 1 --2- 172.21.15.16:0/2667159071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b1806acb0 0x7f0b1810f4d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000b0d0 tx=0x7f0b1000b3e0 comp rx=0 tx=0).stop 2024-03-27T06:42:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.907+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/2667159071 shutdown_connections 2024-03-27T06:42:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.907+0000 7f0b1f2cd700 1 --2- 172.21.15.16:0/2667159071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b1806acb0 0x7f0b1810f4d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.907+0000 7f0b1f2cd700 1 --2- 172.21.15.16:0/2667159071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b18108670 0x7f0b18108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.907+0000 7f0b1f2cd700 1 --2- 172.21.15.16:0/2667159071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b1806a2f0 0x7f0b1806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.907+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/2667159071 >> 172.21.15.16:0/2667159071 conn(0x7f0b18078470 msgr2=0x7f0b18078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.907+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/2667159071 shutdown_connections 2024-03-27T06:42:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.907+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/2667159071 wait complete. 2024-03-27T06:42:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.907+0000 7f0b1f2cd700 1 Processor -- start 2024-03-27T06:42:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1f2cd700 1 -- start start 2024-03-27T06:42:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1f2cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b1806a2f0 0x7f0b1819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1f2cd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1806acb0 0x7f0b1819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1f2cd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b18108670 0x7f0b1819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1f2cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b18112a40 con 0x7f0b1806a2f0 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1f2cd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f0b181128c0 con 0x7f0b18108670 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1f2cd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0b18112bc0 con 0x7f0b1806acb0 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1d069700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b1806a2f0 0x7f0b1819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1d069700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b1806a2f0 0x7f0b1819aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33188/0 (socket says 172.21.15.16:33188) 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1d069700 1 -- 172.21.15.16:0/1834553783 learned_addr learned my addr 172.21.15.16:0/1834553783 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1d86a700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b18108670 0x7f0b1819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1c868700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1806acb0 0x7f0b1819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1d069700 1 -- 172.21.15.16:0/1834553783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1806acb0 msgr2=0x7f0b1819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1d069700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1806acb0 0x7f0b1819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1d069700 1 -- 172.21.15.16:0/1834553783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b18108670 msgr2=0x7f0b1819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1d069700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b18108670 0x7f0b1819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1d069700 1 -- 172.21.15.16:0/1834553783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1819ff00 con 0x7f0b1806a2f0 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1d069700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b1806a2f0 0x7f0b1819aef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800e9c0 tx=0x7f0b0800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b167fc700 1 -- 172.21.15.16:0/1834553783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800cbd0 con 0x7f0b1806a2f0 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b167fc700 1 -- 172.21.15.16:0/1834553783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0b0800ee60 con 0x7f0b1806a2f0 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/1834553783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b181a01f0 con 0x7f0b1806a2f0 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b167fc700 1 -- 172.21.15.16:0/1834553783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08010430 con 0x7f0b1806a2f0 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.908+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/1834553783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b181a7a70 con 0x7f0b1806a2f0 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.909+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/1834553783 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b1810e260 con 0x7f0b1806a2f0 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.910+0000 7f0b167fc700 1 -- 172.21.15.16:0/1834553783 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f0b08004750 con 0x7f0b1806a2f0 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.911+0000 7f0b167fc700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0b000a3730 0x7f0b000a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.911+0000 7f0b1c868700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0b000a3730 0x7f0b000a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.911+0000 7f0b167fc700 1 -- 172.21.15.16:0/1834553783 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0b080d0390 con 0x7f0b1806a2f0 2024-03-27T06:42:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.911+0000 7f0b167fc700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551] conn(0x7f0b000acf10 0x7f0b000af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:01.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.911+0000 7f0b167fc700 1 -- 172.21.15.16:0/1834553783 --> [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b0801ee50 con 0x7f0b000acf10 2024-03-27T06:42:01.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.911+0000 7f0b167fc700 1 -- 172.21.15.16:0/1834553783 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0b080d0870 con 0x7f0b1806a2f0 2024-03-27T06:42:01.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.911+0000 7f0b1d86a700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551] conn(0x7f0b000acf10 0x7f0b000af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:01.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.911+0000 7f0b1c868700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0b000a3730 0x7f0b000a5bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0b18078d00 tx=0x7f0b0c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:01.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.912+0000 7f0b1d86a700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551] conn(0x7f0b000acf10 0x7f0b000af330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:01.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.912+0000 7f0b167fc700 1 -- 172.21.15.16:0/1834553783 <== osd.8 v2:172.21.15.170:6816/504489551 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b0801ee50 con 0x7f0b000acf10 2024-03-27T06:42:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.929+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/1834553783 --> [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b18103d40 con 0x7f0b000acf10 2024-03-27T06:42:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b167fc700 1 -- 172.21.15.16:0/1834553783 <== osd.8 v2:172.21.15.170:6816/504489551 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0b18103d40 con 0x7f0b000acf10 2024-03-27T06:42:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/1834553783 >> [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551] conn(0x7f0b000acf10 msgr2=0x7f0b000af330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b1f2cd700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551] conn(0x7f0b000acf10 0x7f0b000af330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/1834553783 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0b000a3730 msgr2=0x7f0b000a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b1f2cd700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f0b000a3730 0x7f0b000a5bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0b18078d00 tx=0x7f0b0c009260 comp rx=0 tx=0).stop 2024-03-27T06:42:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/1834553783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b1806a2f0 msgr2=0x7f0b1819aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b1f2cd700 1 --2- 172.21.15.16:0/1834553783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b1806a2f0 0x7f0b1819aef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800e9c0 tx=0x7f0b0800ecd0 comp rx=0 tx=0).stop 2024-03-27T06:42:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b1d86a700 1 -- 172.21.15.16:0/1834553783 reap_dead start 2024-03-27T06:42:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/1834553783 shutdown_connections 2024-03-27T06:42:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/1834553783 >> 172.21.15.16:0/1834553783 conn(0x7f0b18078470 msgr2=0x7f0b1810d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/1834553783 shutdown_connections 2024-03-27T06:42:01.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:01.930+0000 7f0b1f2cd700 1 -- 172.21.15.16:0/1834553783 wait complete. 2024-03-27T06:42:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:02.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:02 smithi016 ceph-mon[31908]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-27T06:42:02.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:02 smithi170 ceph-mon[36438]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-27T06:42:02.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:02 smithi156 ceph-mon[36261]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-27T06:42:04.168 INFO:teuthology.orchestra.run.smithi016.stdout:167503724594 2024-03-27T06:42:04.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-27T06:42:04.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:04 smithi016 ceph-mon[31908]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T06:42:04.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:04 smithi156 ceph-mon[36261]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T06:42:05.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:04 smithi170 ceph-mon[36438]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T06:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.496+0000 7fd47c2c0700 1 -- 172.21.15.16:0/3396260981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd474101ee0 msgr2=0x7fd47410e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.496+0000 7fd47c2c0700 1 --2- 172.21.15.16:0/3396260981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd474101ee0 0x7fd47410e3d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0099c0 tx=0x7fd46c009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.496+0000 7fd47c2c0700 1 -- 172.21.15.16:0/3396260981 shutdown_connections 2024-03-27T06:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.496+0000 7fd47c2c0700 1 --2- 172.21.15.16:0/3396260981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd474101ee0 0x7fd47410e3d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.496+0000 7fd47c2c0700 1 --2- 172.21.15.16:0/3396260981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4741064d0 0x7fd4741068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.496+0000 7fd47c2c0700 1 --2- 172.21.15.16:0/3396260981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd474101520 0x7fd4741019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.496+0000 7fd47c2c0700 1 -- 172.21.15.16:0/3396260981 >> 172.21.15.16:0/3396260981 conn(0x7fd474078550 msgr2=0x7fd474078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.496+0000 7fd47c2c0700 1 -- 172.21.15.16:0/3396260981 shutdown_connections 2024-03-27T06:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.496+0000 7fd47c2c0700 1 -- 172.21.15.16:0/3396260981 wait complete. 2024-03-27T06:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.496+0000 7fd47c2c0700 1 Processor -- start 2024-03-27T06:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47c2c0700 1 -- start start 2024-03-27T06:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47c2c0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd474101520 0x7fd474198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47c2c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd474101ee0 0x7fd474199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47c2c0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4741064d0 0x7fd47419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47c2c0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd474111970 con 0x7fd474101ee0 2024-03-27T06:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47c2c0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741117f0 con 0x7fd474101520 2024-03-27T06:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47c2c0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd474111af0 con 0x7fd4741064d0 2024-03-27T06:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47985b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd474101ee0 0x7fd474199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47985b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd474101ee0 0x7fd474199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33212/0 (socket says 172.21.15.16:33212) 2024-03-27T06:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47985b700 1 -- 172.21.15.16:0/1159874515 learned_addr learned my addr 172.21.15.16:0/1159874515 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47a05c700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd474101520 0x7fd474198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47a85d700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4741064d0 0x7fd47419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47985b700 1 -- 172.21.15.16:0/1159874515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4741064d0 msgr2=0x7fd47419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47985b700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4741064d0 0x7fd47419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47985b700 1 -- 172.21.15.16:0/1159874515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd474101520 msgr2=0x7fd474198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47985b700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd474101520 0x7fd474198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47985b700 1 -- 172.21.15.16:0/1159874515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47419de50 con 0x7fd474101ee0 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.497+0000 7fd47985b700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd474101ee0 0x7fd474199380 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd46800d980 tx=0x7fd46800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.498+0000 7fd4737fe700 1 -- 172.21.15.16:0/1159874515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468016070 con 0x7fd474101ee0 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.498+0000 7fd47c2c0700 1 -- 172.21.15.16:0/1159874515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47419e140 con 0x7fd474101ee0 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.498+0000 7fd4737fe700 1 -- 172.21.15.16:0/1159874515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd468004d30 con 0x7fd474101ee0 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.498+0000 7fd4737fe700 1 -- 172.21.15.16:0/1159874515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4680056c0 con 0x7fd474101ee0 2024-03-27T06:42:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.498+0000 7fd47c2c0700 1 -- 172.21.15.16:0/1159874515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4741a59c0 con 0x7fd474101ee0 2024-03-27T06:42:05.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.499+0000 7fd47c2c0700 1 -- 172.21.15.16:0/1159874515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd474062dd0 con 0x7fd474101ee0 2024-03-27T06:42:05.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.500+0000 7fd4737fe700 1 -- 172.21.15.16:0/1159874515 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fd468004220 con 0x7fd474101ee0 2024-03-27T06:42:05.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.500+0000 7fd4737fe700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd45c0a3610 0x7fd45c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:05.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.500+0000 7fd47a05c700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd45c0a3610 0x7fd45c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:05.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.500+0000 7fd4737fe700 1 -- 172.21.15.16:0/1159874515 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd4680cf850 con 0x7fd474101ee0 2024-03-27T06:42:05.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.500+0000 7fd4737fe700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379] conn(0x7fd45c0acdf0 0x7fd45c0af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:05.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.500+0000 7fd4737fe700 1 -- 172.21.15.16:0/1159874515 --> [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd468012e70 con 0x7fd45c0acdf0 2024-03-27T06:42:05.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.500+0000 7fd4737fe700 1 -- 172.21.15.16:0/1159874515 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd4680d42c0 con 0x7fd474101ee0 2024-03-27T06:42:05.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.501+0000 7fd47a05c700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd45c0a3610 0x7fd45c0a5ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd474102730 tx=0x7fd46400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:05.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.501+0000 7fd47a85d700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379] conn(0x7fd45c0acdf0 0x7fd45c0af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:05.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.501+0000 7fd47a85d700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379] conn(0x7fd45c0acdf0 0x7fd45c0af210 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:05.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.502+0000 7fd4737fe700 1 -- 172.21.15.16:0/1159874515 <== osd.2 v2:172.21.15.170:6800/3044432379 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd468012e70 con 0x7fd45c0acdf0 2024-03-27T06:42:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.519+0000 7fd47c2c0700 1 -- 172.21.15.16:0/1159874515 --> [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd474063250 con 0x7fd45c0acdf0 2024-03-27T06:42:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.519+0000 7fd4737fe700 1 -- 172.21.15.16:0/1159874515 <== osd.2 v2:172.21.15.170:6800/3044432379 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd474063250 con 0x7fd45c0acdf0 2024-03-27T06:42:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.519+0000 7fd47c2c0700 1 -- 172.21.15.16:0/1159874515 >> [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379] conn(0x7fd45c0acdf0 msgr2=0x7fd45c0af210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.519+0000 7fd47c2c0700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379] conn(0x7fd45c0acdf0 0x7fd45c0af210 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.519+0000 7fd47c2c0700 1 -- 172.21.15.16:0/1159874515 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd45c0a3610 msgr2=0x7fd45c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.519+0000 7fd47c2c0700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd45c0a3610 0x7fd45c0a5ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd474102730 tx=0x7fd46400b040 comp rx=0 tx=0).stop 2024-03-27T06:42:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.519+0000 7fd47c2c0700 1 -- 172.21.15.16:0/1159874515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd474101ee0 msgr2=0x7fd474199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.519+0000 7fd47c2c0700 1 --2- 172.21.15.16:0/1159874515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd474101ee0 0x7fd474199380 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd46800d980 tx=0x7fd46800dc90 comp rx=0 tx=0).stop 2024-03-27T06:42:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.519+0000 7fd47a85d700 1 -- 172.21.15.16:0/1159874515 reap_dead start 2024-03-27T06:42:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.519+0000 7fd47c2c0700 1 -- 172.21.15.16:0/1159874515 shutdown_connections 2024-03-27T06:42:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.519+0000 7fd47c2c0700 1 -- 172.21.15.16:0/1159874515 >> 172.21.15.16:0/1159874515 conn(0x7fd474078550 msgr2=0x7fd47406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.520+0000 7fd47c2c0700 1 -- 172.21.15.16:0/1159874515 shutdown_connections 2024-03-27T06:42:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:05.520+0000 7fd47c2c0700 1 -- 172.21.15.16:0/1159874515 wait complete. 2024-03-27T06:42:06.118 INFO:teuthology.orchestra.run.smithi016.stdout:257698037795 2024-03-27T06:42:06.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-27T06:42:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.399+0000 7f020ad03700 1 -- 172.21.15.16:0/417351760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0204101d40 msgr2=0x7f02041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.399+0000 7f020ad03700 1 --2- 172.21.15.16:0/417351760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0204101d40 0x7f02041021c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f01f00099c0 tx=0x7f01f0009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.399+0000 7f020ad03700 1 -- 172.21.15.16:0/417351760 shutdown_connections 2024-03-27T06:42:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.399+0000 7f020ad03700 1 --2- 172.21.15.16:0/417351760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0204102700 0x7f020410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.399+0000 7f020ad03700 1 --2- 172.21.15.16:0/417351760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0204107650 0x7f0204107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.399+0000 7f020ad03700 1 --2- 172.21.15.16:0/417351760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0204101d40 0x7f02041021c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.399+0000 7f020ad03700 1 -- 172.21.15.16:0/417351760 >> 172.21.15.16:0/417351760 conn(0x7f0204077b40 msgr2=0x7f02040fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:06.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f020ad03700 1 -- 172.21.15.16:0/417351760 shutdown_connections 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f020ad03700 1 -- 172.21.15.16:0/417351760 wait complete. 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f020ad03700 1 Processor -- start 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f020ad03700 1 -- start start 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f020ad03700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0204101d40 0x7f0204103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f020ad03700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0204102700 0x7f0204103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f020ad03700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0204107650 0x7f0204103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f020ad03700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02041119d0 con 0x7f0204107650 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f020ad03700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f0204111850 con 0x7f0204101d40 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f020ad03700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0204111b50 con 0x7f0204102700 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f02092a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0204107650 0x7f0204103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.400+0000 7f0208a9f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0204101d40 0x7f0204103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f02092a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0204107650 0x7f0204103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53966/0 (socket says 172.21.15.16:53966) 2024-03-27T06:42:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f0208a9f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0204101d40 0x7f0204103210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:49432/0 (socket says 172.21.15.16:49432) 2024-03-27T06:42:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f02092a0700 1 -- 172.21.15.16:0/607772842 learned_addr learned my addr 172.21.15.16:0/607772842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f0203fff700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0204102700 0x7f0204103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f02092a0700 1 -- 172.21.15.16:0/607772842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0204102700 msgr2=0x7f0204103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f02092a0700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0204102700 0x7f0204103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f02092a0700 1 -- 172.21.15.16:0/607772842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0204101d40 msgr2=0x7f0204103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f02092a0700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0204101d40 0x7f0204103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f02092a0700 1 -- 172.21.15.16:0/607772842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02041a7370 con 0x7f0204107650 2024-03-27T06:42:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f02092a0700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0204107650 0x7f0204103c90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00ef30 tx=0x7f01fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f0201ffb700 1 -- 172.21.15.16:0/607772842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0092e0 con 0x7f0204107650 2024-03-27T06:42:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f020ad03700 1 -- 172.21.15.16:0/607772842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02041a7600 con 0x7f0204107650 2024-03-27T06:42:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f0201ffb700 1 -- 172.21.15.16:0/607772842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f01fc004bb0 con 0x7f0204107650 2024-03-27T06:42:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f0201ffb700 1 -- 172.21.15.16:0/607772842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc004280 con 0x7f0204107650 2024-03-27T06:42:06.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.401+0000 7f020ad03700 1 -- 172.21.15.16:0/607772842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041a7b60 con 0x7f0204107650 2024-03-27T06:42:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.402+0000 7f020ad03700 1 -- 172.21.15.16:0/607772842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0204105600 con 0x7f0204107650 2024-03-27T06:42:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.403+0000 7f0201ffb700 1 -- 172.21.15.16:0/607772842 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f01fc01b030 con 0x7f0204107650 2024-03-27T06:42:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.404+0000 7f0201ffb700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f01f40a3730 0x7f01f40a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.404+0000 7f0208a9f700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f01f40a3730 0x7f01f40a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.404+0000 7f0201ffb700 1 -- 172.21.15.16:0/607772842 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f01fc0d01f0 con 0x7f0204107650 2024-03-27T06:42:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.404+0000 7f0201ffb700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547] conn(0x7f01f40acf10 0x7f01f40af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.404+0000 7f0201ffb700 1 -- 172.21.15.16:0/607772842 --> [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01fc01fe50 con 0x7f01f40acf10 2024-03-27T06:42:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.404+0000 7f0201ffb700 1 -- 172.21.15.16:0/607772842 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f01fc0d05a0 con 0x7f0204107650 2024-03-27T06:42:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.404+0000 7f0203fff700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547] conn(0x7f01f40acf10 0x7f01f40af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.404+0000 7f0208a9f700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f01f40a3730 0x7f01f40a5bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009be0 tx=0x7f01f8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.405+0000 7f0203fff700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547] conn(0x7f01f40acf10 0x7f01f40af330 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:06.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.405+0000 7f0201ffb700 1 -- 172.21.15.16:0/607772842 <== osd.6 v2:172.21.15.16:6818/237875547 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01fc01fe50 con 0x7f01f40acf10 2024-03-27T06:42:06.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.422+0000 7f020ad03700 1 -- 172.21.15.16:0/607772842 --> [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0204102d50 con 0x7f01f40acf10 2024-03-27T06:42:06.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.422+0000 7f0201ffb700 1 -- 172.21.15.16:0/607772842 <== osd.6 v2:172.21.15.16:6818/237875547 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0204102d50 con 0x7f01f40acf10 2024-03-27T06:42:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.422+0000 7f020ad03700 1 -- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547] conn(0x7f01f40acf10 msgr2=0x7f01f40af330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.422+0000 7f020ad03700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547] conn(0x7f01f40acf10 0x7f01f40af330 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.422+0000 7f020ad03700 1 -- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f01f40a3730 msgr2=0x7f01f40a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.422+0000 7f020ad03700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f01f40a3730 0x7f01f40a5bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009be0 tx=0x7f01f8009260 comp rx=0 tx=0).stop 2024-03-27T06:42:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.422+0000 7f020ad03700 1 -- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0204107650 msgr2=0x7f0204103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.422+0000 7f020ad03700 1 --2- 172.21.15.16:0/607772842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0204107650 0x7f0204103c90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00ef30 tx=0x7f01fc00c3a0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.422+0000 7f02092a0700 1 -- 172.21.15.16:0/607772842 reap_dead start 2024-03-27T06:42:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.422+0000 7f020ad03700 1 -- 172.21.15.16:0/607772842 shutdown_connections 2024-03-27T06:42:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.422+0000 7f020ad03700 1 -- 172.21.15.16:0/607772842 >> 172.21.15.16:0/607772842 conn(0x7f0204077b40 msgr2=0x7f02040ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.423+0000 7f020ad03700 1 -- 172.21.15.16:0/607772842 shutdown_connections 2024-03-27T06:42:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.423+0000 7f020ad03700 1 -- 172.21.15.16:0/607772842 wait complete. 2024-03-27T06:42:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.596+0000 7fd3fc19d700 1 -- 172.21.15.16:0/2784414885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3f41075b0 msgr2=0x7fd3f4107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.596+0000 7fd3fc19d700 1 --2- 172.21.15.16:0/2784414885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3f41075b0 0x7fd3f4107990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40099c0 tx=0x7fd3e4009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.596+0000 7fd3fc19d700 1 -- 172.21.15.16:0/2784414885 shutdown_connections 2024-03-27T06:42:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.596+0000 7fd3fc19d700 1 --2- 172.21.15.16:0/2784414885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4101f30 0x7fd3f410e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.596+0000 7fd3fc19d700 1 --2- 172.21.15.16:0/2784414885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3f41075b0 0x7fd3f4107990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.596+0000 7fd3fc19d700 1 --2- 172.21.15.16:0/2784414885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4101570 0x7fd3f41019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.596+0000 7fd3fc19d700 1 -- 172.21.15.16:0/2784414885 >> 172.21.15.16:0/2784414885 conn(0x7fd3f4077590 msgr2=0x7fd3f40779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.596+0000 7fd3fc19d700 1 -- 172.21.15.16:0/2784414885 shutdown_connections 2024-03-27T06:42:06.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.596+0000 7fd3fc19d700 1 -- 172.21.15.16:0/2784414885 wait complete. 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3fc19d700 1 Processor -- start 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3fc19d700 1 -- start start 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3fc19d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4101570 0x7fd3f419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3fc19d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4101f30 0x7fd3f419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3fc19d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3f41075b0 0x7fd3f4197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3fc19d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41118f0 con 0x7fd3f4101f30 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3fc19d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4111770 con 0x7fd3f41075b0 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3fc19d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4111a70 con 0x7fd3f4101570 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3f9738700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4101f30 0x7fd3f419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3f9738700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4101f30 0x7fd3f419df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53980/0 (socket says 172.21.15.16:53980) 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3f9738700 1 -- 172.21.15.16:0/4232672155 learned_addr learned my addr 172.21.15.16:0/4232672155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3f9f39700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4101570 0x7fd3f419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.597+0000 7fd3fa73a700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3f41075b0 0x7fd3f4197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.598+0000 7fd3f9738700 1 -- 172.21.15.16:0/4232672155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4101570 msgr2=0x7fd3f419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.598+0000 7fd3f9738700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3f4101570 0x7fd3f419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.598+0000 7fd3f9738700 1 -- 172.21.15.16:0/4232672155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3f41075b0 msgr2=0x7fd3f4197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.598+0000 7fd3f9738700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3f41075b0 0x7fd3f4197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.598+0000 7fd3f9738700 1 -- 172.21.15.16:0/4232672155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f4198450 con 0x7fd3f4101f30 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.598+0000 7fd3f9738700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4101f30 0x7fd3f419df90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800ec40 tx=0x7fd3e800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.598+0000 7fd3f2ffd700 1 -- 172.21.15.16:0/4232672155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800ba80 con 0x7fd3f4101f30 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.598+0000 7fd3fc19d700 1 -- 172.21.15.16:0/4232672155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f41986e0 con 0x7fd3f4101f30 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.598+0000 7fd3f2ffd700 1 -- 172.21.15.16:0/4232672155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd3e8004520 con 0x7fd3f4101f30 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.598+0000 7fd3f2ffd700 1 -- 172.21.15.16:0/4232672155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8017410 con 0x7fd3f4101f30 2024-03-27T06:42:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.598+0000 7fd3fc19d700 1 -- 172.21.15.16:0/4232672155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f406ad50 con 0x7fd3f4101f30 2024-03-27T06:42:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.599+0000 7fd3fc19d700 1 -- 172.21.15.16:0/4232672155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3f406b2c0 con 0x7fd3f4101f30 2024-03-27T06:42:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.600+0000 7fd3f2ffd700 1 -- 172.21.15.16:0/4232672155 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fd3e8003ea0 con 0x7fd3f4101f30 2024-03-27T06:42:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.600+0000 7fd3f2ffd700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd3dc0a3730 0x7fd3dc0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.601+0000 7fd3f9f39700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd3dc0a3730 0x7fd3dc0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.601+0000 7fd3f2ffd700 1 -- 172.21.15.16:0/4232672155 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd3e80d1ba0 con 0x7fd3f4101f30 2024-03-27T06:42:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.601+0000 7fd3f2ffd700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561] conn(0x7fd3dc0acf10 0x7fd3dc0af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.601+0000 7fd3f2ffd700 1 -- 172.21.15.16:0/4232672155 --> [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3e800fe80 con 0x7fd3dc0acf10 2024-03-27T06:42:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.601+0000 7fd3f2ffd700 1 -- 172.21.15.16:0/4232672155 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd3e80d1f50 con 0x7fd3f4101f30 2024-03-27T06:42:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.601+0000 7fd3fa73a700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561] conn(0x7fd3dc0acf10 0x7fd3dc0af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.601+0000 7fd3f9f39700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd3dc0a3730 0x7fd3dc0a5bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4000c00 tx=0x7fd3e4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.601+0000 7fd3fa73a700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561] conn(0x7fd3dc0acf10 0x7fd3dc0af330 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.602+0000 7fd3f2ffd700 1 -- 172.21.15.16:0/4232672155 <== osd.10 v2:172.21.15.156:6824/399510561 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd3e800fe80 con 0x7fd3dc0acf10 2024-03-27T06:42:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.619+0000 7fd3fc19d700 1 -- 172.21.15.16:0/4232672155 --> [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd3f406b6c0 con 0x7fd3dc0acf10 2024-03-27T06:42:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.619+0000 7fd3f2ffd700 1 -- 172.21.15.16:0/4232672155 <== osd.10 v2:172.21.15.156:6824/399510561 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd3f406b6c0 con 0x7fd3dc0acf10 2024-03-27T06:42:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.620+0000 7fd3fc19d700 1 -- 172.21.15.16:0/4232672155 >> [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561] conn(0x7fd3dc0acf10 msgr2=0x7fd3dc0af330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.620+0000 7fd3fc19d700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561] conn(0x7fd3dc0acf10 0x7fd3dc0af330 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.620+0000 7fd3fc19d700 1 -- 172.21.15.16:0/4232672155 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd3dc0a3730 msgr2=0x7fd3dc0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.620+0000 7fd3fc19d700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd3dc0a3730 0x7fd3dc0a5bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4000c00 tx=0x7fd3e4005fb0 comp rx=0 tx=0).stop 2024-03-27T06:42:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.620+0000 7fd3fc19d700 1 -- 172.21.15.16:0/4232672155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4101f30 msgr2=0x7fd3f419df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.620+0000 7fd3fc19d700 1 --2- 172.21.15.16:0/4232672155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3f4101f30 0x7fd3f419df90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800ec40 tx=0x7fd3e800ef50 comp rx=0 tx=0).stop 2024-03-27T06:42:06.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.620+0000 7fd3fa73a700 1 -- 172.21.15.16:0/4232672155 reap_dead start 2024-03-27T06:42:06.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.620+0000 7fd3fc19d700 1 -- 172.21.15.16:0/4232672155 shutdown_connections 2024-03-27T06:42:06.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.620+0000 7fd3fc19d700 1 -- 172.21.15.16:0/4232672155 >> 172.21.15.16:0/4232672155 conn(0x7fd3f4077590 msgr2=0x7fd3f4100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:06.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.620+0000 7fd3fc19d700 1 -- 172.21.15.16:0/4232672155 shutdown_connections 2024-03-27T06:42:06.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.620+0000 7fd3fc19d700 1 -- 172.21.15.16:0/4232672155 wait complete. 2024-03-27T06:42:06.706 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:06 smithi016 ceph-mon[31908]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T06:42:06.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:06 smithi156 ceph-mon[36261]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T06:42:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.298+0000 7f6d446af700 1 -- 172.21.15.16:0/1902509623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c101840 msgr2=0x7f6d3c107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.298+0000 7f6d446af700 1 --2- 172.21.15.16:0/1902509623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c101840 0x7f6d3c107290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0099c0 tx=0x7f6d2c009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.298+0000 7f6d446af700 1 -- 172.21.15.16:0/1902509623 shutdown_connections 2024-03-27T06:42:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.298+0000 7f6d446af700 1 --2- 172.21.15.16:0/1902509623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3c1079c0 0x7f6d3c10f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.298+0000 7f6d446af700 1 --2- 172.21.15.16:0/1902509623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d3c100e90 0x7f6d3c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.298+0000 7f6d446af700 1 --2- 172.21.15.16:0/1902509623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c101840 0x7f6d3c107290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.298+0000 7f6d446af700 1 -- 172.21.15.16:0/1902509623 >> 172.21.15.16:0/1902509623 conn(0x7f6d3c077b40 msgr2=0x7f6d3c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.298+0000 7f6d446af700 1 -- 172.21.15.16:0/1902509623 shutdown_connections 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.298+0000 7f6d446af700 1 -- 172.21.15.16:0/1902509623 wait complete. 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.298+0000 7f6d446af700 1 Processor -- start 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.298+0000 7f6d446af700 1 -- start start 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d446af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c100e90 0x7f6d3c101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d446af700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d3c101840 0x7f6d3c102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d446af700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3c1079c0 0x7f6d3c102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d446af700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c112a90 con 0x7f6d3c100e90 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d446af700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c112910 con 0x7f6d3c101840 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d446af700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c112c10 con 0x7f6d3c1079c0 2024-03-27T06:42:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d4244b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c100e90 0x7f6d3c101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d42c4c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3c1079c0 0x7f6d3c102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d4244b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c100e90 0x7f6d3c101e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53950/0 (socket says 172.21.15.16:53950) 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d4244b700 1 -- 172.21.15.16:0/236251080 learned_addr learned my addr 172.21.15.16:0/236251080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d41c4a700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d3c101840 0x7f6d3c102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d4244b700 1 -- 172.21.15.16:0/236251080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3c1079c0 msgr2=0x7f6d3c102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d4244b700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3c1079c0 0x7f6d3c102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d4244b700 1 -- 172.21.15.16:0/236251080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d3c101840 msgr2=0x7f6d3c102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d4244b700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d3c101840 0x7f6d3c102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d4244b700 1 -- 172.21.15.16:0/236251080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c1a7400 con 0x7f6d3c100e90 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d42c4c700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d3c1079c0 0x7f6d3c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d4244b700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c100e90 0x7f6d3c101e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800e9c0 tx=0x7f6d3800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.299+0000 7f6d337fe700 1 -- 172.21.15.16:0/236251080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3800cbd0 con 0x7f6d3c100e90 2024-03-27T06:42:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.300+0000 7f6d446af700 1 -- 172.21.15.16:0/236251080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c1a76f0 con 0x7f6d3c100e90 2024-03-27T06:42:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.300+0000 7f6d337fe700 1 -- 172.21.15.16:0/236251080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6d3800ee60 con 0x7f6d3c100e90 2024-03-27T06:42:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.300+0000 7f6d337fe700 1 -- 172.21.15.16:0/236251080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38005670 con 0x7f6d3c100e90 2024-03-27T06:42:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.300+0000 7f6d446af700 1 -- 172.21.15.16:0/236251080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c1a7c00 con 0x7f6d3c100e90 2024-03-27T06:42:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.301+0000 7f6d446af700 1 -- 172.21.15.16:0/236251080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d3c062d50 con 0x7f6d3c100e90 2024-03-27T06:42:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.302+0000 7f6d337fe700 1 -- 172.21.15.16:0/236251080 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f6d38004750 con 0x7f6d3c100e90 2024-03-27T06:42:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.302+0000 7f6d337fe700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6d280a3730 0x7f6d280a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.303+0000 7f6d337fe700 1 -- 172.21.15.16:0/236251080 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6d380d0420 con 0x7f6d3c100e90 2024-03-27T06:42:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.303+0000 7f6d41c4a700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6d280a3730 0x7f6d280a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.303+0000 7f6d337fe700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476] conn(0x7f6d280acf10 0x7f6d280af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.303+0000 7f6d337fe700 1 -- 172.21.15.16:0/236251080 --> [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d38004500 con 0x7f6d280acf10 2024-03-27T06:42:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.303+0000 7f6d337fe700 1 -- 172.21.15.16:0/236251080 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f6d380d0900 con 0x7f6d3c100e90 2024-03-27T06:42:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.303+0000 7f6d42c4c700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476] conn(0x7f6d280acf10 0x7f6d280af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.303+0000 7f6d41c4a700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6d280a3730 0x7f6d280a5bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c103850 tx=0x7f6d2c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.304+0000 7f6d42c4c700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476] conn(0x7f6d280acf10 0x7f6d280af330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.304+0000 7f6d337fe700 1 -- 172.21.15.16:0/236251080 <== osd.7 v2:172.21.15.156:6816/229134476 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d38004500 con 0x7f6d280acf10 2024-03-27T06:42:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.321+0000 7f6d446af700 1 -- 172.21.15.16:0/236251080 --> [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d3c062fe0 con 0x7f6d280acf10 2024-03-27T06:42:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.321+0000 7f6d337fe700 1 -- 172.21.15.16:0/236251080 <== osd.7 v2:172.21.15.156:6816/229134476 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6d3c062fe0 con 0x7f6d280acf10 2024-03-27T06:42:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.321+0000 7f6d446af700 1 -- 172.21.15.16:0/236251080 >> [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476] conn(0x7f6d280acf10 msgr2=0x7f6d280af330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.321+0000 7f6d446af700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476] conn(0x7f6d280acf10 0x7f6d280af330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.322+0000 7f6d446af700 1 -- 172.21.15.16:0/236251080 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6d280a3730 msgr2=0x7f6d280a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.322+0000 7f6d446af700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f6d280a3730 0x7f6d280a5bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c103850 tx=0x7f6d2c005fb0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.322+0000 7f6d446af700 1 -- 172.21.15.16:0/236251080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c100e90 msgr2=0x7f6d3c101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.322+0000 7f6d446af700 1 --2- 172.21.15.16:0/236251080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c100e90 0x7f6d3c101e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800e9c0 tx=0x7f6d3800ecd0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.322+0000 7f6d42c4c700 1 -- 172.21.15.16:0/236251080 reap_dead start 2024-03-27T06:42:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.322+0000 7f6d446af700 1 -- 172.21.15.16:0/236251080 shutdown_connections 2024-03-27T06:42:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.322+0000 7f6d446af700 1 -- 172.21.15.16:0/236251080 >> 172.21.15.16:0/236251080 conn(0x7f6d3c077b40 msgr2=0x7f6d3c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.322+0000 7f6d446af700 1 -- 172.21.15.16:0/236251080 shutdown_connections 2024-03-27T06:42:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:06.322+0000 7f6d446af700 1 -- 172.21.15.16:0/236251080 wait complete. 2024-03-27T06:42:07.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:06 smithi170 ceph-mon[36438]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T06:42:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.512+0000 7ffa1639b700 1 -- 172.21.15.16:0/2801330711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1006b640 msgr2=0x7ffa1010ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.512+0000 7ffa1639b700 1 --2- 172.21.15.16:0/2801330711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1006b640 0x7ffa1010ec10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009990 tx=0x7ffa04009ca0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.512+0000 7ffa1639b700 1 -- 172.21.15.16:0/2801330711 shutdown_connections 2024-03-27T06:42:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.512+0000 7ffa1639b700 1 --2- 172.21.15.16:0/2801330711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa1010f150 0x7ffa10111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.512+0000 7ffa1639b700 1 --2- 172.21.15.16:0/2801330711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1006ad20 0x7ffa1006b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.512+0000 7ffa1639b700 1 --2- 172.21.15.16:0/2801330711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1006b640 0x7ffa1010ec10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.512+0000 7ffa1639b700 1 -- 172.21.15.16:0/2801330711 >> 172.21.15.16:0/2801330711 conn(0x7ffa10077590 msgr2=0x7ffa100779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.512+0000 7ffa1639b700 1 -- 172.21.15.16:0/2801330711 shutdown_connections 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.512+0000 7ffa1639b700 1 -- 172.21.15.16:0/2801330711 wait complete. 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa1639b700 1 Processor -- start 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa1639b700 1 -- start start 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa1639b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1006ad20 0x7ffa1019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa1639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1006b640 0x7ffa1019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa1639b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa1010f150 0x7ffa1019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa1639b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10114ce0 con 0x7ffa1006ad20 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa1639b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10114b60 con 0x7ffa1006b640 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa1639b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10114e60 con 0x7ffa1010f150 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa0ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1006ad20 0x7ffa1019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa0f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1006b640 0x7ffa1019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa0ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1006ad20 0x7ffa1019b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54002/0 (socket says 172.21.15.16:54002) 2024-03-27T06:42:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa0ffff700 1 -- 172.21.15.16:0/2514186370 learned_addr learned my addr 172.21.15.16:0/2514186370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.513+0000 7ffa14938700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa1010f150 0x7ffa1019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa0ffff700 1 -- 172.21.15.16:0/2514186370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa1010f150 msgr2=0x7ffa1019fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa0ffff700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa1010f150 0x7ffa1019fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa0ffff700 1 -- 172.21.15.16:0/2514186370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1006b640 msgr2=0x7ffa1019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa0ffff700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1006b640 0x7ffa1019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa0ffff700 1 -- 172.21.15.16:0/2514186370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa101a0140 con 0x7ffa1006ad20 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa0f7fe700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1006b640 0x7ffa1019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa0ffff700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1006ad20 0x7ffa1019b130 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000e9a0 tx=0x7ffa0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa0d7fa700 1 -- 172.21.15.16:0/2514186370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0000ca10 con 0x7ffa1006ad20 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa0d7fa700 1 -- 172.21.15.16:0/2514186370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffa0000cbb0 con 0x7ffa1006ad20 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa1639b700 1 -- 172.21.15.16:0/2514186370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa101a0430 con 0x7ffa1006ad20 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa0d7fa700 1 -- 172.21.15.16:0/2514186370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00018710 con 0x7ffa1006ad20 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.514+0000 7ffa1639b700 1 -- 172.21.15.16:0/2514186370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa101a7cb0 con 0x7ffa1006ad20 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.515+0000 7ffa1639b700 1 -- 172.21.15.16:0/2514186370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa10062d10 con 0x7ffa1006ad20 2024-03-27T06:42:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.517+0000 7ffa0d7fa700 1 -- 172.21.15.16:0/2514186370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7ffa00010430 con 0x7ffa1006ad20 2024-03-27T06:42:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.517+0000 7ffa0d7fa700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff9f80a3730 0x7ff9f80a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.517+0000 7ffa0f7fe700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff9f80a3730 0x7ff9f80a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.517+0000 7ffa0d7fa700 1 -- 172.21.15.16:0/2514186370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ffa000d0690 con 0x7ffa1006ad20 2024-03-27T06:42:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.517+0000 7ffa0d7fa700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184] conn(0x7ff9f80acf10 0x7ff9f80af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.517+0000 7ffa0d7fa700 1 -- 172.21.15.16:0/2514186370 --> [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa0001bec0 con 0x7ff9f80acf10 2024-03-27T06:42:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.517+0000 7ffa0d7fa700 1 -- 172.21.15.16:0/2514186370 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ffa000d0b70 con 0x7ffa1006ad20 2024-03-27T06:42:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.518+0000 7ffa14938700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184] conn(0x7ff9f80acf10 0x7ff9f80af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.518+0000 7ffa0f7fe700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff9f80a3730 0x7ff9f80a5bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400b2e0 tx=0x7ffa0401a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.518+0000 7ffa14938700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184] conn(0x7ff9f80acf10 0x7ff9f80af330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.519+0000 7ffa0d7fa700 1 -- 172.21.15.16:0/2514186370 <== osd.9 v2:172.21.15.16:6826/2308288184 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa0001bec0 con 0x7ff9f80acf10 2024-03-27T06:42:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.535+0000 7ffa1639b700 1 -- 172.21.15.16:0/2514186370 --> [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa10063190 con 0x7ff9f80acf10 2024-03-27T06:42:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.535+0000 7ffa0d7fa700 1 -- 172.21.15.16:0/2514186370 <== osd.9 v2:172.21.15.16:6826/2308288184 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffa10063190 con 0x7ff9f80acf10 2024-03-27T06:42:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.536+0000 7ffa1639b700 1 -- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184] conn(0x7ff9f80acf10 msgr2=0x7ff9f80af330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.536+0000 7ffa1639b700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184] conn(0x7ff9f80acf10 0x7ff9f80af330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.536+0000 7ffa1639b700 1 -- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff9f80a3730 msgr2=0x7ff9f80a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.536+0000 7ffa1639b700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff9f80a3730 0x7ff9f80a5bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400b2e0 tx=0x7ffa0401a040 comp rx=0 tx=0).stop 2024-03-27T06:42:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.536+0000 7ffa1639b700 1 -- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1006ad20 msgr2=0x7ffa1019b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.536+0000 7ffa1639b700 1 --2- 172.21.15.16:0/2514186370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1006ad20 0x7ffa1019b130 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000e9a0 tx=0x7ffa0000ecb0 comp rx=0 tx=0).stop 2024-03-27T06:42:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.536+0000 7ffa14938700 1 -- 172.21.15.16:0/2514186370 reap_dead start 2024-03-27T06:42:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.536+0000 7ffa1639b700 1 -- 172.21.15.16:0/2514186370 shutdown_connections 2024-03-27T06:42:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.536+0000 7ffa1639b700 1 -- 172.21.15.16:0/2514186370 >> 172.21.15.16:0/2514186370 conn(0x7ffa10077590 msgr2=0x7ffa10110500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.536+0000 7ffa1639b700 1 -- 172.21.15.16:0/2514186370 shutdown_connections 2024-03-27T06:42:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:07.536+0000 7ffa1639b700 1 -- 172.21.15.16:0/2514186370 wait complete. 2024-03-27T06:42:08.565 INFO:teuthology.orchestra.run.smithi016.stdout:292057776159 2024-03-27T06:42:08.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-27T06:42:08.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:08 smithi016 ceph-mon[31908]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-27T06:42:08.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:08 smithi156 ceph-mon[36261]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-27T06:42:09.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:08 smithi170 ceph-mon[36438]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-27T06:42:09.170 INFO:teuthology.orchestra.run.smithi016.stdout:111669149756 2024-03-27T06:42:09.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-27T06:42:09.420 INFO:teuthology.orchestra.run.smithi016.stdout:223338299433 2024-03-27T06:42:09.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-27T06:42:09.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.388+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3774269982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc4100e90 msgr2=0x7f3bc4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:09.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.388+0000 7f3bc8a92700 1 --2- 172.21.15.16:0/3774269982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc4100e90 0x7f3bc4101270 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40099c0 tx=0x7f3bb4009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:09.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.388+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3774269982 shutdown_connections 2024-03-27T06:42:09.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.388+0000 7f3bc8a92700 1 --2- 172.21.15.16:0/3774269982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc41079c0 0x7f3bc410f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.388+0000 7f3bc8a92700 1 --2- 172.21.15.16:0/3774269982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc4100e90 0x7f3bc4101270 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.388+0000 7f3bc8a92700 1 --2- 172.21.15.16:0/3774269982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc4101840 0x7f3bc4107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.388+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3774269982 >> 172.21.15.16:0/3774269982 conn(0x7f3bc4077b40 msgr2=0x7f3bc40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.388+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3774269982 shutdown_connections 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.388+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3774269982 wait complete. 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc8a92700 1 Processor -- start 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc8a92700 1 -- start start 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc8a92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc4100e90 0x7f3bc4198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc8a92700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc4101840 0x7f3bc4199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc8a92700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc41079c0 0x7f3bc419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc8a92700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4112aa0 con 0x7f3bc4100e90 2024-03-27T06:42:09.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc8a92700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4112920 con 0x7f3bc4101840 2024-03-27T06:42:09.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc8a92700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4112c20 con 0x7f3bc41079c0 2024-03-27T06:42:09.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc4100e90 0x7f3bc4198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:09.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc4100e90 0x7f3bc4198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54016/0 (socket says 172.21.15.16:54016) 2024-03-27T06:42:09.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc259c700 1 -- 172.21.15.16:0/3673200141 learned_addr learned my addr 172.21.15.16:0/3673200141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:09.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc1d9b700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc4101840 0x7f3bc4199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:09.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc2d9d700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc41079c0 0x7f3bc419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc259c700 1 -- 172.21.15.16:0/3673200141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc41079c0 msgr2=0x7f3bc419d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc259c700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc41079c0 0x7f3bc419d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc259c700 1 -- 172.21.15.16:0/3673200141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc4101840 msgr2=0x7f3bc4199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc259c700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bc4101840 0x7f3bc4199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.389+0000 7f3bc259c700 1 -- 172.21.15.16:0/3673200141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc419def0 con 0x7f3bc4100e90 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.390+0000 7f3bc259c700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc4100e90 0x7f3bc4198f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006010 tx=0x7f3bb4003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.390+0000 7f3bb37fe700 1 -- 172.21.15.16:0/3673200141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4003e60 con 0x7f3bc4100e90 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.390+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3673200141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc419e180 con 0x7f3bc4100e90 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.390+0000 7f3bb37fe700 1 -- 172.21.15.16:0/3673200141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3bb400be40 con 0x7f3bc4100e90 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.390+0000 7f3bb37fe700 1 -- 172.21.15.16:0/3673200141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40173f0 con 0x7f3bc4100e90 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.390+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3673200141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc41a5a60 con 0x7f3bc4100e90 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.391+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3673200141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3bc4062d50 con 0x7f3bc4100e90 2024-03-27T06:42:09.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.392+0000 7f3bb37fe700 1 -- 172.21.15.16:0/3673200141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f3bb400f710 con 0x7f3bc4100e90 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.392+0000 7f3bb37fe700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3bac0a3730 0x7f3bac0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.393+0000 7f3bc1d9b700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3bac0a3730 0x7f3bac0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.393+0000 7f3bb37fe700 1 -- 172.21.15.16:0/3673200141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3bb40d0c70 con 0x7f3bc4100e90 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.393+0000 7f3bb37fe700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974] conn(0x7f3bac0acf10 0x7f3bac0af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.393+0000 7f3bb37fe700 1 -- 172.21.15.16:0/3673200141 --> [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3bb4001d30 con 0x7f3bac0acf10 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.393+0000 7f3bb37fe700 1 -- 172.21.15.16:0/3673200141 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f3bb40d1150 con 0x7f3bc4100e90 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.393+0000 7f3bc1d9b700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3bac0a3730 0x7f3bac0a5bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4109320 tx=0x7f3bb80073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.393+0000 7f3bc2d9d700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974] conn(0x7f3bac0acf10 0x7f3bac0af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.393+0000 7f3bc2d9d700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974] conn(0x7f3bac0acf10 0x7f3bac0af330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.394+0000 7f3bb37fe700 1 -- 172.21.15.16:0/3673200141 <== osd.4 v2:172.21.15.156:6808/3654211974 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bb4001d30 con 0x7f3bac0acf10 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.420+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3673200141 --> [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3bc4063150 con 0x7f3bac0acf10 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.420+0000 7f3bb37fe700 1 -- 172.21.15.16:0/3673200141 <== osd.4 v2:172.21.15.156:6808/3654211974 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3bc4063150 con 0x7f3bac0acf10 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.421+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3673200141 >> [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974] conn(0x7f3bac0acf10 msgr2=0x7f3bac0af330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.421+0000 7f3bc8a92700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974] conn(0x7f3bac0acf10 0x7f3bac0af330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:09.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.421+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3673200141 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3bac0a3730 msgr2=0x7f3bac0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:09.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.421+0000 7f3bc8a92700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3bac0a3730 0x7f3bac0a5bf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4109320 tx=0x7f3bb80073d0 comp rx=0 tx=0).stop 2024-03-27T06:42:09.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.421+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3673200141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc4100e90 msgr2=0x7f3bc4198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:09.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.421+0000 7f3bc8a92700 1 --2- 172.21.15.16:0/3673200141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc4100e90 0x7f3bc4198f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006010 tx=0x7f3bb4003c30 comp rx=0 tx=0).stop 2024-03-27T06:42:09.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.421+0000 7f3bc2d9d700 1 -- 172.21.15.16:0/3673200141 reap_dead start 2024-03-27T06:42:09.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.421+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3673200141 shutdown_connections 2024-03-27T06:42:09.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.421+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3673200141 >> 172.21.15.16:0/3673200141 conn(0x7f3bc4077b40 msgr2=0x7f3bc40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:09.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.421+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3673200141 shutdown_connections 2024-03-27T06:42:09.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:09.421+0000 7f3bc8a92700 1 -- 172.21.15.16:0/3673200141 wait complete. 2024-03-27T06:42:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:10 smithi156 ceph-mon[36261]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-27T06:42:11.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:10 smithi170 ceph-mon[36438]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-27T06:42:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:10 smithi016 ceph-mon[31908]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-27T06:42:11.155 INFO:teuthology.orchestra.run.smithi016.stdout:77309411395 2024-03-27T06:42:11.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-27T06:42:12.162 INFO:teuthology.orchestra.run.smithi016.stdout:201863462959 2024-03-27T06:42:12.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-27T06:42:12.946 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:12 smithi016 ceph-mon[31908]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T06:42:12.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:12 smithi156 ceph-mon[36261]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T06:42:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.055+0000 7f31c4239700 1 -- 172.21.15.16:0/1094544826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc06acb0 msgr2=0x7f31bc10f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:13.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.055+0000 7f31c4239700 1 --2- 172.21.15.16:0/1094544826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc06acb0 0x7f31bc10f4d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f31b40099c0 tx=0x7f31b4009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:13.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.055+0000 7f31c4239700 1 -- 172.21.15.16:0/1094544826 shutdown_connections 2024-03-27T06:42:13.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.055+0000 7f31c4239700 1 --2- 172.21.15.16:0/1094544826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc06acb0 0x7f31bc10f4d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:13.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.055+0000 7f31c4239700 1 --2- 172.21.15.16:0/1094544826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31bc108670 0x7f31bc108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:13.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.055+0000 7f31c4239700 1 --2- 172.21.15.16:0/1094544826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31bc06a2f0 0x7f31bc06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:13.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.055+0000 7f31c4239700 1 -- 172.21.15.16:0/1094544826 >> 172.21.15.16:0/1094544826 conn(0x7f31bc078470 msgr2=0x7f31bc078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:13.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.055+0000 7f31c4239700 1 -- 172.21.15.16:0/1094544826 shutdown_connections 2024-03-27T06:42:13.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.055+0000 7f31c4239700 1 -- 172.21.15.16:0/1094544826 wait complete. 2024-03-27T06:42:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.055+0000 7f31c4239700 1 Processor -- start 2024-03-27T06:42:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.055+0000 7f31c4239700 1 -- start start 2024-03-27T06:42:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c4239700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31bc06a2f0 0x7f31bc198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c4239700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31bc06acb0 0x7f31bc199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c4239700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc108670 0x7f31bc19d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c4239700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc112990 con 0x7f31bc108670 2024-03-27T06:42:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c4239700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc112810 con 0x7f31bc06acb0 2024-03-27T06:42:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c4239700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc112b10 con 0x7f31bc06a2f0 2024-03-27T06:42:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c27d6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc108670 0x7f31bc19d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c17d4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31bc06acb0 0x7f31bc199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c1fd5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31bc06a2f0 0x7f31bc198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c27d6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc108670 0x7f31bc19d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54040/0 (socket says 172.21.15.16:54040) 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c27d6700 1 -- 172.21.15.16:0/704722694 learned_addr learned my addr 172.21.15.16:0/704722694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c1fd5700 1 -- 172.21.15.16:0/704722694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31bc06acb0 msgr2=0x7f31bc199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c1fd5700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31bc06acb0 0x7f31bc199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c1fd5700 1 -- 172.21.15.16:0/704722694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc108670 msgr2=0x7f31bc19d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c1fd5700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc108670 0x7f31bc19d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c1fd5700 1 -- 172.21.15.16:0/704722694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc19dd30 con 0x7f31bc06a2f0 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31c1fd5700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31bc06a2f0 0x7f31bc198d20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f31b8007ec0 tx=0x7f31b800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.056+0000 7f31b2ffd700 1 -- 172.21.15.16:0/704722694 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8004db0 con 0x7f31bc06a2f0 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.057+0000 7f31c4239700 1 -- 172.21.15.16:0/704722694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31bc19e020 con 0x7f31bc06a2f0 2024-03-27T06:42:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.057+0000 7f31b2ffd700 1 -- 172.21.15.16:0/704722694 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f31b800db20 con 0x7f31bc06a2f0 2024-03-27T06:42:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.057+0000 7f31b2ffd700 1 -- 172.21.15.16:0/704722694 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8004770 con 0x7f31bc06a2f0 2024-03-27T06:42:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.057+0000 7f31c4239700 1 -- 172.21.15.16:0/704722694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31bc1a58a0 con 0x7f31bc06a2f0 2024-03-27T06:42:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.057+0000 7f31c4239700 1 -- 172.21.15.16:0/704722694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31bc062dd0 con 0x7f31bc06a2f0 2024-03-27T06:42:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.059+0000 7f31b2ffd700 1 -- 172.21.15.16:0/704722694 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f31b801c060 con 0x7f31bc06a2f0 2024-03-27T06:42:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.059+0000 7f31b2ffd700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f31a80a3730 0x7f31a80a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.059+0000 7f31c17d4700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f31a80a3730 0x7f31a80a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.059+0000 7f31b2ffd700 1 -- 172.21.15.16:0/704722694 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f31b801a070 con 0x7f31bc06a2f0 2024-03-27T06:42:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.060+0000 7f31b2ffd700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995] conn(0x7f31a80acf10 0x7f31a80af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.060+0000 7f31b2ffd700 1 -- 172.21.15.16:0/704722694 --> [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31a80af9e0 con 0x7f31a80acf10 2024-03-27T06:42:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.060+0000 7f31b2ffd700 1 -- 172.21.15.16:0/704722694 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f31b80d0230 con 0x7f31bc06a2f0 2024-03-27T06:42:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.060+0000 7f31c17d4700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f31a80a3730 0x7f31a80a5bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31bc078a00 tx=0x7f31ac009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.060+0000 7f31c27d6700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995] conn(0x7f31a80acf10 0x7f31a80af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.060+0000 7f31c27d6700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995] conn(0x7f31a80acf10 0x7f31a80af330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:13.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.061+0000 7f31b2ffd700 1 -- 172.21.15.16:0/704722694 <== osd.1 v2:172.21.15.156:6800/1327802995 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31a80af9e0 con 0x7f31a80acf10 2024-03-27T06:42:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.079+0000 7f31c4239700 1 -- 172.21.15.16:0/704722694 --> [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f31bc063250 con 0x7f31a80acf10 2024-03-27T06:42:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31b2ffd700 1 -- 172.21.15.16:0/704722694 <== osd.1 v2:172.21.15.156:6800/1327802995 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f31bc063250 con 0x7f31a80acf10 2024-03-27T06:42:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31c4239700 1 -- 172.21.15.16:0/704722694 >> [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995] conn(0x7f31a80acf10 msgr2=0x7f31a80af330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31c4239700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995] conn(0x7f31a80acf10 0x7f31a80af330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31c4239700 1 -- 172.21.15.16:0/704722694 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f31a80a3730 msgr2=0x7f31a80a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31c4239700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f31a80a3730 0x7f31a80a5bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31bc078a00 tx=0x7f31ac009260 comp rx=0 tx=0).stop 2024-03-27T06:42:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31c4239700 1 -- 172.21.15.16:0/704722694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31bc06a2f0 msgr2=0x7f31bc198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31c4239700 1 --2- 172.21.15.16:0/704722694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31bc06a2f0 0x7f31bc198d20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f31b8007ec0 tx=0x7f31b800d3b0 comp rx=0 tx=0).stop 2024-03-27T06:42:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31c27d6700 1 -- 172.21.15.16:0/704722694 reap_dead start 2024-03-27T06:42:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31c4239700 1 -- 172.21.15.16:0/704722694 shutdown_connections 2024-03-27T06:42:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31c4239700 1 -- 172.21.15.16:0/704722694 >> 172.21.15.16:0/704722694 conn(0x7f31bc078470 msgr2=0x7f31bc0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31c4239700 1 -- 172.21.15.16:0/704722694 shutdown_connections 2024-03-27T06:42:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:13.080+0000 7f31c4239700 1 -- 172.21.15.16:0/704722694 wait complete. 2024-03-27T06:42:13.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:12 smithi170 ceph-mon[36438]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-27T06:42:13.583 INFO:teuthology.orchestra.run.smithi016.stdout:313532612633 2024-03-27T06:42:13.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-27T06:42:14.124 INFO:teuthology.orchestra.run.smithi016.stdout:146028888122 2024-03-27T06:42:14.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-27T06:42:14.709 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:14 smithi016 ceph-mon[31908]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-27T06:42:14.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.817+0000 7f86ccddf700 1 -- 172.21.15.16:0/2782809202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86c8108670 msgr2=0x7f86c8108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:14.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.817+0000 7f86ccddf700 1 --2- 172.21.15.16:0/2782809202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86c8108670 0x7f86c8108a50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f86b80099c0 tx=0x7f86b8009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:14.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.817+0000 7f86ccddf700 1 -- 172.21.15.16:0/2782809202 shutdown_connections 2024-03-27T06:42:14.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.817+0000 7f86ccddf700 1 --2- 172.21.15.16:0/2782809202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c8103050 0x7f86c810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:14.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.817+0000 7f86ccddf700 1 --2- 172.21.15.16:0/2782809202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86c8108670 0x7f86c8108a50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:14.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.817+0000 7f86ccddf700 1 --2- 172.21.15.16:0/2782809202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c8102690 0x7f86c8102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:14.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.817+0000 7f86ccddf700 1 -- 172.21.15.16:0/2782809202 >> 172.21.15.16:0/2782809202 conn(0x7f86c8079770 msgr2=0x7f86c8079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:14.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.817+0000 7f86ccddf700 1 -- 172.21.15.16:0/2782809202 shutdown_connections 2024-03-27T06:42:14.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.817+0000 7f86ccddf700 1 -- 172.21.15.16:0/2782809202 wait complete. 2024-03-27T06:42:14.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.818+0000 7f86ccddf700 1 Processor -- start 2024-03-27T06:42:14.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.818+0000 7f86ccddf700 1 -- start start 2024-03-27T06:42:14.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.818+0000 7f86ccddf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86c8102690 0x7f86c819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:14.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.818+0000 7f86ccddf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c8103050 0x7f86c819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:14.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.818+0000 7f86ccddf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c8108670 0x7f86c819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:14.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86ccddf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86c8112a50 con 0x7f86c8102690 2024-03-27T06:42:14.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86ccddf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f86c81128d0 con 0x7f86c8108670 2024-03-27T06:42:14.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86ccddf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86c8112bd0 con 0x7f86c8103050 2024-03-27T06:42:14.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86c659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86c8102690 0x7f86c819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:14.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86c659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86c8102690 0x7f86c819b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54062/0 (socket says 172.21.15.16:54062) 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86c5d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c8103050 0x7f86c819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86c659c700 1 -- 172.21.15.16:0/2900033809 learned_addr learned my addr 172.21.15.16:0/2900033809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86c6d9d700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c8108670 0x7f86c819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86c659c700 1 -- 172.21.15.16:0/2900033809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c8103050 msgr2=0x7f86c819b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86c659c700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86c8103050 0x7f86c819b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86c659c700 1 -- 172.21.15.16:0/2900033809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c8108670 msgr2=0x7f86c819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86c659c700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c8108670 0x7f86c819fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86c659c700 1 -- 172.21.15.16:0/2900033809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c81a01c0 con 0x7f86c8102690 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86c659c700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86c8102690 0x7f86c819b200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86b8005910 tx=0x7f86b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86bf7fe700 1 -- 172.21.15.16:0/2900033809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8027070 con 0x7f86c8102690 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86ccddf700 1 -- 172.21.15.16:0/2900033809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c81a0450 con 0x7f86c8102690 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86bf7fe700 1 -- 172.21.15.16:0/2900033809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f86b800be40 con 0x7f86c8102690 2024-03-27T06:42:14.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86bf7fe700 1 -- 172.21.15.16:0/2900033809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8022e00 con 0x7f86c8102690 2024-03-27T06:42:14.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.819+0000 7f86ccddf700 1 -- 172.21.15.16:0/2900033809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c81a7d30 con 0x7f86c8102690 2024-03-27T06:42:14.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.820+0000 7f86ccddf700 1 -- 172.21.15.16:0/2900033809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86c8101a80 con 0x7f86c8102690 2024-03-27T06:42:14.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.821+0000 7f86bf7fe700 1 -- 172.21.15.16:0/2900033809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f86b8022500 con 0x7f86c8102690 2024-03-27T06:42:14.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.822+0000 7f86bf7fe700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86b40a3730 0x7f86b40a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:14.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.822+0000 7f86c5d9b700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86b40a3730 0x7f86b40a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:14.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.822+0000 7f86bf7fe700 1 -- 172.21.15.16:0/2900033809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f86b80d0a20 con 0x7f86c8102690 2024-03-27T06:42:14.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.822+0000 7f86bf7fe700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078] conn(0x7f86b40acf10 0x7f86b40af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:14.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.822+0000 7f86bf7fe700 1 -- 172.21.15.16:0/2900033809 --> [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86b802ae50 con 0x7f86b40acf10 2024-03-27T06:42:14.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.822+0000 7f86bf7fe700 1 -- 172.21.15.16:0/2900033809 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f86b80d0f00 con 0x7f86c8102690 2024-03-27T06:42:14.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.822+0000 7f86c6d9d700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078] conn(0x7f86b40acf10 0x7f86b40af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:14.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.822+0000 7f86c5d9b700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86b40a3730 0x7f86b40a5bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f86c810f860 tx=0x7f86b0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:14.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.823+0000 7f86c6d9d700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078] conn(0x7f86b40acf10 0x7f86b40af330 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:14.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.823+0000 7f86bf7fe700 1 -- 172.21.15.16:0/2900033809 <== osd.0 v2:172.21.15.16:6802/1317357078 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86b802ae50 con 0x7f86b40acf10 2024-03-27T06:42:14.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.840+0000 7f86ccddf700 1 -- 172.21.15.16:0/2900033809 --> [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86c8103d20 con 0x7f86b40acf10 2024-03-27T06:42:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.840+0000 7f86bf7fe700 1 -- 172.21.15.16:0/2900033809 <== osd.0 v2:172.21.15.16:6802/1317357078 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f86c8103d20 con 0x7f86b40acf10 2024-03-27T06:42:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.840+0000 7f86ccddf700 1 -- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078] conn(0x7f86b40acf10 msgr2=0x7f86b40af330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.840+0000 7f86ccddf700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078] conn(0x7f86b40acf10 0x7f86b40af330 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.841+0000 7f86ccddf700 1 -- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86b40a3730 msgr2=0x7f86b40a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.841+0000 7f86ccddf700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86b40a3730 0x7f86b40a5bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f86c810f860 tx=0x7f86b0009260 comp rx=0 tx=0).stop 2024-03-27T06:42:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.841+0000 7f86ccddf700 1 -- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86c8102690 msgr2=0x7f86c819b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.841+0000 7f86ccddf700 1 --2- 172.21.15.16:0/2900033809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86c8102690 0x7f86c819b200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86b8005910 tx=0x7f86b800f690 comp rx=0 tx=0).stop 2024-03-27T06:42:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.841+0000 7f86c6d9d700 1 -- 172.21.15.16:0/2900033809 reap_dead start 2024-03-27T06:42:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.841+0000 7f86ccddf700 1 -- 172.21.15.16:0/2900033809 shutdown_connections 2024-03-27T06:42:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.841+0000 7f86ccddf700 1 -- 172.21.15.16:0/2900033809 >> 172.21.15.16:0/2900033809 conn(0x7f86c8079770 msgr2=0x7f86c81010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.841+0000 7f86ccddf700 1 -- 172.21.15.16:0/2900033809 shutdown_connections 2024-03-27T06:42:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:14.841+0000 7f86ccddf700 1 -- 172.21.15.16:0/2900033809 wait complete. 2024-03-27T06:42:14.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:14 smithi156 ceph-mon[36261]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-27T06:42:15.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:14 smithi170 ceph-mon[36438]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-27T06:42:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:15.300 INFO:teuthology.orchestra.run.smithi016.stdout:55834574921 2024-03-27T06:42:15.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-27T06:42:16.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:16 smithi016 ceph-mon[31908]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.919+0000 7f953e58e700 1 -- 172.21.15.16:0/2731044343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9538102180 msgr2=0x7f95381061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.919+0000 7f953e58e700 1 --2- 172.21.15.16:0/2731044343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9538102180 0x7f95381061a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f95280099c0 tx=0x7f9528009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.919+0000 7f953e58e700 1 -- 172.21.15.16:0/2731044343 shutdown_connections 2024-03-27T06:42:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.919+0000 7f953e58e700 1 --2- 172.21.15.16:0/2731044343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95381068d0 0x7f953810e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.919+0000 7f953e58e700 1 --2- 172.21.15.16:0/2731044343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95381017d0 0x7f9538101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.919+0000 7f953e58e700 1 --2- 172.21.15.16:0/2731044343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9538102180 0x7f95381061a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:16.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.919+0000 7f953e58e700 1 -- 172.21.15.16:0/2731044343 >> 172.21.15.16:0/2731044343 conn(0x7f9538077560 msgr2=0x7f9538077970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.920+0000 7f953e58e700 1 -- 172.21.15.16:0/2731044343 shutdown_connections 2024-03-27T06:42:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.920+0000 7f953e58e700 1 -- 172.21.15.16:0/2731044343 wait complete. 2024-03-27T06:42:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.920+0000 7f953e58e700 1 Processor -- start 2024-03-27T06:42:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.920+0000 7f953e58e700 1 -- start start 2024-03-27T06:42:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f953e58e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95381017d0 0x7f9538199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f953e58e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9538102180 0x7f953819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f953e58e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381068d0 0x7f95381a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:16.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f953e58e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95381118d0 con 0x7f95381068d0 2024-03-27T06:42:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f953e58e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f9538111750 con 0x7f95381017d0 2024-03-27T06:42:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f953e58e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9538111a50 con 0x7f9538102180 2024-03-27T06:42:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f9537fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95381017d0 0x7f9538199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f95377fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9538102180 0x7f953819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f953cb2b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381068d0 0x7f95381a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f9537fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95381017d0 0x7f9538199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:51502/0 (socket says 172.21.15.16:51502) 2024-03-27T06:42:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f95377fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9538102180 0x7f953819a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:50870/0 (socket says 172.21.15.16:50870) 2024-03-27T06:42:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f9537fff700 1 -- 172.21.15.16:0/928143690 learned_addr learned my addr 172.21.15.16:0/928143690 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f9537fff700 1 -- 172.21.15.16:0/928143690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9538102180 msgr2=0x7f953819a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:16.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f9537fff700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9538102180 0x7f953819a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f9537fff700 1 -- 172.21.15.16:0/928143690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381068d0 msgr2=0x7f95381a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f9537fff700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381068d0 0x7f95381a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f9537fff700 1 -- 172.21.15.16:0/928143690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95381a51d0 con 0x7f95381017d0 2024-03-27T06:42:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f95377fe700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9538102180 0x7f953819a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f953cb2b700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95381068d0 0x7f95381a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f9537fff700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95381017d0 0x7f9538199c10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f952c00e9a0 tx=0x7f952c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.921+0000 7f95357fa700 1 -- 172.21.15.16:0/928143690 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c00ca10 con 0x7f95381017d0 2024-03-27T06:42:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.922+0000 7f95357fa700 1 -- 172.21.15.16:0/928143690 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f952c00cbb0 con 0x7f95381017d0 2024-03-27T06:42:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.922+0000 7f95357fa700 1 -- 172.21.15.16:0/928143690 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c018710 con 0x7f95381017d0 2024-03-27T06:42:16.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.922+0000 7f953e58e700 1 -- 172.21.15.16:0/928143690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95381a5400 con 0x7f95381017d0 2024-03-27T06:42:16.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.922+0000 7f953e58e700 1 -- 172.21.15.16:0/928143690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95381a5960 con 0x7f95381017d0 2024-03-27T06:42:16.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.922+0000 7f953e58e700 1 -- 172.21.15.16:0/928143690 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9538062dd0 con 0x7f95381017d0 2024-03-27T06:42:16.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.923+0000 7f95357fa700 1 -- 172.21.15.16:0/928143690 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f952c010430 con 0x7f95381017d0 2024-03-27T06:42:16.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.924+0000 7f95357fa700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f95200a37b0 0x7f95200a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.924+0000 7f95377fe700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f95200a37b0 0x7f95200a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.924+0000 7f95357fa700 1 -- 172.21.15.16:0/928143690 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f952c0d06f0 con 0x7f95381017d0 2024-03-27T06:42:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.924+0000 7f95357fa700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164] conn(0x7f95200acf90 0x7f95200af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.924+0000 7f95357fa700 1 -- 172.21.15.16:0/928143690 --> [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f952c01bec0 con 0x7f95200acf90 2024-03-27T06:42:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.924+0000 7f95357fa700 1 -- 172.21.15.16:0/928143690 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f952c0d0aa0 con 0x7f95381017d0 2024-03-27T06:42:16.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.924+0000 7f95377fe700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f95200a37b0 0x7f95200a5c70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f953806b890 tx=0x7f9528005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.925+0000 7f953cb2b700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164] conn(0x7f95200acf90 0x7f95200af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.925+0000 7f953cb2b700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164] conn(0x7f95200acf90 0x7f95200af3b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:16.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.926+0000 7f95357fa700 1 -- 172.21.15.16:0/928143690 <== osd.11 v2:172.21.15.170:6824/703899164 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f952c01bec0 con 0x7f95200acf90 2024-03-27T06:42:16.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f953e58e700 1 -- 172.21.15.16:0/928143690 --> [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9538063250 con 0x7f95200acf90 2024-03-27T06:42:16.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f95357fa700 1 -- 172.21.15.16:0/928143690 <== osd.11 v2:172.21.15.170:6824/703899164 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9538063250 con 0x7f95200acf90 2024-03-27T06:42:16.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f953e58e700 1 -- 172.21.15.16:0/928143690 >> [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164] conn(0x7f95200acf90 msgr2=0x7f95200af3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f953e58e700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164] conn(0x7f95200acf90 0x7f95200af3b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f953e58e700 1 -- 172.21.15.16:0/928143690 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f95200a37b0 msgr2=0x7f95200a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f953e58e700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f95200a37b0 0x7f95200a5c70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f953806b890 tx=0x7f9528005fb0 comp rx=0 tx=0).stop 2024-03-27T06:42:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f953e58e700 1 -- 172.21.15.16:0/928143690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95381017d0 msgr2=0x7f9538199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f953e58e700 1 --2- 172.21.15.16:0/928143690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95381017d0 0x7f9538199c10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f952c00e9a0 tx=0x7f952c00ecb0 comp rx=0 tx=0).stop 2024-03-27T06:42:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f953cb2b700 1 -- 172.21.15.16:0/928143690 reap_dead start 2024-03-27T06:42:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f953e58e700 1 -- 172.21.15.16:0/928143690 shutdown_connections 2024-03-27T06:42:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f953e58e700 1 -- 172.21.15.16:0/928143690 >> 172.21.15.16:0/928143690 conn(0x7f9538077560 msgr2=0x7f95381052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.943+0000 7f953e58e700 1 -- 172.21.15.16:0/928143690 shutdown_connections 2024-03-27T06:42:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:16.944+0000 7f953e58e700 1 -- 172.21.15.16:0/928143690 wait complete. 2024-03-27T06:42:16.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:16 smithi156 ceph-mon[36261]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:17.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:16 smithi170 ceph-mon[36438]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:18.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:18 smithi016 ceph-mon[31908]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-27T06:42:18.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:18 smithi156 ceph-mon[36261]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-27T06:42:19.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:18 smithi170 ceph-mon[36438]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-27T06:42:19.530 INFO:teuthology.orchestra.run.smithi016.stdout:347892350998 2024-03-27T06:42:19.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-27T06:42:19.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-27T06:42:19.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-27T06:42:19.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-27T06:42:19.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-27T06:42:19.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-27T06:42:19.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-27T06:42:19.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-27T06:42:19.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:19 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-27T06:42:20.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-27T06:42:20.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-27T06:42:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-27T06:42:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-27T06:42:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-27T06:42:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-27T06:42:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-27T06:42:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:19 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-27T06:42:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:20.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-27T06:42:20.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-27T06:42:20.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-27T06:42:20.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-27T06:42:20.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-27T06:42:20.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-27T06:42:20.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-27T06:42:20.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:19 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-27T06:42:20.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:20 smithi156 ceph-mon[36261]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-27T06:42:20.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-27T06:42:20.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-27T06:42:20.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-27T06:42:20.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-27T06:42:20.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-27T06:42:20.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-27T06:42:20.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-27T06:42:20.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:20 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-27T06:42:20.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:20 smithi156 ceph-mon[36261]: osdmap e87: 12 total, 12 up, 12 in 2024-03-27T06:42:21.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:20 smithi016 ceph-mon[31908]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-27T06:42:21.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-27T06:42:21.024 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-27T06:42:21.024 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-27T06:42:21.024 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-27T06:42:21.024 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-27T06:42:21.024 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-27T06:42:21.024 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-27T06:42:21.024 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:20 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-27T06:42:21.024 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:20 smithi016 ceph-mon[31908]: osdmap e87: 12 total, 12 up, 12 in 2024-03-27T06:42:21.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:20 smithi170 ceph-mon[36438]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-27T06:42:21.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-27T06:42:21.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-27T06:42:21.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-27T06:42:21.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-27T06:42:21.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-27T06:42:21.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-27T06:42:21.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-27T06:42:21.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:20 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-27T06:42:21.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:20 smithi170 ceph-mon[36438]: osdmap e87: 12 total, 12 up, 12 in 2024-03-27T06:42:21.463 INFO:teuthology.orchestra.run.smithi016.stdout:34359738446 2024-03-27T06:42:21.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-27T06:42:21.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:21 smithi016 ceph-mon[31908]: osdmap e88: 12 total, 12 up, 12 in 2024-03-27T06:42:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:21 smithi156 ceph-mon[36261]: osdmap e88: 12 total, 12 up, 12 in 2024-03-27T06:42:22.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:21 smithi170 ceph-mon[36438]: osdmap e88: 12 total, 12 up, 12 in 2024-03-27T06:42:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.513+0000 7f82f164d700 1 -- 172.21.15.16:0/511952009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ec06a2f0 msgr2=0x7f82ec06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.513+0000 7f82f164d700 1 --2- 172.21.15.16:0/511952009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ec06a2f0 0x7f82ec06a770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f82e00099c0 tx=0x7f82e0009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.513+0000 7f82f164d700 1 -- 172.21.15.16:0/511952009 shutdown_connections 2024-03-27T06:42:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.513+0000 7f82f164d700 1 --2- 172.21.15.16:0/511952009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82ec06acb0 0x7f82ec10f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.513+0000 7f82f164d700 1 --2- 172.21.15.16:0/511952009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82ec108670 0x7f82ec108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:22.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.513+0000 7f82f164d700 1 --2- 172.21.15.16:0/511952009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ec06a2f0 0x7f82ec06a770 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:22.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.513+0000 7f82f164d700 1 -- 172.21.15.16:0/511952009 >> 172.21.15.16:0/511952009 conn(0x7f82ec078470 msgr2=0x7f82ec078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:22.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.513+0000 7f82f164d700 1 -- 172.21.15.16:0/511952009 shutdown_connections 2024-03-27T06:42:22.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.514+0000 7f82f164d700 1 -- 172.21.15.16:0/511952009 wait complete. 2024-03-27T06:42:22.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.515+0000 7f82f164d700 1 Processor -- start 2024-03-27T06:42:22.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.515+0000 7f82f164d700 1 -- start start 2024-03-27T06:42:22.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.515+0000 7f82f164d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82ec06a2f0 0x7f82ec198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:22.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82f164d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ec06acb0 0x7f82ec1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:22.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82f164d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82ec108670 0x7f82ec19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82f164d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec112990 con 0x7f82ec06acb0 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82f164d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec112810 con 0x7f82ec06a2f0 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82f164d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec112b10 con 0x7f82ec108670 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82ea7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ec06acb0 0x7f82ec1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82ea7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ec06acb0 0x7f82ec1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39420/0 (socket says 172.21.15.16:39420) 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82ea7fc700 1 -- 172.21.15.16:0/852939719 learned_addr learned my addr 172.21.15.16:0/852939719 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82eb7fe700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82ec108670 0x7f82ec19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82eaffd700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82ec06a2f0 0x7f82ec198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82eb7fe700 1 -- 172.21.15.16:0/852939719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82ec06a2f0 msgr2=0x7f82ec198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82eb7fe700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82ec06a2f0 0x7f82ec198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82eb7fe700 1 -- 172.21.15.16:0/852939719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ec06acb0 msgr2=0x7f82ec1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82eb7fe700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ec06acb0 0x7f82ec1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82eb7fe700 1 -- 172.21.15.16:0/852939719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec19dd70 con 0x7f82ec108670 2024-03-27T06:42:22.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82eb7fe700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82ec108670 0x7f82ec19d630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82e400efc0 tx=0x7f82e400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:22.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.516+0000 7f82d3fff700 1 -- 172.21.15.16:0/852939719 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e40092e0 con 0x7f82ec108670 2024-03-27T06:42:22.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.517+0000 7f82d3fff700 1 -- 172.21.15.16:0/852939719 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f82e4004bb0 con 0x7f82ec108670 2024-03-27T06:42:22.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.517+0000 7f82f164d700 1 -- 172.21.15.16:0/852939719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ec19e000 con 0x7f82ec108670 2024-03-27T06:42:22.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.517+0000 7f82d3fff700 1 -- 172.21.15.16:0/852939719 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4004280 con 0x7f82ec108670 2024-03-27T06:42:22.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.517+0000 7f82f164d700 1 -- 172.21.15.16:0/852939719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ec1a58e0 con 0x7f82ec108670 2024-03-27T06:42:22.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.518+0000 7f82f164d700 1 -- 172.21.15.16:0/852939719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ec069300 con 0x7f82ec108670 2024-03-27T06:42:22.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.519+0000 7f82d3fff700 1 -- 172.21.15.16:0/852939719 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f82e401b030 con 0x7f82ec108670 2024-03-27T06:42:22.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.520+0000 7f82d3fff700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f82d40a3730 0x7f82d40a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:22.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.520+0000 7f82eaffd700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f82d40a3730 0x7f82d40a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:22.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.520+0000 7f82d3fff700 1 -- 172.21.15.16:0/852939719 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f82e40cf990 con 0x7f82ec108670 2024-03-27T06:42:22.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.521+0000 7f82eaffd700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f82d40a3730 0x7f82d40a5bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009bf0 tx=0x7f82dc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:22.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.525+0000 7f82d3fff700 1 -- 172.21.15.16:0/852939719 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f82e4022020 con 0x7f82ec108670 2024-03-27T06:42:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.734+0000 7f82f164d700 1 -- 172.21.15.16:0/852939719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f82ec02dd30 con 0x7f82ec108670 2024-03-27T06:42:22.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.735+0000 7f82d3fff700 1 -- 172.21.15.16:0/852939719 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f82e4010040 con 0x7f82ec108670 2024-03-27T06:42:22.729 INFO:teuthology.orchestra.run.smithi016.stdout:167503724598 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 -- 172.21.15.16:0/852939719 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f82d40a3730 msgr2=0x7f82d40a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f82d40a3730 0x7f82d40a5bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009bf0 tx=0x7f82dc009260 comp rx=0 tx=0).stop 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 -- 172.21.15.16:0/852939719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82ec108670 msgr2=0x7f82ec19d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82ec108670 0x7f82ec19d630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82e400efc0 tx=0x7f82e400c3a0 comp rx=0 tx=0).stop 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 -- 172.21.15.16:0/852939719 shutdown_connections 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f82d40a3730 0x7f82d40a5bf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82ec108670 0x7f82ec19d630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82ec06a2f0 0x7f82ec198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 --2- 172.21.15.16:0/852939719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82ec06acb0 0x7f82ec1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 -- 172.21.15.16:0/852939719 >> 172.21.15.16:0/852939719 conn(0x7f82ec078470 msgr2=0x7f82ec100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 -- 172.21.15.16:0/852939719 shutdown_connections 2024-03-27T06:42:22.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:22.737+0000 7f82f164d700 1 -- 172.21.15.16:0/852939719 wait complete. 2024-03-27T06:42:22.927 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:22 smithi016 ceph-mon[31908]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:22.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:22 smithi156 ceph-mon[36261]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:23.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:22 smithi170 ceph-mon[36438]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:23.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:23 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/852939719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-27T06:42:23.984 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:23 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/852939719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-27T06:42:24.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:23 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/852939719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-27T06:42:24.832 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:24 smithi016 ceph-mon[31908]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:24.873 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724594 got 167503724598 for osd.5 2024-03-27T06:42:24.873 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:24.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:24 smithi156 ceph-mon[36261]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:25.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:24 smithi170 ceph-mon[36438]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:26.897 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:26 smithi016 ceph-mon[31908]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:26.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:26 smithi156 ceph-mon[36261]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:27.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:26 smithi170 ceph-mon[36438]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.767+0000 7f8089b77700 1 -- 172.21.15.16:0/3151186449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8084108680 msgr2=0x7f8084108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.767+0000 7f8089b77700 1 --2- 172.21.15.16:0/3151186449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8084108680 0x7f8084108a60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f807400b0a0 tx=0x7f807400b3b0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.767+0000 7f8089b77700 1 -- 172.21.15.16:0/3151186449 shutdown_connections 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.767+0000 7f8089b77700 1 --2- 172.21.15.16:0/3151186449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f808406b540 0x7f808410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.767+0000 7f8089b77700 1 --2- 172.21.15.16:0/3151186449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8084108680 0x7f8084108a60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.767+0000 7f8089b77700 1 --2- 172.21.15.16:0/3151186449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f808406ab80 0x7f808406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.767+0000 7f8089b77700 1 -- 172.21.15.16:0/3151186449 >> 172.21.15.16:0/3151186449 conn(0x7f8084077590 msgr2=0x7f80840779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.767+0000 7f8089b77700 1 -- 172.21.15.16:0/3151186449 shutdown_connections 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.767+0000 7f8089b77700 1 -- 172.21.15.16:0/3151186449 wait complete. 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f8089b77700 1 Processor -- start 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f8089b77700 1 -- start start 2024-03-27T06:42:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f8089b77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808406ab80 0x7f8084101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f8089b77700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f808406b540 0x7f8084101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f8089b77700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8084108680 0x7f80841020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f8089b77700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8084112a50 con 0x7f808406ab80 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f8089b77700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f80841128d0 con 0x7f808406b540 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f8089b77700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8084112bd0 con 0x7f8084108680 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f80837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808406ab80 0x7f8084101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f80837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808406ab80 0x7f8084101640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52550/0 (socket says 172.21.15.16:52550) 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f80837fe700 1 -- 172.21.15.16:0/3349292189 learned_addr learned my addr 172.21.15.16:0/3349292189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.768+0000 7f8082ffd700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f808406b540 0x7f8084101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f8083fff700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8084108680 0x7f80841020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f80837fe700 1 -- 172.21.15.16:0/3349292189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8084108680 msgr2=0x7f80841020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f80837fe700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8084108680 0x7f80841020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f80837fe700 1 -- 172.21.15.16:0/3349292189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f808406b540 msgr2=0x7f8084101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f80837fe700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f808406b540 0x7f8084101b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f80837fe700 1 -- 172.21.15.16:0/3349292189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80841a73d0 con 0x7f808406ab80 2024-03-27T06:42:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f80837fe700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808406ab80 0x7f8084101640 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f807400b070 tx=0x7f8074004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f8080ff9700 1 -- 172.21.15.16:0/3349292189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074021070 con 0x7f808406ab80 2024-03-27T06:42:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f8089b77700 1 -- 172.21.15.16:0/3349292189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80841a7660 con 0x7f808406ab80 2024-03-27T06:42:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f8080ff9700 1 -- 172.21.15.16:0/3349292189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8074003c00 con 0x7f808406ab80 2024-03-27T06:42:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f8080ff9700 1 -- 172.21.15.16:0/3349292189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074004740 con 0x7f808406ab80 2024-03-27T06:42:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.769+0000 7f8089b77700 1 -- 172.21.15.16:0/3349292189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80841a7b10 con 0x7f808406ab80 2024-03-27T06:42:27.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.770+0000 7f8089b77700 1 -- 172.21.15.16:0/3349292189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808410ab80 con 0x7f808406ab80 2024-03-27T06:42:27.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.771+0000 7f8080ff9700 1 -- 172.21.15.16:0/3349292189 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f8074007980 con 0x7f808406ab80 2024-03-27T06:42:27.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.772+0000 7f8080ff9700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f806c0a3730 0x7f806c0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:27.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.772+0000 7f8082ffd700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f806c0a3730 0x7f806c0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:27.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.772+0000 7f8080ff9700 1 -- 172.21.15.16:0/3349292189 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f807400f020 con 0x7f808406ab80 2024-03-27T06:42:27.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.772+0000 7f8082ffd700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f806c0a3730 0x7f806c0a5bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8084077b70 tx=0x7f8078009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.776+0000 7f8080ff9700 1 -- 172.21.15.16:0/3349292189 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f807402a020 con 0x7f808406ab80 2024-03-27T06:42:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.981+0000 7f00f4b92700 1 -- 172.21.15.16:0/3368966203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0108620 msgr2=0x7f00f0108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.981+0000 7f00f4b92700 1 --2- 172.21.15.16:0/3368966203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0108620 0x7f00f0108a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009990 tx=0x7f00e0009ca0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.981+0000 7f00f4b92700 1 -- 172.21.15.16:0/3368966203 shutdown_connections 2024-03-27T06:42:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.981+0000 7f00f4b92700 1 --2- 172.21.15.16:0/3368966203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f0102fe0 0x7f00f010f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.981+0000 7f00f4b92700 1 --2- 172.21.15.16:0/3368966203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0108620 0x7f00f0108a00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.981+0000 7f00f4b92700 1 --2- 172.21.15.16:0/3368966203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0102620 0x7f00f0102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.981+0000 7f00f4b92700 1 -- 172.21.15.16:0/3368966203 >> 172.21.15.16:0/3368966203 conn(0x7f00f00fe0f0 msgr2=0x7f00f0100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.981+0000 7f00f4b92700 1 -- 172.21.15.16:0/3368966203 shutdown_connections 2024-03-27T06:42:27.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.981+0000 7f00f4b92700 1 -- 172.21.15.16:0/3368966203 wait complete. 2024-03-27T06:42:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00f4b92700 1 Processor -- start 2024-03-27T06:42:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00f4b92700 1 -- start start 2024-03-27T06:42:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00f4b92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0102620 0x7f00f0074940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00f4b92700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0102fe0 0x7f00f0074e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00f4b92700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f0108620 0x7f00f00753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00f4b92700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0112a90 con 0x7f00f0102620 2024-03-27T06:42:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00f4b92700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0112910 con 0x7f00f0108620 2024-03-27T06:42:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00f4b92700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0112c10 con 0x7f00f0102fe0 2024-03-27T06:42:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00edd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0102fe0 0x7f00f0074e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:27.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00edd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0102fe0 0x7f00f0074e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:43210/0 (socket says 172.21.15.16:43210) 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00edd9b700 1 -- 172.21.15.16:0/2704909051 learned_addr learned my addr 172.21.15.16:0/2704909051 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00eed9d700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f0108620 0x7f00f00753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.982+0000 7f00ee59c700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0102620 0x7f00f0074940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00edd9b700 1 -- 172.21.15.16:0/2704909051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f0108620 msgr2=0x7f00f00753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00edd9b700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f0108620 0x7f00f00753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00edd9b700 1 -- 172.21.15.16:0/2704909051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0102620 msgr2=0x7f00f0074940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00edd9b700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0102620 0x7f00f0074940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00edd9b700 1 -- 172.21.15.16:0/2704909051 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f01a73b0 con 0x7f00f0102fe0 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00eed9d700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f0108620 0x7f00f00753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00edd9b700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0102fe0 0x7f00f0074e80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f00e400e9a0 tx=0x7f00e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00df7fe700 1 -- 172.21.15.16:0/2704909051 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e400ca10 con 0x7f00f0102fe0 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00df7fe700 1 -- 172.21.15.16:0/2704909051 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f00e400cbb0 con 0x7f00f0102fe0 2024-03-27T06:42:27.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00df7fe700 1 -- 172.21.15.16:0/2704909051 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4018710 con 0x7f00f0102fe0 2024-03-27T06:42:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00f4b92700 1 -- 172.21.15.16:0/2704909051 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f01a76a0 con 0x7f00f0102fe0 2024-03-27T06:42:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.983+0000 7f00f4b92700 1 -- 172.21.15.16:0/2704909051 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f01a7ad0 con 0x7f00f0102fe0 2024-03-27T06:42:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.984+0000 7f00f4b92700 1 -- 172.21.15.16:0/2704909051 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f010abb0 con 0x7f00f0102fe0 2024-03-27T06:42:27.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.986+0000 7f00df7fe700 1 -- 172.21.15.16:0/2704909051 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f00e4010430 con 0x7f00f0102fe0 2024-03-27T06:42:27.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.987+0000 7f00df7fe700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f00d80a3830 0x7f00d80a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.987+0000 7f00df7fe700 1 -- 172.21.15.16:0/2704909051 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f00e40d0000 con 0x7f00f0102fe0 2024-03-27T06:42:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.987+0000 7f00ee59c700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f00d80a3830 0x7f00d80a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:27.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.988+0000 7f00ee59c700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f00d80a3830 0x7f00d80a5cf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f00e000b2e0 tx=0x7f00e0005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:27.984 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.991+0000 7f8089b77700 1 -- 172.21.15.16:0/3349292189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f8084102b70 con 0x7f808406ab80 2024-03-27T06:42:27.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.991+0000 7f00df7fe700 1 -- 172.21.15.16:0/2704909051 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f00e409ca20 con 0x7f00f0102fe0 2024-03-27T06:42:27.991 INFO:teuthology.orchestra.run.smithi016.stdout:292057776163 2024-03-27T06:42:27.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:27.993+0000 7f8080ff9700 1 -- 172.21.15.16:0/3349292189 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f807409dc90 con 0x7f808406ab80 2024-03-27T06:42:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.000+0000 7f806a7fc700 1 -- 172.21.15.16:0/3349292189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f806c0a3730 msgr2=0x7f806c0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.000+0000 7f806a7fc700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f806c0a3730 0x7f806c0a5bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8084077b70 tx=0x7f8078009260 comp rx=0 tx=0).stop 2024-03-27T06:42:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.000+0000 7f806a7fc700 1 -- 172.21.15.16:0/3349292189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808406ab80 msgr2=0x7f8084101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.000+0000 7f806a7fc700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808406ab80 0x7f8084101640 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f807400b070 tx=0x7f8074004ce0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.001+0000 7f806a7fc700 1 -- 172.21.15.16:0/3349292189 shutdown_connections 2024-03-27T06:42:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.001+0000 7f806a7fc700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f806c0a3730 0x7f806c0a5bf0 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8084077b70 tx=0x7f8078009260 comp rx=0 tx=0).stop 2024-03-27T06:42:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.001+0000 7f806a7fc700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8084108680 0x7f80841020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.001+0000 7f806a7fc700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f808406ab80 0x7f8084101640 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.001+0000 7f806a7fc700 1 --2- 172.21.15.16:0/3349292189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f808406b540 0x7f8084101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.001+0000 7f806a7fc700 1 -- 172.21.15.16:0/3349292189 >> 172.21.15.16:0/3349292189 conn(0x7f8084077590 msgr2=0x7f8084100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:27.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.003+0000 7f806a7fc700 1 -- 172.21.15.16:0/3349292189 shutdown_connections 2024-03-27T06:42:28.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.004+0000 7f806a7fc700 1 -- 172.21.15.16:0/3349292189 wait complete. 2024-03-27T06:42:28.005 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.221+0000 7f00f4b92700 1 -- 172.21.15.16:0/2704909051 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f00f010a590 con 0x7f00f0102fe0 2024-03-27T06:42:28.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.222+0000 7f00df7fe700 1 -- 172.21.15.16:0/2704909051 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f00e409c320 con 0x7f00f0102fe0 2024-03-27T06:42:28.217 INFO:teuthology.orchestra.run.smithi016.stdout:223338299437 2024-03-27T06:42:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 -- 172.21.15.16:0/2704909051 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f00d80a3830 msgr2=0x7f00d80a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f00d80a3830 0x7f00d80a5cf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f00e000b2e0 tx=0x7f00e0005960 comp rx=0 tx=0).stop 2024-03-27T06:42:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 -- 172.21.15.16:0/2704909051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0102fe0 msgr2=0x7f00f0074e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0102fe0 0x7f00f0074e80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f00e400e9a0 tx=0x7f00e400ecb0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 -- 172.21.15.16:0/2704909051 shutdown_connections 2024-03-27T06:42:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00f0108620 0x7f00f00753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0102620 0x7f00f0074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f00d80a3830 0x7f00d80a5cf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 --2- 172.21.15.16:0/2704909051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00f0102fe0 0x7f00f0074e80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 -- 172.21.15.16:0/2704909051 >> 172.21.15.16:0/2704909051 conn(0x7f00f00fe0f0 msgr2=0x7f00f00fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 -- 172.21.15.16:0/2704909051 shutdown_connections 2024-03-27T06:42:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.224+0000 7f00f4b92700 1 -- 172.21.15.16:0/2704909051 wait complete. 2024-03-27T06:42:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.712+0000 7f816d809700 1 -- 172.21.15.16:0/76077705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81681068d0 msgr2=0x7f816810e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.712+0000 7f816d809700 1 --2- 172.21.15.16:0/76077705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81681068d0 0x7f816810e450 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f816000b0d0 tx=0x7f816000b3e0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.712+0000 7f816d809700 1 -- 172.21.15.16:0/76077705 shutdown_connections 2024-03-27T06:42:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.712+0000 7f816d809700 1 --2- 172.21.15.16:0/76077705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81681068d0 0x7f816810e450 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.712+0000 7f816d809700 1 --2- 172.21.15.16:0/76077705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81681017d0 0x7f8168101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.712+0000 7f816d809700 1 --2- 172.21.15.16:0/76077705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8168102180 0x7f81681061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.712+0000 7f816d809700 1 -- 172.21.15.16:0/76077705 >> 172.21.15.16:0/76077705 conn(0x7f8168077560 msgr2=0x7f8168077970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:28.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.712+0000 7f816d809700 1 -- 172.21.15.16:0/76077705 shutdown_connections 2024-03-27T06:42:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.712+0000 7f816d809700 1 -- 172.21.15.16:0/76077705 wait complete. 2024-03-27T06:42:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.712+0000 7f816d809700 1 Processor -- start 2024-03-27T06:42:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f816d809700 1 -- start start 2024-03-27T06:42:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f816d809700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81681017d0 0x7f81681031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f816d809700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8168102180 0x7f81681036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:28.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f816d809700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81681068d0 0x7f8168103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f816d809700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8168111930 con 0x7f81681017d0 2024-03-27T06:42:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f816d809700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f81681117b0 con 0x7f81681068d0 2024-03-27T06:42:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f816d809700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8168111ab0 con 0x7f8168102180 2024-03-27T06:42:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f8166ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81681017d0 0x7f81681031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f81667fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8168102180 0x7f81681036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f8166ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81681017d0 0x7f81681031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52592/0 (socket says 172.21.15.16:52592) 2024-03-27T06:42:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f81667fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8168102180 0x7f81681036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:43224/0 (socket says 172.21.15.16:43224) 2024-03-27T06:42:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f8166ffd700 1 -- 172.21.15.16:0/2239349507 learned_addr learned my addr 172.21.15.16:0/2239349507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f81677fe700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81681068d0 0x7f8168103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f81667fc700 1 -- 172.21.15.16:0/2239349507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81681068d0 msgr2=0x7f8168103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f81667fc700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81681068d0 0x7f8168103c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f81667fc700 1 -- 172.21.15.16:0/2239349507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81681017d0 msgr2=0x7f81681031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f81667fc700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81681017d0 0x7f81681031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f81667fc700 1 -- 172.21.15.16:0/2239349507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81681a7340 con 0x7f8168102180 2024-03-27T06:42:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.713+0000 7f8166ffd700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81681017d0 0x7f81681031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.714+0000 7f81667fc700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8168102180 0x7f81681036f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f815800d750 tx=0x7f815800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.714+0000 7f816c807700 1 -- 172.21.15.16:0/2239349507 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158014030 con 0x7f8168102180 2024-03-27T06:42:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.714+0000 7f816d809700 1 -- 172.21.15.16:0/2239349507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81681a7630 con 0x7f8168102180 2024-03-27T06:42:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.714+0000 7f816c807700 1 -- 172.21.15.16:0/2239349507 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8158004520 con 0x7f8168102180 2024-03-27T06:42:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.714+0000 7f816d809700 1 -- 172.21.15.16:0/2239349507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81681a7bc0 con 0x7f8168102180 2024-03-27T06:42:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.714+0000 7f816c807700 1 -- 172.21.15.16:0/2239349507 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81580054a0 con 0x7f8168102180 2024-03-27T06:42:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.715+0000 7f816d809700 1 -- 172.21.15.16:0/2239349507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8168102c60 con 0x7f8168102180 2024-03-27T06:42:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.716+0000 7f816c807700 1 -- 172.21.15.16:0/2239349507 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f8158004750 con 0x7f8168102180 2024-03-27T06:42:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.717+0000 7f816c807700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81500a3730 0x7f81500a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.717+0000 7f8166ffd700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81500a3730 0x7f81500a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.717+0000 7f816c807700 1 -- 172.21.15.16:0/2239349507 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f81580cfac0 con 0x7f8168102180 2024-03-27T06:42:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.717+0000 7f8166ffd700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81500a3730 0x7f81500a5bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f815c0104e0 tx=0x7f815c007480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:28.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.724+0000 7f816c807700 1 -- 172.21.15.16:0/2239349507 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f815809c4e0 con 0x7f8168102180 2024-03-27T06:42:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:28 smithi016 ceph-mon[31908]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:28 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3349292189' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-27T06:42:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:28 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2704909051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-27T06:42:28.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:28 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:42:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.938+0000 7f816d809700 1 -- 172.21.15.16:0/2239349507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f81681046e0 con 0x7f8168102180 2024-03-27T06:42:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.939+0000 7f816c807700 1 -- 172.21.15.16:0/2239349507 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f815809bde0 con 0x7f8168102180 2024-03-27T06:42:28.934 INFO:teuthology.orchestra.run.smithi016.stdout:111669149762 2024-03-27T06:42:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 -- 172.21.15.16:0/2239349507 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81500a3730 msgr2=0x7f81500a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81500a3730 0x7f81500a5bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f815c0104e0 tx=0x7f815c007480 comp rx=0 tx=0).stop 2024-03-27T06:42:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 -- 172.21.15.16:0/2239349507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8168102180 msgr2=0x7f81681036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8168102180 0x7f81681036f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f815800d750 tx=0x7f815800da60 comp rx=0 tx=0).stop 2024-03-27T06:42:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 -- 172.21.15.16:0/2239349507 shutdown_connections 2024-03-27T06:42:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81681068d0 0x7f8168103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81681017d0 0x7f81681031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f81500a3730 0x7f81500a5bf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 --2- 172.21.15.16:0/2239349507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8168102180 0x7f81681036f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 -- 172.21.15.16:0/2239349507 >> 172.21.15.16:0/2239349507 conn(0x7f8168077560 msgr2=0x7f8168105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 -- 172.21.15.16:0/2239349507 shutdown_connections 2024-03-27T06:42:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:28.941+0000 7f816d809700 1 -- 172.21.15.16:0/2239349507 wait complete. 2024-03-27T06:42:28.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:28 smithi156 ceph-mon[36261]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:28.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:28 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3349292189' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-27T06:42:28.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:28 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2704909051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-27T06:42:28.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:28 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:42:29.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:28 smithi170 ceph-mon[36438]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s; 27 B/s, 1 objects/s recovering 2024-03-27T06:42:29.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:28 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3349292189' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-27T06:42:29.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:28 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2704909051' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-27T06:42:29.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:28 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:42:29.670 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149756 got 111669149762 for osd.3 2024-03-27T06:42:29.671 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:29 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2239349507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-27T06:42:29.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:29 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2239349507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-27T06:42:30.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:29 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2239349507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-27T06:42:30.102 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299433 got 223338299437 for osd.7 2024-03-27T06:42:30.102 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.185+0000 7fcd0a245700 1 -- 172.21.15.16:0/3274982790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04107640 msgr2=0x7fcd04107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.185+0000 7fcd0a245700 1 --2- 172.21.15.16:0/3274982790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04107640 0x7fcd04107a20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009990 tx=0x7fccf4009ca0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.185+0000 7fcd0a245700 1 -- 172.21.15.16:0/3274982790 shutdown_connections 2024-03-27T06:42:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.185+0000 7fcd0a245700 1 --2- 172.21.15.16:0/3274982790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd04101ef0 0x7fcd0410e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.185+0000 7fcd0a245700 1 --2- 172.21.15.16:0/3274982790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04107640 0x7fcd04107a20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.185+0000 7fcd0a245700 1 --2- 172.21.15.16:0/3274982790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd04101530 0x7fcd041019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.185+0000 7fcd0a245700 1 -- 172.21.15.16:0/3274982790 >> 172.21.15.16:0/3274982790 conn(0x7fcd040784b0 msgr2=0x7fcd040788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.185+0000 7fcd0a245700 1 -- 172.21.15.16:0/3274982790 shutdown_connections 2024-03-27T06:42:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.185+0000 7fcd0a245700 1 -- 172.21.15.16:0/3274982790 wait complete. 2024-03-27T06:42:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd0a245700 1 Processor -- start 2024-03-27T06:42:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd0a245700 1 -- start start 2024-03-27T06:42:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd0a245700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd04101530 0x7fcd04198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd0a245700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd04101ef0 0x7fcd04199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd0a245700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04107640 0x7fcd0419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd0a245700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd04111930 con 0x7fcd04107640 2024-03-27T06:42:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd0a245700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fcd041117b0 con 0x7fcd04101530 2024-03-27T06:42:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd0a245700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcd04111ab0 con 0x7fcd04101ef0 2024-03-27T06:42:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd03fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04107640 0x7fcd0419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd03fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04107640 0x7fcd0419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52620/0 (socket says 172.21.15.16:52620) 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd03fff700 1 -- 172.21.15.16:0/3527594801 learned_addr learned my addr 172.21.15.16:0/3527594801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd037fe700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd04101530 0x7fcd04198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd02ffd700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd04101ef0 0x7fcd04199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd03fff700 1 -- 172.21.15.16:0/3527594801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd04101ef0 msgr2=0x7fcd04199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd03fff700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd04101ef0 0x7fcd04199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd03fff700 1 -- 172.21.15.16:0/3527594801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd04101530 msgr2=0x7fcd04198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd03fff700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd04101530 0x7fcd04198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.186+0000 7fcd03fff700 1 -- 172.21.15.16:0/3527594801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0419dde0 con 0x7fcd04107640 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.187+0000 7fcd03fff700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04107640 0x7fcd0419d6a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00ef30 tx=0x7fccfc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.187+0000 7fcd00ff9700 1 -- 172.21.15.16:0/3527594801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0092e0 con 0x7fcd04107640 2024-03-27T06:42:31.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.187+0000 7fcd0a245700 1 -- 172.21.15.16:0/3527594801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0419e070 con 0x7fcd04107640 2024-03-27T06:42:31.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.187+0000 7fcd00ff9700 1 -- 172.21.15.16:0/3527594801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fccfc004bb0 con 0x7fcd04107640 2024-03-27T06:42:31.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.187+0000 7fcd00ff9700 1 -- 172.21.15.16:0/3527594801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc004280 con 0x7fcd04107640 2024-03-27T06:42:31.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.187+0000 7fcd0a245700 1 -- 172.21.15.16:0/3527594801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd041a5950 con 0x7fcd04107640 2024-03-27T06:42:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.188+0000 7fcd0a245700 1 -- 172.21.15.16:0/3527594801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd04102c50 con 0x7fcd04107640 2024-03-27T06:42:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.189+0000 7fcd00ff9700 1 -- 172.21.15.16:0/3527594801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fccfc01b030 con 0x7fcd04107640 2024-03-27T06:42:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.189+0000 7fcd00ff9700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fccec0a3730 0x7fccec0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.190+0000 7fcd037fe700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fccec0a3730 0x7fccec0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.190+0000 7fcd00ff9700 1 -- 172.21.15.16:0/3527594801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fccfc0cf920 con 0x7fcd04107640 2024-03-27T06:42:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.190+0000 7fcd037fe700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fccec0a3730 0x7fccec0a5bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fccf40098e0 tx=0x7fccf4005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:31.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.194+0000 7fcd00ff9700 1 -- 172.21.15.16:0/3527594801 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fccfc022020 con 0x7fcd04107640 2024-03-27T06:42:31.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.403+0000 7fcd0a245700 1 -- 172.21.15.16:0/3527594801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fcd041a5b20 con 0x7fcd04107640 2024-03-27T06:42:31.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.404+0000 7fcd00ff9700 1 -- 172.21.15.16:0/3527594801 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fccfc010040 con 0x7fcd04107640 2024-03-27T06:42:31.398 INFO:teuthology.orchestra.run.smithi016.stdout:77309411400 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 -- 172.21.15.16:0/3527594801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fccec0a3730 msgr2=0x7fccec0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fccec0a3730 0x7fccec0a5bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fccf40098e0 tx=0x7fccf4005960 comp rx=0 tx=0).stop 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 -- 172.21.15.16:0/3527594801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04107640 msgr2=0x7fcd0419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04107640 0x7fcd0419d6a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00ef30 tx=0x7fccfc00c3a0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 -- 172.21.15.16:0/3527594801 shutdown_connections 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04107640 0x7fcd0419d6a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd04101530 0x7fcd04198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fccec0a3730 0x7fccec0a5bf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 --2- 172.21.15.16:0/3527594801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd04101ef0 0x7fcd04199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 -- 172.21.15.16:0/3527594801 >> 172.21.15.16:0/3527594801 conn(0x7fcd040784b0 msgr2=0x7fcd04100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 -- 172.21.15.16:0/3527594801 shutdown_connections 2024-03-27T06:42:31.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.406+0000 7fcd0a245700 1 -- 172.21.15.16:0/3527594801 wait complete. 2024-03-27T06:42:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.429+0000 7fef84d0f700 1 -- 172.21.15.16:0/3680773706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef801085f0 msgr2=0x7fef801089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.429+0000 7fef84d0f700 1 --2- 172.21.15.16:0/3680773706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef801085f0 0x7fef801089d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fef700099c0 tx=0x7fef70009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.430+0000 7fef84d0f700 1 -- 172.21.15.16:0/3680773706 shutdown_connections 2024-03-27T06:42:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.430+0000 7fef84d0f700 1 --2- 172.21.15.16:0/3680773706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef8006b540 0x7fef8010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.430+0000 7fef84d0f700 1 --2- 172.21.15.16:0/3680773706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef801085f0 0x7fef801089d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.430+0000 7fef84d0f700 1 --2- 172.21.15.16:0/3680773706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8006ab80 0x7fef8006b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.430+0000 7fef84d0f700 1 -- 172.21.15.16:0/3680773706 >> 172.21.15.16:0/3680773706 conn(0x7fef80077590 msgr2=0x7fef800779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.430+0000 7fef84d0f700 1 -- 172.21.15.16:0/3680773706 shutdown_connections 2024-03-27T06:42:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.430+0000 7fef84d0f700 1 -- 172.21.15.16:0/3680773706 wait complete. 2024-03-27T06:42:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef84d0f700 1 Processor -- start 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef84d0f700 1 -- start start 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef84d0f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8006ab80 0x7fef80198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef84d0f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8006b540 0x7fef80199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef84d0f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef801085f0 0x7fef8019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef84d0f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef80112a50 con 0x7fef8006b540 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef84d0f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fef801128d0 con 0x7fef801085f0 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef84d0f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef80112bd0 con 0x7fef8006ab80 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef7dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8006b540 0x7fef80199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef7ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef801085f0 0x7fef8019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef7e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8006ab80 0x7fef80198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef7e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8006ab80 0x7fef80198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:43252/0 (socket says 172.21.15.16:43252) 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef7dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8006b540 0x7fef80199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52632/0 (socket says 172.21.15.16:52632) 2024-03-27T06:42:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef7ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef801085f0 0x7fef8019d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:57770/0 (socket says 172.21.15.16:57770) 2024-03-27T06:42:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef7e59c700 1 -- 172.21.15.16:0/3827504217 learned_addr learned my addr 172.21.15.16:0/3827504217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef7e59c700 1 -- 172.21.15.16:0/3827504217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef801085f0 msgr2=0x7fef8019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef7e59c700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef801085f0 0x7fef8019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef7e59c700 1 -- 172.21.15.16:0/3827504217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8006b540 msgr2=0x7fef80199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.431+0000 7fef7e59c700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8006b540 0x7fef80199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.432+0000 7fef7e59c700 1 -- 172.21.15.16:0/3827504217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8019de30 con 0x7fef8006ab80 2024-03-27T06:42:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.432+0000 7fef7e59c700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8006ab80 0x7fef80198e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef700059d0 tx=0x7fef7000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.432+0000 7fef6f7fe700 1 -- 172.21.15.16:0/3827504217 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70027070 con 0x7fef8006ab80 2024-03-27T06:42:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.432+0000 7fef84d0f700 1 -- 172.21.15.16:0/3827504217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8019e0c0 con 0x7fef8006ab80 2024-03-27T06:42:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.432+0000 7fef6f7fe700 1 -- 172.21.15.16:0/3827504217 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fef7000be40 con 0x7fef8006ab80 2024-03-27T06:42:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.432+0000 7fef6f7fe700 1 -- 172.21.15.16:0/3827504217 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef700173f0 con 0x7fef8006ab80 2024-03-27T06:42:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.432+0000 7fef84d0f700 1 -- 172.21.15.16:0/3827504217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef801a59a0 con 0x7fef8006ab80 2024-03-27T06:42:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.433+0000 7fef84d0f700 1 -- 172.21.15.16:0/3827504217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef80103d70 con 0x7fef8006ab80 2024-03-27T06:42:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.434+0000 7fef6f7fe700 1 -- 172.21.15.16:0/3827504217 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fef7002a070 con 0x7fef8006ab80 2024-03-27T06:42:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.434+0000 7fef6f7fe700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fef680a37b0 0x7fef680a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.435+0000 7fef7dd9b700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fef680a37b0 0x7fef680a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.435+0000 7fef6f7fe700 1 -- 172.21.15.16:0/3827504217 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fef700d0ab0 con 0x7fef8006ab80 2024-03-27T06:42:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.435+0000 7fef7dd9b700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fef680a37b0 0x7fef680a5c70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef80077b90 tx=0x7fef74009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.439+0000 7fef6f7fe700 1 -- 172.21.15.16:0/3827504217 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fef7005ecb0 con 0x7fef8006ab80 2024-03-27T06:42:31.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:31 smithi156 ceph-mon[36261]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 22 B/s, 1 objects/s recovering 2024-03-27T06:42:31.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:31 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:31 smithi170 ceph-mon[36438]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 22 B/s, 1 objects/s recovering 2024-03-27T06:42:31.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:31 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:31 smithi016 ceph-mon[31908]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 22 B/s, 1 objects/s recovering 2024-03-27T06:42:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:31 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:31.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.652+0000 7fef84d0f700 1 -- 172.21.15.16:0/3827504217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fef8002dd30 con 0x7fef8006ab80 2024-03-27T06:42:31.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.652+0000 7fef6f7fe700 1 -- 172.21.15.16:0/3827504217 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fef70013070 con 0x7fef8006ab80 2024-03-27T06:42:31.647 INFO:teuthology.orchestra.run.smithi016.stdout:257698037800 2024-03-27T06:42:31.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 -- 172.21.15.16:0/3827504217 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fef680a37b0 msgr2=0x7fef680a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:31.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fef680a37b0 0x7fef680a5c70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef80077b90 tx=0x7fef74009260 comp rx=0 tx=0).stop 2024-03-27T06:42:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 -- 172.21.15.16:0/3827504217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8006ab80 msgr2=0x7fef80198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8006ab80 0x7fef80198e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef700059d0 tx=0x7fef7000f690 comp rx=0 tx=0).stop 2024-03-27T06:42:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 -- 172.21.15.16:0/3827504217 shutdown_connections 2024-03-27T06:42:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef801085f0 0x7fef8019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fef680a37b0 0x7fef680a5c70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8006ab80 0x7fef80198e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 --2- 172.21.15.16:0/3827504217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8006b540 0x7fef80199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 -- 172.21.15.16:0/3827504217 >> 172.21.15.16:0/3827504217 conn(0x7fef80077590 msgr2=0x7fef801001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 -- 172.21.15.16:0/3827504217 shutdown_connections 2024-03-27T06:42:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:31.655+0000 7fef84d0f700 1 -- 172.21.15.16:0/3827504217 wait complete. 2024-03-27T06:42:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:32.460 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:32 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3527594801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-27T06:42:32.460 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:32 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3827504217' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-27T06:42:32.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:32 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3527594801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-27T06:42:32.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:32 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3827504217' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-27T06:42:32.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:32 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3527594801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-27T06:42:32.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:32 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3827504217' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-27T06:42:32.496 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776159 got 292057776163 for osd.9 2024-03-27T06:42:32.496 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.149+0000 7f17bc0a8700 1 -- 172.21.15.16:0/1418001757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b406ad10 msgr2=0x7f17b406b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.149+0000 7f17bc0a8700 1 --2- 172.21.15.16:0/1418001757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b406ad10 0x7f17b406b190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f17a40099c0 tx=0x7f17a4009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.149+0000 7f17bc0a8700 1 -- 172.21.15.16:0/1418001757 shutdown_connections 2024-03-27T06:42:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.149+0000 7f17bc0a8700 1 --2- 172.21.15.16:0/1418001757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17b406b6d0 0x7f17b410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.149+0000 7f17bc0a8700 1 --2- 172.21.15.16:0/1418001757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b41085e0 0x7f17b41089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.149+0000 7f17bc0a8700 1 --2- 172.21.15.16:0/1418001757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b406ad10 0x7f17b406b190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.149+0000 7f17bc0a8700 1 -- 172.21.15.16:0/1418001757 >> 172.21.15.16:0/1418001757 conn(0x7f17b40784b0 msgr2=0x7f17b40788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.149+0000 7f17bc0a8700 1 -- 172.21.15.16:0/1418001757 shutdown_connections 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.149+0000 7f17bc0a8700 1 -- 172.21.15.16:0/1418001757 wait complete. 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17bc0a8700 1 Processor -- start 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17bc0a8700 1 -- start start 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17bc0a8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b406ad10 0x7f17b4198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17bc0a8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17b406b6d0 0x7f17b4199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17bc0a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b41085e0 0x7f17b419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17bc0a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41129f0 con 0x7f17b41085e0 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17bc0a8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f17b4112870 con 0x7f17b406b6d0 2024-03-27T06:42:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17bc0a8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f17b4112b70 con 0x7f17b406ad10 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17ba645700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b41085e0 0x7f17b419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17b9e44700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b406ad10 0x7f17b4198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17ba645700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b41085e0 0x7f17b419d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52660/0 (socket says 172.21.15.16:52660) 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17ba645700 1 -- 172.21.15.16:0/2538048556 learned_addr learned my addr 172.21.15.16:0/2538048556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17b9643700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17b406b6d0 0x7f17b4199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17b9e44700 1 -- 172.21.15.16:0/2538048556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17b406b6d0 msgr2=0x7f17b4199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17b9e44700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17b406b6d0 0x7f17b4199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17b9e44700 1 -- 172.21.15.16:0/2538048556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b41085e0 msgr2=0x7f17b419d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17b9e44700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b41085e0 0x7f17b419d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17b9e44700 1 -- 172.21.15.16:0/2538048556 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b419dec0 con 0x7f17b406ad10 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17ba645700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b41085e0 0x7f17b419d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.150+0000 7f17b9e44700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b406ad10 0x7f17b4198f00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17b000ef10 tx=0x7f17b000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.151+0000 7f17aaffd700 1 -- 172.21.15.16:0/2538048556 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b000c9c0 con 0x7f17b406ad10 2024-03-27T06:42:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.151+0000 7f17bc0a8700 1 -- 172.21.15.16:0/2538048556 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b419e150 con 0x7f17b406ad10 2024-03-27T06:42:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.151+0000 7f17aaffd700 1 -- 172.21.15.16:0/2538048556 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17b0019070 con 0x7f17b406ad10 2024-03-27T06:42:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.151+0000 7f17aaffd700 1 -- 172.21.15.16:0/2538048556 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b00146e0 con 0x7f17b406ad10 2024-03-27T06:42:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.151+0000 7f17bc0a8700 1 -- 172.21.15.16:0/2538048556 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b41a5a30 con 0x7f17b406ad10 2024-03-27T06:42:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.151+0000 7f17bc0a8700 1 -- 172.21.15.16:0/2538048556 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b4103d60 con 0x7f17b406ad10 2024-03-27T06:42:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.153+0000 7f17aaffd700 1 -- 172.21.15.16:0/2538048556 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f17b0012050 con 0x7f17b406ad10 2024-03-27T06:42:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.153+0000 7f17aaffd700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f17a00a3730 0x7f17a00a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.153+0000 7f17b9643700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f17a00a3730 0x7f17a00a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.153+0000 7f17aaffd700 1 -- 172.21.15.16:0/2538048556 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f17b00cfd90 con 0x7f17b406ad10 2024-03-27T06:42:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.154+0000 7f17b9643700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f17a00a3730 0x7f17a00a5bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f17b4078a90 tx=0x7f17a4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.157+0000 7f17aaffd700 1 -- 172.21.15.16:0/2538048556 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f17b0010040 con 0x7f17b406ad10 2024-03-27T06:42:33.293 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:33 smithi016 ceph-mon[31908]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 20 B/s, 1 objects/s recovering 2024-03-27T06:42:33.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.371+0000 7f17bc0a8700 1 -- 172.21.15.16:0/2538048556 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f17b402dd30 con 0x7f17b406ad10 2024-03-27T06:42:33.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.371+0000 7f17aaffd700 1 -- 172.21.15.16:0/2538048556 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f17b009c810 con 0x7f17b406ad10 2024-03-27T06:42:33.366 INFO:teuthology.orchestra.run.smithi016.stdout:146028888126 2024-03-27T06:42:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 -- 172.21.15.16:0/2538048556 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f17a00a3730 msgr2=0x7f17a00a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f17a00a3730 0x7f17a00a5bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f17b4078a90 tx=0x7f17a4005fb0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 -- 172.21.15.16:0/2538048556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b406ad10 msgr2=0x7f17b4198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b406ad10 0x7f17b4198f00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17b000ef10 tx=0x7f17b000c3a0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 -- 172.21.15.16:0/2538048556 shutdown_connections 2024-03-27T06:42:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b41085e0 0x7f17b419d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f17a00a3730 0x7f17a00a5bf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b406ad10 0x7f17b4198f00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 --2- 172.21.15.16:0/2538048556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17b406b6d0 0x7f17b4199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 -- 172.21.15.16:0/2538048556 >> 172.21.15.16:0/2538048556 conn(0x7f17b40784b0 msgr2=0x7f17b41001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 -- 172.21.15.16:0/2538048556 shutdown_connections 2024-03-27T06:42:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:33.373+0000 7f17bc0a8700 1 -- 172.21.15.16:0/2538048556 wait complete. 2024-03-27T06:42:33.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:33 smithi156 ceph-mon[36261]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 20 B/s, 1 objects/s recovering 2024-03-27T06:42:33.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:33 smithi170 ceph-mon[36438]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 20 B/s, 1 objects/s recovering 2024-03-27T06:42:34.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:34 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2538048556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-27T06:42:34.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:34 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2538048556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-27T06:42:34.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:34 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2538048556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-27T06:42:35.078 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411400 for osd.2 2024-03-27T06:42:35.079 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:35 smithi016 ceph-mon[31908]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-03-27T06:42:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:35 smithi156 ceph-mon[36261]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-03-27T06:42:35.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:35 smithi170 ceph-mon[36438]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-03-27T06:42:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.058+0000 7f8dca729700 1 -- 172.21.15.16:0/2726237258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc4108640 msgr2=0x7f8dc4108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.058+0000 7f8dca729700 1 --2- 172.21.15.16:0/2726237258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc4108640 0x7f8dc4108a20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009990 tx=0x7f8db4009ca0 comp rx=0 tx=0).stop 2024-03-27T06:42:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.058+0000 7f8dca729700 1 -- 172.21.15.16:0/2726237258 shutdown_connections 2024-03-27T06:42:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.058+0000 7f8dca729700 1 --2- 172.21.15.16:0/2726237258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc4103080 0x7f8dc410f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.058+0000 7f8dca729700 1 --2- 172.21.15.16:0/2726237258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc4108640 0x7f8dc4108a20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.058+0000 7f8dca729700 1 --2- 172.21.15.16:0/2726237258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc41026c0 0x7f8dc4102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.058+0000 7f8dca729700 1 -- 172.21.15.16:0/2726237258 >> 172.21.15.16:0/2726237258 conn(0x7f8dc4079790 msgr2=0x7f8dc4079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.058+0000 7f8dca729700 1 -- 172.21.15.16:0/2726237258 shutdown_connections 2024-03-27T06:42:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.058+0000 7f8dca729700 1 -- 172.21.15.16:0/2726237258 wait complete. 2024-03-27T06:42:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dca729700 1 Processor -- start 2024-03-27T06:42:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dca729700 1 -- start start 2024-03-27T06:42:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dca729700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc41026c0 0x7f8dc4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dca729700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc4103080 0x7f8dc41993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dca729700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc4108640 0x7f8dc419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dca729700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc4112ae0 con 0x7f8dc41026c0 2024-03-27T06:42:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dca729700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc4112960 con 0x7f8dc4108640 2024-03-27T06:42:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dca729700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc4112c60 con 0x7f8dc4103080 2024-03-27T06:42:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dc3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc41026c0 0x7f8dc4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dc3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc41026c0 0x7f8dc4198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56022/0 (socket says 172.21.15.16:56022) 2024-03-27T06:42:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dc3fff700 1 -- 172.21.15.16:0/285012786 learned_addr learned my addr 172.21.15.16:0/285012786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dc37fe700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc4103080 0x7f8dc41993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.059+0000 7f8dc8cc6700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc4108640 0x7f8dc419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.060+0000 7f8dc3fff700 1 -- 172.21.15.16:0/285012786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc4103080 msgr2=0x7f8dc41993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.060+0000 7f8dc3fff700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc4103080 0x7f8dc41993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.060+0000 7f8dc3fff700 1 -- 172.21.15.16:0/285012786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc4108640 msgr2=0x7f8dc419d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.060+0000 7f8dc3fff700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc4108640 0x7f8dc419d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.060+0000 7f8dc3fff700 1 -- 172.21.15.16:0/285012786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc419dea0 con 0x7f8dc41026c0 2024-03-27T06:42:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.060+0000 7f8dc3fff700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc41026c0 0x7f8dc4198e90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8db4005f50 tx=0x7f8db400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.060+0000 7f8dc17fa700 1 -- 172.21.15.16:0/285012786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4027070 con 0x7f8dc41026c0 2024-03-27T06:42:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.060+0000 7f8dca729700 1 -- 172.21.15.16:0/285012786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc419e130 con 0x7f8dc41026c0 2024-03-27T06:42:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.060+0000 7f8dc17fa700 1 -- 172.21.15.16:0/285012786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8db400be60 con 0x7f8dc41026c0 2024-03-27T06:42:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.060+0000 7f8dc17fa700 1 -- 172.21.15.16:0/285012786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db40173f0 con 0x7f8dc41026c0 2024-03-27T06:42:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.060+0000 7f8dca729700 1 -- 172.21.15.16:0/285012786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc41a5a10 con 0x7f8dc41026c0 2024-03-27T06:42:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.062+0000 7f8dca729700 1 -- 172.21.15.16:0/285012786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc4069300 con 0x7f8dc41026c0 2024-03-27T06:42:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.064+0000 7f8dc17fa700 1 -- 172.21.15.16:0/285012786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f8db402a070 con 0x7f8dc41026c0 2024-03-27T06:42:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.064+0000 7f8dc17fa700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8dac0a3730 0x7f8dac0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.065+0000 7f8dc37fe700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8dac0a3730 0x7f8dac0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.065+0000 7f8dc17fa700 1 -- 172.21.15.16:0/285012786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8db40d0c20 con 0x7f8dc41026c0 2024-03-27T06:42:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.065+0000 7f8dc37fe700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8dac0a3730 0x7f8dac0a5bf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4079c40 tx=0x7f8db8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.071+0000 7f8dc17fa700 1 -- 172.21.15.16:0/285012786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8db4030070 con 0x7f8dc41026c0 2024-03-27T06:42:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.280+0000 7f8dca729700 1 -- 172.21.15.16:0/285012786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8dc41a5be0 con 0x7f8dc41026c0 2024-03-27T06:42:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.281+0000 7f8dc17fa700 1 -- 172.21.15.16:0/285012786 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8db4013070 con 0x7f8dc41026c0 2024-03-27T06:42:37.276 INFO:teuthology.orchestra.run.smithi016.stdout:55834574926 2024-03-27T06:42:37.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 -- 172.21.15.16:0/285012786 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8dac0a3730 msgr2=0x7f8dac0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8dac0a3730 0x7f8dac0a5bf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4079c40 tx=0x7f8db8009260 comp rx=0 tx=0).stop 2024-03-27T06:42:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 -- 172.21.15.16:0/285012786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc41026c0 msgr2=0x7f8dc4198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc41026c0 0x7f8dc4198e90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8db4005f50 tx=0x7f8db400b840 comp rx=0 tx=0).stop 2024-03-27T06:42:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 -- 172.21.15.16:0/285012786 shutdown_connections 2024-03-27T06:42:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc4108640 0x7f8dc419d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc41026c0 0x7f8dc4198e90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8dac0a3730 0x7f8dac0a5bf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 --2- 172.21.15.16:0/285012786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc4103080 0x7f8dc41993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 -- 172.21.15.16:0/285012786 >> 172.21.15.16:0/285012786 conn(0x7f8dc4079790 msgr2=0x7f8dc4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 -- 172.21.15.16:0/285012786 shutdown_connections 2024-03-27T06:42:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:37.283+0000 7f8dca729700 1 -- 172.21.15.16:0/285012786 wait complete. 2024-03-27T06:42:37.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:37 smithi016 ceph-mon[31908]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-27T06:42:37.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:37 smithi156 ceph-mon[36261]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-27T06:42:37.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:37 smithi170 ceph-mon[36438]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-27T06:42:38.175 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888122 got 146028888126 for osd.4 2024-03-27T06:42:38.175 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:38.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:38 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/285012786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-27T06:42:38.581 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:38 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/285012786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-27T06:42:38.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:38 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/285012786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-27T06:42:38.613 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037795 got 257698037800 for osd.8 2024-03-27T06:42:38.613 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:39.462 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:39 smithi016 ceph-mon[31908]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:39.471 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:39 smithi156 ceph-mon[36261]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:39.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:39 smithi170 ceph-mon[36438]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:40.033 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574926 for osd.1 2024-03-27T06:42:40.034 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:41 smithi016 ceph-mon[31908]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:42:41.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:41 smithi156 ceph-mon[36261]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:42:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:41 smithi170 ceph-mon[36438]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:42:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 -- 172.21.15.16:0/1938339993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc107640 msgr2=0x7f21cc107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 --2- 172.21.15.16:0/1938339993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc107640 0x7f21cc107a20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009990 tx=0x7f21c8009ca0 comp rx=0 tx=0).stop 2024-03-27T06:42:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 -- 172.21.15.16:0/1938339993 shutdown_connections 2024-03-27T06:42:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 --2- 172.21.15.16:0/1938339993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21cc101ef0 0x7f21cc10e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 --2- 172.21.15.16:0/1938339993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc107640 0x7f21cc107a20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 --2- 172.21.15.16:0/1938339993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc101530 0x7f21cc1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 -- 172.21.15.16:0/1938339993 >> 172.21.15.16:0/1938339993 conn(0x7f21cc0784b0 msgr2=0x7f21cc0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 -- 172.21.15.16:0/1938339993 shutdown_connections 2024-03-27T06:42:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 -- 172.21.15.16:0/1938339993 wait complete. 2024-03-27T06:42:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 Processor -- start 2024-03-27T06:42:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 -- start start 2024-03-27T06:42:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.653+0000 7f21d44f5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21cc101530 0x7f21cc199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d44f5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc101ef0 0x7f21cc19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d44f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc107640 0x7f21cc1a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d44f5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc111950 con 0x7f21cc107640 2024-03-27T06:42:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d44f5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc1117d0 con 0x7f21cc101530 2024-03-27T06:42:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d44f5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc111ad0 con 0x7f21cc101ef0 2024-03-27T06:42:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d2a92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc107640 0x7f21cc1a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d2291700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21cc101530 0x7f21cc199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:41.965 INFO:teuthology.orchestra.run.smithi016.stdout:201863462966 2024-03-27T06:42:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d1a90700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc101ef0 0x7f21cc19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d2a92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc107640 0x7f21cc1a4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56026/0 (socket says 172.21.15.16:56026) 2024-03-27T06:42:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d2a92700 1 -- 172.21.15.16:0/1385226160 learned_addr learned my addr 172.21.15.16:0/1385226160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d2a92700 1 -- 172.21.15.16:0/1385226160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc101ef0 msgr2=0x7f21cc19a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d2a92700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc101ef0 0x7f21cc19a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d2a92700 1 -- 172.21.15.16:0/1385226160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21cc101530 msgr2=0x7f21cc199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d2a92700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21cc101530 0x7f21cc199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d2a92700 1 -- 172.21.15.16:0/1385226160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc1a5210 con 0x7f21cc107640 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d1a90700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc101ef0 0x7f21cc19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d2291700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21cc101530 0x7f21cc199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d2a92700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc107640 0x7f21cc1a4cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f21c4007ee0 tx=0x7f21c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21c37fe700 1 -- 172.21.15.16:0/1385226160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c400dd20 con 0x7f21cc107640 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21d44f5700 1 -- 172.21.15.16:0/1385226160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc1a54a0 con 0x7f21cc107640 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21c37fe700 1 -- 172.21.15.16:0/1385226160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f21c4004020 con 0x7f21cc107640 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.654+0000 7f21c37fe700 1 -- 172.21.15.16:0/1385226160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4015660 con 0x7f21cc107640 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.655+0000 7f21d44f5700 1 -- 172.21.15.16:0/1385226160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc1a5960 con 0x7f21cc107640 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.656+0000 7f21d44f5700 1 -- 172.21.15.16:0/1385226160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc109a90 con 0x7f21cc107640 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.657+0000 7f21c37fe700 1 -- 172.21.15.16:0/1385226160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f21c40041d0 con 0x7f21cc107640 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.657+0000 7f21c37fe700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f21b80a3730 0x7f21b80a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.658+0000 7f21d2291700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f21b80a3730 0x7f21b80a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.658+0000 7f21c37fe700 1 -- 172.21.15.16:0/1385226160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f21c40cf7a0 con 0x7f21cc107640 2024-03-27T06:42:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.658+0000 7f21d2291700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f21b80a3730 0x7f21b80a5bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f21c800b2e0 tx=0x7f21c801a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.662+0000 7f21c37fe700 1 -- 172.21.15.16:0/1385226160 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f21c4010070 con 0x7f21cc107640 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.881+0000 7f21d44f5700 1 -- 172.21.15.16:0/1385226160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f21cc06b330 con 0x7f21cc107640 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.882+0000 7f21c37fe700 1 -- 172.21.15.16:0/1385226160 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f21c409c220 con 0x7f21cc107640 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 -- 172.21.15.16:0/1385226160 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f21b80a3730 msgr2=0x7f21b80a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f21b80a3730 0x7f21b80a5bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f21c800b2e0 tx=0x7f21c801a040 comp rx=0 tx=0).stop 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 -- 172.21.15.16:0/1385226160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc107640 msgr2=0x7f21cc1a4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc107640 0x7f21cc1a4cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f21c4007ee0 tx=0x7f21c400d3b0 comp rx=0 tx=0).stop 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 -- 172.21.15.16:0/1385226160 shutdown_connections 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21cc107640 0x7f21cc1a4cd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21cc101530 0x7f21cc199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f21b80a3730 0x7f21b80a5bf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 --2- 172.21.15.16:0/1385226160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21cc101ef0 0x7f21cc19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 -- 172.21.15.16:0/1385226160 >> 172.21.15.16:0/1385226160 conn(0x7f21cc0784b0 msgr2=0x7f21cc1000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 -- 172.21.15.16:0/1385226160 shutdown_connections 2024-03-27T06:42:41.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.884+0000 7f21d44f5700 1 -- 172.21.15.16:0/1385226160 wait complete. 2024-03-27T06:42:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.754+0000 7fe8b031a700 1 -- 172.21.15.16:0/3900398646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a8101f00 msgr2=0x7fe8a810e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.754+0000 7fe8b031a700 1 --2- 172.21.15.16:0/3900398646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a8101f00 0x7fe8a810e390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000b0a0 tx=0x7fe8a000b3b0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.754+0000 7fe8b031a700 1 -- 172.21.15.16:0/3900398646 shutdown_connections 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.754+0000 7fe8b031a700 1 --2- 172.21.15.16:0/3900398646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a8101f00 0x7fe8a810e390 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.754+0000 7fe8b031a700 1 --2- 172.21.15.16:0/3900398646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8107550 0x7fe8a8107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.754+0000 7fe8b031a700 1 --2- 172.21.15.16:0/3900398646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a8101540 0x7fe8a81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.754+0000 7fe8b031a700 1 -- 172.21.15.16:0/3900398646 >> 172.21.15.16:0/3900398646 conn(0x7fe8a8078470 msgr2=0x7fe8a8078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.754+0000 7fe8b031a700 1 -- 172.21.15.16:0/3900398646 shutdown_connections 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.754+0000 7fe8b031a700 1 -- 172.21.15.16:0/3900398646 wait complete. 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8b031a700 1 Processor -- start 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8b031a700 1 -- start start 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8b031a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a8101540 0x7fe8a819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8b031a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8101f00 0x7fe8a819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8b031a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a8107550 0x7fe8a819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8b031a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8111930 con 0x7fe8a8101540 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8b031a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81117b0 con 0x7fe8a8107550 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8b031a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8111ab0 con 0x7fe8a8101f00 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ae0b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a8101540 0x7fe8a819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ae0b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a8101540 0x7fe8a819afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56048/0 (socket says 172.21.15.16:56048) 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ae0b6700 1 -- 172.21.15.16:0/2289796145 learned_addr learned my addr 172.21.15.16:0/2289796145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ad8b5700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8101f00 0x7fe8a819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ae8b7700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a8107550 0x7fe8a819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ae0b6700 1 -- 172.21.15.16:0/2289796145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8101f00 msgr2=0x7fe8a819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ae0b6700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8101f00 0x7fe8a819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ae0b6700 1 -- 172.21.15.16:0/2289796145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a8107550 msgr2=0x7fe8a819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ae0b6700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a8107550 0x7fe8a819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ae0b6700 1 -- 172.21.15.16:0/2289796145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a819ffb0 con 0x7fe8a8101540 2024-03-27T06:42:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ad8b5700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8101f00 0x7fe8a819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.755+0000 7fe8ae0b6700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a8101540 0x7fe8a819afa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00e9c0 tx=0x7fe89c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.756+0000 7fe89b7fe700 1 -- 172.21.15.16:0/2289796145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c00cbd0 con 0x7fe8a8101540 2024-03-27T06:42:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.756+0000 7fe8b031a700 1 -- 172.21.15.16:0/2289796145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a81a02a0 con 0x7fe8a8101540 2024-03-27T06:42:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.756+0000 7fe89b7fe700 1 -- 172.21.15.16:0/2289796145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe89c00ee60 con 0x7fe8a8101540 2024-03-27T06:42:42.277 INFO:teuthology.orchestra.run.smithi016.stdout:34359738451 2024-03-27T06:42:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.756+0000 7fe89b7fe700 1 -- 172.21.15.16:0/2289796145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c010430 con 0x7fe8a8101540 2024-03-27T06:42:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.756+0000 7fe8b031a700 1 -- 172.21.15.16:0/2289796145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a81a7b20 con 0x7fe8a8101540 2024-03-27T06:42:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.757+0000 7fe8b031a700 1 -- 172.21.15.16:0/2289796145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a8069300 con 0x7fe8a8101540 2024-03-27T06:42:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.758+0000 7fe89b7fe700 1 -- 172.21.15.16:0/2289796145 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7fe89c004750 con 0x7fe8a8101540 2024-03-27T06:42:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.758+0000 7fe89b7fe700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe8940a3730 0x7fe8940a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.758+0000 7fe8ad8b5700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe8940a3730 0x7fe8940a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.758+0000 7fe89b7fe700 1 -- 172.21.15.16:0/2289796145 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe89c0d0100 con 0x7fe8a8101540 2024-03-27T06:42:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.759+0000 7fe8ad8b5700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe8940a3730 0x7fe8940a5bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe8a810e6c0 tx=0x7fe8a4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.763+0000 7fe89b7fe700 1 -- 172.21.15.16:0/2289796145 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fe89c09cb80 con 0x7fe8a8101540 2024-03-27T06:42:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.977+0000 7fe8b031a700 1 -- 172.21.15.16:0/2289796145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe8a81a7d50 con 0x7fe8a8101540 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.977+0000 7fe89b7fe700 1 -- 172.21.15.16:0/2289796145 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe89c09c480 con 0x7fe8a8101540 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 -- 172.21.15.16:0/2289796145 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe8940a3730 msgr2=0x7fe8940a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe8940a3730 0x7fe8940a5bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe8a810e6c0 tx=0x7fe8a4009260 comp rx=0 tx=0).stop 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 -- 172.21.15.16:0/2289796145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a8101540 msgr2=0x7fe8a819afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a8101540 0x7fe8a819afa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00e9c0 tx=0x7fe89c00ecd0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 -- 172.21.15.16:0/2289796145 shutdown_connections 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8a8107550 0x7fe8a819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8a8101540 0x7fe8a819afa0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe8940a3730 0x7fe8940a5bf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 --2- 172.21.15.16:0/2289796145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8101f00 0x7fe8a819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 -- 172.21.15.16:0/2289796145 >> 172.21.15.16:0/2289796145 conn(0x7fe8a8078470 msgr2=0x7fe8a806a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 -- 172.21.15.16:0/2289796145 shutdown_connections 2024-03-27T06:42:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:41.979+0000 7fe8b031a700 1 -- 172.21.15.16:0/2289796145 wait complete. 2024-03-27T06:42:42.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:42 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1385226160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-27T06:42:42.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:42 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2289796145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-27T06:42:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:42 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1385226160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-27T06:42:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:42 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2289796145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-27T06:42:42.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:42 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1385226160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-27T06:42:42.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:42 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2289796145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-27T06:42:42.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.734+0000 7f5de98a6700 1 -- 172.21.15.16:0/330264742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de41085e0 msgr2=0x7f5de41089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.734+0000 7f5de98a6700 1 --2- 172.21.15.16:0/330264742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de41085e0 0x7f5de41089c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40099c0 tx=0x7f5dd4009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.734+0000 7f5de98a6700 1 -- 172.21.15.16:0/330264742 shutdown_connections 2024-03-27T06:42:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.734+0000 7f5de98a6700 1 --2- 172.21.15.16:0/330264742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5de406b6d0 0x7f5de410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.734+0000 7f5de98a6700 1 --2- 172.21.15.16:0/330264742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de41085e0 0x7f5de41089c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.734+0000 7f5de98a6700 1 --2- 172.21.15.16:0/330264742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de406ad10 0x7f5de406b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.734+0000 7f5de98a6700 1 -- 172.21.15.16:0/330264742 >> 172.21.15.16:0/330264742 conn(0x7f5de40784b0 msgr2=0x7f5de40788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.734+0000 7f5de98a6700 1 -- 172.21.15.16:0/330264742 shutdown_connections 2024-03-27T06:42:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.734+0000 7f5de98a6700 1 -- 172.21.15.16:0/330264742 wait complete. 2024-03-27T06:42:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de98a6700 1 Processor -- start 2024-03-27T06:42:42.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de98a6700 1 -- start start 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de98a6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de406ad10 0x7f5de419b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de98a6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de406b6d0 0x7f5de419b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de98a6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5de41085e0 0x7f5de419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de98a6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4112a00 con 0x7f5de406ad10 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de98a6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4112880 con 0x7f5de41085e0 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de98a6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4112b80 con 0x7f5de406b6d0 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de406ad10 0x7f5de419b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de406ad10 0x7f5de419b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56066/0 (socket says 172.21.15.16:56066) 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de2ffd700 1 -- 172.21.15.16:0/3123492837 learned_addr learned my addr 172.21.15.16:0/3123492837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5ddbfff700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de406b6d0 0x7f5de419b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de37fe700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5de41085e0 0x7f5de419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de2ffd700 1 -- 172.21.15.16:0/3123492837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de406b6d0 msgr2=0x7f5de419b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de2ffd700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de406b6d0 0x7f5de419b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de2ffd700 1 -- 172.21.15.16:0/3123492837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5de41085e0 msgr2=0x7f5de419f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de2ffd700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5de41085e0 0x7f5de419f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de2ffd700 1 -- 172.21.15.16:0/3123492837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de41a0110 con 0x7f5de406ad10 2024-03-27T06:42:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5ddbfff700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de406b6d0 0x7f5de419b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.735+0000 7f5de2ffd700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de406ad10 0x7f5de419b100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40059d0 tx=0x7f5dd400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.736+0000 7f5de88a4700 1 -- 172.21.15.16:0/3123492837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4027070 con 0x7f5de406ad10 2024-03-27T06:42:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.736+0000 7f5de98a6700 1 -- 172.21.15.16:0/3123492837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de41a03a0 con 0x7f5de406ad10 2024-03-27T06:42:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.736+0000 7f5de88a4700 1 -- 172.21.15.16:0/3123492837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5dd400be40 con 0x7f5de406ad10 2024-03-27T06:42:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.736+0000 7f5de88a4700 1 -- 172.21.15.16:0/3123492837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40173f0 con 0x7f5de406ad10 2024-03-27T06:42:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.736+0000 7f5de98a6700 1 -- 172.21.15.16:0/3123492837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de41a7c80 con 0x7f5de406ad10 2024-03-27T06:42:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.737+0000 7f5de98a6700 1 -- 172.21.15.16:0/3123492837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de4103d60 con 0x7f5de406ad10 2024-03-27T06:42:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.739+0000 7f5de88a4700 1 -- 172.21.15.16:0/3123492837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f5dd402a070 con 0x7f5de406ad10 2024-03-27T06:42:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.739+0000 7f5de88a4700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5dd00a3730 0x7f5dd00a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.739+0000 7f5ddbfff700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5dd00a3730 0x7f5dd00a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.739+0000 7f5de88a4700 1 -- 172.21.15.16:0/3123492837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5dd40d0a70 con 0x7f5de406ad10 2024-03-27T06:42:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.740+0000 7f5ddbfff700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5dd00a3730 0x7f5dd00a5bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5de41035c0 tx=0x7f5dcc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.743+0000 7f5de88a4700 1 -- 172.21.15.16:0/3123492837 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5dd4013070 con 0x7f5de406ad10 2024-03-27T06:42:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.952+0000 7f5de98a6700 1 -- 172.21.15.16:0/3123492837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f5de419c460 con 0x7f5de406ad10 2024-03-27T06:42:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.952+0000 7f5de88a4700 1 -- 172.21.15.16:0/3123492837 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5dd409d580 con 0x7f5de406ad10 2024-03-27T06:42:42.947 INFO:teuthology.orchestra.run.smithi016.stdout:313532612640 2024-03-27T06:42:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 -- 172.21.15.16:0/3123492837 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5dd00a3730 msgr2=0x7f5dd00a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5dd00a3730 0x7f5dd00a5bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5de41035c0 tx=0x7f5dcc009260 comp rx=0 tx=0).stop 2024-03-27T06:42:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 -- 172.21.15.16:0/3123492837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de406ad10 msgr2=0x7f5de419b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de406ad10 0x7f5de419b100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40059d0 tx=0x7f5dd400f690 comp rx=0 tx=0).stop 2024-03-27T06:42:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 -- 172.21.15.16:0/3123492837 shutdown_connections 2024-03-27T06:42:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5de41085e0 0x7f5de419f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de406ad10 0x7f5de419b100 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5dd00a3730 0x7f5dd00a5bf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 --2- 172.21.15.16:0/3123492837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de406b6d0 0x7f5de419b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 -- 172.21.15.16:0/3123492837 >> 172.21.15.16:0/3123492837 conn(0x7f5de40784b0 msgr2=0x7f5de410d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 -- 172.21.15.16:0/3123492837 shutdown_connections 2024-03-27T06:42:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:42.955+0000 7f5de98a6700 1 -- 172.21.15.16:0/3123492837 wait complete. 2024-03-27T06:42:43.190 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738451 for osd.0 2024-03-27T06:42:43.190 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:43 smithi156 ceph-mon[36261]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:42:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:43 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3123492837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-27T06:42:43.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:43 smithi016 ceph-mon[31908]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:42:43.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:43 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3123492837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-27T06:42:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:43 smithi170 ceph-mon[36438]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:42:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:43 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3123492837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-27T06:42:43.857 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612633 got 313532612640 for osd.10 2024-03-27T06:42:43.858 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:44.309 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462966 for osd.6 2024-03-27T06:42:44.309 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.828+0000 7f788ea7a700 1 -- 172.21.15.16:0/3227715596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f788806ad90 msgr2=0x7f788806b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.828+0000 7f788ea7a700 1 --2- 172.21.15.16:0/3227715596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f788806ad90 0x7f788806b210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f78740099c0 tx=0x7f7874009cd0 comp rx=0 tx=0).stop 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.828+0000 7f788ea7a700 1 -- 172.21.15.16:0/3227715596 shutdown_connections 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.828+0000 7f788ea7a700 1 --2- 172.21.15.16:0/3227715596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f788806b750 0x7f788810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.828+0000 7f788ea7a700 1 --2- 172.21.15.16:0/3227715596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7888108670 0x7f7888108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.828+0000 7f788ea7a700 1 --2- 172.21.15.16:0/3227715596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f788806ad90 0x7f788806b210 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.828+0000 7f788ea7a700 1 -- 172.21.15.16:0/3227715596 >> 172.21.15.16:0/3227715596 conn(0x7f7888078550 msgr2=0x7f7888078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.828+0000 7f788ea7a700 1 -- 172.21.15.16:0/3227715596 shutdown_connections 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.828+0000 7f788ea7a700 1 -- 172.21.15.16:0/3227715596 wait complete. 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f788ea7a700 1 Processor -- start 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f788ea7a700 1 -- start start 2024-03-27T06:42:44.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f788ea7a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f788806ad90 0x7f78881015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f788ea7a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f788806b750 0x7f7888101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f788ea7a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7888108670 0x7f7888102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f788ea7a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7888112a30 con 0x7f788806b750 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f788ea7a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f78881128b0 con 0x7f7888108670 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f788ea7a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7888112bb0 con 0x7f788806ad90 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f7887fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f788806b750 0x7f7888101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f7887fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f788806b750 0x7f7888101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56084/0 (socket says 172.21.15.16:56084) 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f7887fff700 1 -- 172.21.15.16:0/1197984840 learned_addr learned my addr 172.21.15.16:0/1197984840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f788c816700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f788806ad90 0x7f78881015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f788d017700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7888108670 0x7f7888102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.829+0000 7f7887fff700 1 -- 172.21.15.16:0/1197984840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f788806ad90 msgr2=0x7f78881015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.830+0000 7f7887fff700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f788806ad90 0x7f78881015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.830+0000 7f7887fff700 1 -- 172.21.15.16:0/1197984840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7888108670 msgr2=0x7f7888102070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:44.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.830+0000 7f7887fff700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7888108670 0x7f7888102070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:44.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.830+0000 7f7887fff700 1 -- 172.21.15.16:0/1197984840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78881a7490 con 0x7f788806b750 2024-03-27T06:42:44.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.830+0000 7f7887fff700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f788806b750 0x7f7888101b30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7874005910 tx=0x7f787400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:44.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.830+0000 7f7885ffb700 1 -- 172.21.15.16:0/1197984840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874027070 con 0x7f788806b750 2024-03-27T06:42:44.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.830+0000 7f788ea7a700 1 -- 172.21.15.16:0/1197984840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78881a7720 con 0x7f788806b750 2024-03-27T06:42:44.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.830+0000 7f7885ffb700 1 -- 172.21.15.16:0/1197984840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f787400be40 con 0x7f788806b750 2024-03-27T06:42:44.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.830+0000 7f7885ffb700 1 -- 172.21.15.16:0/1197984840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78740173f0 con 0x7f788806b750 2024-03-27T06:42:44.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.830+0000 7f788ea7a700 1 -- 172.21.15.16:0/1197984840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78881a7bd0 con 0x7f788806b750 2024-03-27T06:42:44.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.831+0000 7f788ea7a700 1 -- 172.21.15.16:0/1197984840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7888069300 con 0x7f788806b750 2024-03-27T06:42:44.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.833+0000 7f7885ffb700 1 -- 172.21.15.16:0/1197984840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f7874022600 con 0x7f788806b750 2024-03-27T06:42:44.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.834+0000 7f7885ffb700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78780a3730 0x7f78780a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:44.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.834+0000 7f788c816700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78780a3730 0x7f78780a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:44.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.834+0000 7f7885ffb700 1 -- 172.21.15.16:0/1197984840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f78740d0be0 con 0x7f788806b750 2024-03-27T06:42:44.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.834+0000 7f788c816700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78780a3730 0x7f78780a5bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f787c007b60 tx=0x7f787c0073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:44.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:44.837+0000 7f7885ffb700 1 -- 172.21.15.16:0/1197984840 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7874013070 con 0x7f788806b750 2024-03-27T06:42:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.071+0000 7f788ea7a700 1 -- 172.21.15.16:0/1197984840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f788802dd30 con 0x7f788806b750 2024-03-27T06:42:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.072+0000 7f7885ffb700 1 -- 172.21.15.16:0/1197984840 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f787409d6f0 con 0x7f788806b750 2024-03-27T06:42:45.067 INFO:teuthology.orchestra.run.smithi016.stdout:347892351003 2024-03-27T06:42:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 -- 172.21.15.16:0/1197984840 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78780a3730 msgr2=0x7f78780a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78780a3730 0x7f78780a5bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f787c007b60 tx=0x7f787c0073d0 comp rx=0 tx=0).stop 2024-03-27T06:42:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 -- 172.21.15.16:0/1197984840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f788806b750 msgr2=0x7f7888101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f788806b750 0x7f7888101b30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7874005910 tx=0x7f787400f690 comp rx=0 tx=0).stop 2024-03-27T06:42:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 -- 172.21.15.16:0/1197984840 shutdown_connections 2024-03-27T06:42:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7888108670 0x7f7888102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f78780a3730 0x7f78780a5bf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f788806ad90 0x7f78881015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 --2- 172.21.15.16:0/1197984840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f788806b750 0x7f7888101b30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 -- 172.21.15.16:0/1197984840 >> 172.21.15.16:0/1197984840 conn(0x7f7888078550 msgr2=0x7f78881002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 -- 172.21.15.16:0/1197984840 shutdown_connections 2024-03-27T06:42:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:45.074+0000 7f788ea7a700 1 -- 172.21.15.16:0/1197984840 wait complete. 2024-03-27T06:42:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:45 smithi016 ceph-mon[31908]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:42:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:45 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1197984840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-27T06:42:45.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:45 smithi156 ceph-mon[36261]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:42:45.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:45 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1197984840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-27T06:42:45.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:45 smithi170 ceph-mon[36438]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:42:45.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:45 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1197984840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-27T06:42:45.765 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350998 got 347892351003 for osd.11 2024-03-27T06:42:45.766 DEBUG:teuthology.parallel:result is None 2024-03-27T06:42:45.766 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-27T06:42:45.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-27T06:42:46.022 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:46.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:46 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:46.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:46 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:46.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:46 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:42:46.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:46 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:42:46.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:46 smithi016 ceph-mon[31908]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:46.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:46 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:46.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:46 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:46.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:46 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:42:46.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:46 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:42:46.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:46 smithi156 ceph-mon[36261]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:46.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:46 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:46.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:46 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:46.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:46 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:42:46.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:46 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:42:46.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:46 smithi170 ceph-mon[36438]: from='mgr.14148 172.21.15.16:0/4131223498' entity='mgr.x' 2024-03-27T06:42:47.472 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:47 smithi016 ceph-mon[31908]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:47.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.708+0000 7f257e156700 1 -- 172.21.15.16:0/896842761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2578108640 msgr2=0x7f2578108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:47.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.708+0000 7f257e156700 1 --2- 172.21.15.16:0/896842761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2578108640 0x7f2578108a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2568009990 tx=0x7f2568009ca0 comp rx=0 tx=0).stop 2024-03-27T06:42:47.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.708+0000 7f257e156700 1 -- 172.21.15.16:0/896842761 shutdown_connections 2024-03-27T06:42:47.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.708+0000 7f257e156700 1 --2- 172.21.15.16:0/896842761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2578103000 0x7f257810f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:47.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.708+0000 7f257e156700 1 --2- 172.21.15.16:0/896842761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2578108640 0x7f2578108a20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:47.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.708+0000 7f257e156700 1 --2- 172.21.15.16:0/896842761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2578102640 0x7f2578102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:47.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.708+0000 7f257e156700 1 -- 172.21.15.16:0/896842761 >> 172.21.15.16:0/896842761 conn(0x7f25780fe110 msgr2=0x7f2578100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:47.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.708+0000 7f257e156700 1 -- 172.21.15.16:0/896842761 shutdown_connections 2024-03-27T06:42:47.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.708+0000 7f257e156700 1 -- 172.21.15.16:0/896842761 wait complete. 2024-03-27T06:42:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f257e156700 1 Processor -- start 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f257e156700 1 -- start start 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f257e156700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2578102640 0x7f2578198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f257e156700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2578103000 0x7f2578199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f257e156700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2578108640 0x7f257819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f257e156700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2578112a90 con 0x7f2578108640 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f257e156700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f2578112910 con 0x7f2578103000 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f257e156700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2578112c10 con 0x7f2578102640 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f2577fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2578108640 0x7f257819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f2577fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2578108640 0x7f257819d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43034/0 (socket says 172.21.15.16:43034) 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f2577fff700 1 -- 172.21.15.16:0/4021114899 learned_addr learned my addr 172.21.15.16:0/4021114899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f2576ffd700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2578103000 0x7f2578199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.709+0000 7f25777fe700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2578102640 0x7f2578198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f2577fff700 1 -- 172.21.15.16:0/4021114899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2578102640 msgr2=0x7f2578198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f2577fff700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2578102640 0x7f2578198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f2577fff700 1 -- 172.21.15.16:0/4021114899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2578103000 msgr2=0x7f2578199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f2577fff700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2578103000 0x7f2578199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f2577fff700 1 -- 172.21.15.16:0/4021114899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257819ded0 con 0x7f2578108640 2024-03-27T06:42:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f2576ffd700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2578103000 0x7f2578199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f2577fff700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2578108640 0x7f257819d790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2570007ee0 tx=0x7f257000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f2574ff9700 1 -- 172.21.15.16:0/4021114899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257000dd20 con 0x7f2578108640 2024-03-27T06:42:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f257e156700 1 -- 172.21.15.16:0/4021114899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257819e1c0 con 0x7f2578108640 2024-03-27T06:42:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f2574ff9700 1 -- 172.21.15.16:0/4021114899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2570004020 con 0x7f2578108640 2024-03-27T06:42:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f2574ff9700 1 -- 172.21.15.16:0/4021114899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570015660 con 0x7f2578108640 2024-03-27T06:42:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.710+0000 7f257e156700 1 -- 172.21.15.16:0/4021114899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25781a5a40 con 0x7f2578108640 2024-03-27T06:42:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.711+0000 7f257e156700 1 -- 172.21.15.16:0/4021114899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2578069300 con 0x7f2578108640 2024-03-27T06:42:47.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.713+0000 7f2574ff9700 1 -- 172.21.15.16:0/4021114899 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f25700041d0 con 0x7f2578108640 2024-03-27T06:42:47.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.714+0000 7f2574ff9700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f25600a3730 0x7f25600a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:47.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.714+0000 7f25777fe700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f25600a3730 0x7f25600a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:47.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.714+0000 7f2574ff9700 1 -- 172.21.15.16:0/4021114899 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f25700cffe0 con 0x7f2578108640 2024-03-27T06:42:47.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.714+0000 7f25777fe700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f25600a3730 0x7f25600a5bf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25680098e0 tx=0x7f2568005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:47.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.717+0000 7f2574ff9700 1 -- 172.21.15.16:0/4021114899 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f2570010070 con 0x7f2578108640 2024-03-27T06:42:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:47 smithi156 ceph-mon[36261]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:47 smithi170 ceph-mon[36438]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.917+0000 7f257e156700 1 -- 172.21.15.16:0/4021114899 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2578062a80 con 0x7f25600a3730 2024-03-27T06:42:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.924+0000 7f2574ff9700 1 -- 172.21.15.16:0/4021114899 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258934 (secure 0 0 0) 0x7f2578062a80 con 0x7f25600a3730 2024-03-27T06:42:47.924 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:42:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.927+0000 7f257e156700 1 -- 172.21.15.16:0/4021114899 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f25600a3730 msgr2=0x7f25600a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.927+0000 7f257e156700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f25600a3730 0x7f25600a5bf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25680098e0 tx=0x7f2568005960 comp rx=0 tx=0).stop 2024-03-27T06:42:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.928+0000 7f257e156700 1 -- 172.21.15.16:0/4021114899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2578108640 msgr2=0x7f257819d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.928+0000 7f257e156700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2578108640 0x7f257819d790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2570007ee0 tx=0x7f257000d3b0 comp rx=0 tx=0).stop 2024-03-27T06:42:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.928+0000 7f257e156700 1 -- 172.21.15.16:0/4021114899 shutdown_connections 2024-03-27T06:42:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.928+0000 7f257e156700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2578108640 0x7f257819d790 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.928+0000 7f257e156700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f25600a3730 0x7f25600a5bf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.928+0000 7f257e156700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2578102640 0x7f2578198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.928+0000 7f257e156700 1 --2- 172.21.15.16:0/4021114899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2578103000 0x7f2578199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.928+0000 7f257e156700 1 -- 172.21.15.16:0/4021114899 >> 172.21.15.16:0/4021114899 conn(0x7f25780fe110 msgr2=0x7f25780fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.928+0000 7f257e156700 1 -- 172.21.15.16:0/4021114899 shutdown_connections 2024-03-27T06:42:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:47.928+0000 7f257e156700 1 -- 172.21.15.16:0/4021114899 wait complete. 2024-03-27T06:42:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-03-27T06:42:48.634 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-03-27T06:42:47.580566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657829,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":869,"num_read_kb":553,"num_write":421,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1180960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365180,"kb_used_data":11808,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576644,"statfs":{"total":1151940427776,"available":1151566483456,"internally_reserved":0,"allocated":12091392,"data_stored":7477383,"data_compressed":106419,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003166"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651071+0000","last_change":"2024-03-27T06:41:29.408953+0000","last_active":"2024-03-27T06:42:20.651071+0000","last_peered":"2024-03-27T06:42:20.651071+0000","last_clean":"2024-03-27T06:42:20.651071+0000","last_became_active":"2024-03-27T06:41:29.408143+0000","last_became_peered":"2024-03-27T06:41:29.408143+0000","last_unstale":"2024-03-27T06:42:20.651071+0000","last_undegraded":"2024-03-27T06:42:20.651071+0000","last_fullsized":"2024-03-27T06:42:20.651071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:22.486951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653219+0000","last_change":"2024-03-27T06:41:29.405080+0000","last_active":"2024-03-27T06:42:20.653219+0000","last_peered":"2024-03-27T06:42:20.653219+0000","last_clean":"2024-03-27T06:42:20.653219+0000","last_became_active":"2024-03-27T06:41:29.404743+0000","last_became_peered":"2024-03-27T06:41:29.404743+0000","last_unstale":"2024-03-27T06:42:20.653219+0000","last_undegraded":"2024-03-27T06:42:20.653219+0000","last_fullsized":"2024-03-27T06:42:20.653219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:24:30.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653238+0000","last_change":"2024-03-27T06:41:29.409346+0000","last_active":"2024-03-27T06:42:20.653238+0000","last_peered":"2024-03-27T06:42:20.653238+0000","last_clean":"2024-03-27T06:42:20.653238+0000","last_became_active":"2024-03-27T06:41:29.408837+0000","last_became_peered":"2024-03-27T06:41:29.408837+0000","last_unstale":"2024-03-27T06:42:20.653238+0000","last_undegraded":"2024-03-27T06:42:20.653238+0000","last_fullsized":"2024-03-27T06:42:20.653238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:11:12.274869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.003961+0000","last_change":"2024-03-27T06:41:29.412115+0000","last_active":"2024-03-27T06:42:21.003961+0000","last_peered":"2024-03-27T06:42:21.003961+0000","last_clean":"2024-03-27T06:42:21.003961+0000","last_became_active":"2024-03-27T06:41:29.411702+0000","last_became_peered":"2024-03-27T06:41:29.411702+0000","last_unstale":"2024-03-27T06:42:21.003961+0000","last_undegraded":"2024-03-27T06:42:21.003961+0000","last_fullsized":"2024-03-27T06:42:21.003961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:04:42.347229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653364+0000","last_change":"2024-03-27T06:41:29.409888+0000","last_active":"2024-03-27T06:42:20.653364+0000","last_peered":"2024-03-27T06:42:20.653364+0000","last_clean":"2024-03-27T06:42:20.653364+0000","last_became_active":"2024-03-27T06:41:29.409558+0000","last_became_peered":"2024-03-27T06:41:29.409558+0000","last_unstale":"2024-03-27T06:42:20.653364+0000","last_undegraded":"2024-03-27T06:42:20.653364+0000","last_fullsized":"2024-03-27T06:42:20.653364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:38:51.452793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650936+0000","last_change":"2024-03-27T06:41:29.404088+0000","last_active":"2024-03-27T06:42:20.650936+0000","last_peered":"2024-03-27T06:42:20.650936+0000","last_clean":"2024-03-27T06:42:20.650936+0000","last_became_active":"2024-03-27T06:41:29.403698+0000","last_became_peered":"2024-03-27T06:41:29.403698+0000","last_unstale":"2024-03-27T06:42:20.650936+0000","last_undegraded":"2024-03-27T06:42:20.650936+0000","last_fullsized":"2024-03-27T06:42:20.650936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:14:31.296523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653732+0000","last_change":"2024-03-27T06:41:29.413157+0000","last_active":"2024-03-27T06:42:20.653732+0000","last_peered":"2024-03-27T06:42:20.653732+0000","last_clean":"2024-03-27T06:42:20.653732+0000","last_became_active":"2024-03-27T06:41:29.412669+0000","last_became_peered":"2024-03-27T06:41:29.412669+0000","last_unstale":"2024-03-27T06:42:20.653732+0000","last_undegraded":"2024-03-27T06:42:20.653732+0000","last_fullsized":"2024-03-27T06:42:20.653732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:28:20.823770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658144+0000","last_change":"2024-03-27T06:41:29.417207+0000","last_active":"2024-03-27T06:42:20.658144+0000","last_peered":"2024-03-27T06:42:20.658144+0000","last_clean":"2024-03-27T06:42:20.658144+0000","last_became_active":"2024-03-27T06:41:29.415556+0000","last_became_peered":"2024-03-27T06:41:29.415556+0000","last_unstale":"2024-03-27T06:42:20.658144+0000","last_undegraded":"2024-03-27T06:42:20.658144+0000","last_fullsized":"2024-03-27T06:42:20.658144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:26:04.040296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.656886+0000","last_change":"2024-03-27T06:41:29.400032+0000","last_active":"2024-03-27T06:42:20.656886+0000","last_peered":"2024-03-27T06:42:20.656886+0000","last_clean":"2024-03-27T06:42:20.656886+0000","last_became_active":"2024-03-27T06:41:29.399274+0000","last_became_peered":"2024-03-27T06:41:29.399274+0000","last_unstale":"2024-03-27T06:42:20.656886+0000","last_undegraded":"2024-03-27T06:42:20.656886+0000","last_fullsized":"2024-03-27T06:42:20.656886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:55.306309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651336+0000","last_change":"2024-03-27T06:41:29.412471+0000","last_active":"2024-03-27T06:42:20.651336+0000","last_peered":"2024-03-27T06:42:20.651336+0000","last_clean":"2024-03-27T06:42:20.651336+0000","last_became_active":"2024-03-27T06:41:29.412134+0000","last_became_peered":"2024-03-27T06:41:29.412134+0000","last_unstale":"2024-03-27T06:42:20.651336+0000","last_undegraded":"2024-03-27T06:42:20.651336+0000","last_fullsized":"2024-03-27T06:42:20.651336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:35:57.146944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654091+0000","last_change":"2024-03-27T06:41:29.407915+0000","last_active":"2024-03-27T06:42:20.654091+0000","last_peered":"2024-03-27T06:42:20.654091+0000","last_clean":"2024-03-27T06:42:20.654091+0000","last_became_active":"2024-03-27T06:41:29.407563+0000","last_became_peered":"2024-03-27T06:41:29.407563+0000","last_unstale":"2024-03-27T06:42:20.654091+0000","last_undegraded":"2024-03-27T06:42:20.654091+0000","last_fullsized":"2024-03-27T06:42:20.654091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:09:05.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.003952+0000","last_change":"2024-03-27T06:41:29.413374+0000","last_active":"2024-03-27T06:42:21.003952+0000","last_peered":"2024-03-27T06:42:21.003952+0000","last_clean":"2024-03-27T06:42:21.003952+0000","last_became_active":"2024-03-27T06:41:29.413010+0000","last_became_peered":"2024-03-27T06:41:29.413010+0000","last_unstale":"2024-03-27T06:42:21.003952+0000","last_undegraded":"2024-03-27T06:42:21.003952+0000","last_fullsized":"2024-03-27T06:42:21.003952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:48:52.329611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654602+0000","last_change":"2024-03-27T06:41:29.408640+0000","last_active":"2024-03-27T06:42:20.654602+0000","last_peered":"2024-03-27T06:42:20.654602+0000","last_clean":"2024-03-27T06:42:20.654602+0000","last_became_active":"2024-03-27T06:41:29.408268+0000","last_became_peered":"2024-03-27T06:41:29.408268+0000","last_unstale":"2024-03-27T06:42:20.654602+0000","last_undegraded":"2024-03-27T06:42:20.654602+0000","last_fullsized":"2024-03-27T06:42:20.654602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:51:59.803596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653170+0000","last_change":"2024-03-27T06:41:29.420436+0000","last_active":"2024-03-27T06:42:20.653170+0000","last_peered":"2024-03-27T06:42:20.653170+0000","last_clean":"2024-03-27T06:42:20.653170+0000","last_became_active":"2024-03-27T06:41:29.419426+0000","last_became_peered":"2024-03-27T06:41:29.419426+0000","last_unstale":"2024-03-27T06:42:20.653170+0000","last_undegraded":"2024-03-27T06:42:20.653170+0000","last_fullsized":"2024-03-27T06:42:20.653170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:38:15.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653035+0000","last_change":"2024-03-27T06:41:29.415673+0000","last_active":"2024-03-27T06:42:20.653035+0000","last_peered":"2024-03-27T06:42:20.653035+0000","last_clean":"2024-03-27T06:42:20.653035+0000","last_became_active":"2024-03-27T06:41:29.414741+0000","last_became_peered":"2024-03-27T06:41:29.414741+0000","last_unstale":"2024-03-27T06:42:20.653035+0000","last_undegraded":"2024-03-27T06:42:20.653035+0000","last_fullsized":"2024-03-27T06:42:20.653035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:38:43.183517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.655655+0000","last_change":"2024-03-27T06:41:29.408609+0000","last_active":"2024-03-27T06:42:20.655655+0000","last_peered":"2024-03-27T06:42:20.655655+0000","last_clean":"2024-03-27T06:42:20.655655+0000","last_became_active":"2024-03-27T06:41:29.408252+0000","last_became_peered":"2024-03-27T06:41:29.408252+0000","last_unstale":"2024-03-27T06:42:20.655655+0000","last_undegraded":"2024-03-27T06:42:20.655655+0000","last_fullsized":"2024-03-27T06:42:20.655655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:14:13.609113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651294+0000","last_change":"2024-03-27T06:41:29.409221+0000","last_active":"2024-03-27T06:42:20.651294+0000","last_peered":"2024-03-27T06:42:20.651294+0000","last_clean":"2024-03-27T06:42:20.651294+0000","last_became_active":"2024-03-27T06:41:29.408598+0000","last_became_peered":"2024-03-27T06:41:29.408598+0000","last_unstale":"2024-03-27T06:42:20.651294+0000","last_undegraded":"2024-03-27T06:42:20.651294+0000","last_fullsized":"2024-03-27T06:42:20.651294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:42:16.251149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650671+0000","last_change":"2024-03-27T06:41:29.406143+0000","last_active":"2024-03-27T06:42:20.650671+0000","last_peered":"2024-03-27T06:42:20.650671+0000","last_clean":"2024-03-27T06:42:20.650671+0000","last_became_active":"2024-03-27T06:41:29.405754+0000","last_became_peered":"2024-03-27T06:41:29.405754+0000","last_unstale":"2024-03-27T06:42:20.650671+0000","last_undegraded":"2024-03-27T06:42:20.650671+0000","last_fullsized":"2024-03-27T06:42:20.650671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:42:47.787099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653263+0000","last_change":"2024-03-27T06:41:29.402471+0000","last_active":"2024-03-27T06:42:20.653263+0000","last_peered":"2024-03-27T06:42:20.653263+0000","last_clean":"2024-03-27T06:42:20.653263+0000","last_became_active":"2024-03-27T06:41:29.402113+0000","last_became_peered":"2024-03-27T06:41:29.402113+0000","last_unstale":"2024-03-27T06:42:20.653263+0000","last_undegraded":"2024-03-27T06:42:20.653263+0000","last_fullsized":"2024-03-27T06:42:20.653263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:34:16.408058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:45.552237+0000","last_change":"2024-03-27T06:41:28.397815+0000","last_active":"2024-03-27T06:42:45.552237+0000","last_peered":"2024-03-27T06:42:45.552237+0000","last_clean":"2024-03-27T06:42:45.552237+0000","last_became_active":"2024-03-27T06:41:28.397503+0000","last_became_peered":"2024-03-27T06:41:28.397503+0000","last_unstale":"2024-03-27T06:42:45.552237+0000","last_undegraded":"2024-03-27T06:42:45.552237+0000","last_fullsized":"2024-03-27T06:42:45.552237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:15:53.751809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657518+0000","last_change":"2024-03-27T06:41:29.419528+0000","last_active":"2024-03-27T06:42:20.657518+0000","last_peered":"2024-03-27T06:42:20.657518+0000","last_clean":"2024-03-27T06:42:20.657518+0000","last_became_active":"2024-03-27T06:41:29.417091+0000","last_became_peered":"2024-03-27T06:41:29.417091+0000","last_unstale":"2024-03-27T06:42:20.657518+0000","last_undegraded":"2024-03-27T06:42:20.657518+0000","last_fullsized":"2024-03-27T06:42:20.657518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:13:14.629211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657772+0000","last_change":"2024-03-27T06:41:28.400468+0000","last_active":"2024-03-27T06:42:20.657772+0000","last_peered":"2024-03-27T06:42:20.657772+0000","last_clean":"2024-03-27T06:42:20.657772+0000","last_became_active":"2024-03-27T06:41:28.400190+0000","last_became_peered":"2024-03-27T06:41:28.400190+0000","last_unstale":"2024-03-27T06:42:20.657772+0000","last_undegraded":"2024-03-27T06:42:20.657772+0000","last_fullsized":"2024-03-27T06:42:20.657772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:24:17.059874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653289+0000","last_change":"2024-03-27T06:41:29.409364+0000","last_active":"2024-03-27T06:42:20.653289+0000","last_peered":"2024-03-27T06:42:20.653289+0000","last_clean":"2024-03-27T06:42:20.653289+0000","last_became_active":"2024-03-27T06:41:29.408946+0000","last_became_peered":"2024-03-27T06:41:29.408946+0000","last_unstale":"2024-03-27T06:42:20.653289+0000","last_undegraded":"2024-03-27T06:42:20.653289+0000","last_fullsized":"2024-03-27T06:42:20.653289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:46:17.132071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654300+0000","last_change":"2024-03-27T06:41:28.401327+0000","last_active":"2024-03-27T06:42:20.654300+0000","last_peered":"2024-03-27T06:42:20.654300+0000","last_clean":"2024-03-27T06:42:20.654300+0000","last_became_active":"2024-03-27T06:41:28.400524+0000","last_became_peered":"2024-03-27T06:41:28.400524+0000","last_unstale":"2024-03-27T06:42:20.654300+0000","last_undegraded":"2024-03-27T06:42:20.654300+0000","last_fullsized":"2024-03-27T06:42:20.654300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:11:52.608690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.656060+0000","last_change":"2024-03-27T06:41:29.403042+0000","last_active":"2024-03-27T06:42:20.656060+0000","last_peered":"2024-03-27T06:42:20.656060+0000","last_clean":"2024-03-27T06:42:20.656060+0000","last_became_active":"2024-03-27T06:41:29.402516+0000","last_became_peered":"2024-03-27T06:41:29.402516+0000","last_unstale":"2024-03-27T06:42:20.656060+0000","last_undegraded":"2024-03-27T06:42:20.656060+0000","last_fullsized":"2024-03-27T06:42:20.656060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:20:55.777498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653689+0000","last_change":"2024-03-27T06:41:28.395155+0000","last_active":"2024-03-27T06:42:20.653689+0000","last_peered":"2024-03-27T06:42:20.653689+0000","last_clean":"2024-03-27T06:42:20.653689+0000","last_became_active":"2024-03-27T06:41:28.394760+0000","last_became_peered":"2024-03-27T06:41:28.394760+0000","last_unstale":"2024-03-27T06:42:20.653689+0000","last_undegraded":"2024-03-27T06:42:20.653689+0000","last_fullsized":"2024-03-27T06:42:20.653689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:15:28.182581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.721029+0000","last_change":"2024-03-27T06:41:29.409747+0000","last_active":"2024-03-27T06:42:20.721029+0000","last_peered":"2024-03-27T06:42:20.721029+0000","last_clean":"2024-03-27T06:42:20.721029+0000","last_became_active":"2024-03-27T06:41:29.409425+0000","last_became_peered":"2024-03-27T06:41:29.409425+0000","last_unstale":"2024-03-27T06:42:20.721029+0000","last_undegraded":"2024-03-27T06:42:20.721029+0000","last_fullsized":"2024-03-27T06:42:20.721029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:36.181279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:45.547898+0000","last_change":"2024-03-27T06:41:28.394726+0000","last_active":"2024-03-27T06:42:45.547898+0000","last_peered":"2024-03-27T06:42:45.547898+0000","last_clean":"2024-03-27T06:42:45.547898+0000","last_became_active":"2024-03-27T06:41:28.394378+0000","last_became_peered":"2024-03-27T06:41:28.394378+0000","last_unstale":"2024-03-27T06:42:45.547898+0000","last_undegraded":"2024-03-27T06:42:45.547898+0000","last_fullsized":"2024-03-27T06:42:45.547898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:30:22.194211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.652987+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:42:20.652987+0000","last_peered":"2024-03-27T06:42:20.652987+0000","last_clean":"2024-03-27T06:42:20.652987+0000","last_became_active":"2024-03-27T06:41:29.408826+0000","last_became_peered":"2024-03-27T06:41:29.408826+0000","last_unstale":"2024-03-27T06:42:20.652987+0000","last_undegraded":"2024-03-27T06:42:20.652987+0000","last_fullsized":"2024-03-27T06:42:20.652987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:50:53.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650936+0000","last_change":"2024-03-27T06:41:28.393655+0000","last_active":"2024-03-27T06:42:20.650936+0000","last_peered":"2024-03-27T06:42:20.650936+0000","last_clean":"2024-03-27T06:42:20.650936+0000","last_became_active":"2024-03-27T06:41:28.393245+0000","last_became_peered":"2024-03-27T06:41:28.393245+0000","last_unstale":"2024-03-27T06:42:20.650936+0000","last_undegraded":"2024-03-27T06:42:20.650936+0000","last_fullsized":"2024-03-27T06:42:20.650936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:58:21.169444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.721047+0000","last_change":"2024-03-27T06:41:29.417730+0000","last_active":"2024-03-27T06:42:20.721047+0000","last_peered":"2024-03-27T06:42:20.721047+0000","last_clean":"2024-03-27T06:42:20.721047+0000","last_became_active":"2024-03-27T06:41:29.417319+0000","last_became_peered":"2024-03-27T06:41:29.417319+0000","last_unstale":"2024-03-27T06:42:20.721047+0000","last_undegraded":"2024-03-27T06:42:20.721047+0000","last_fullsized":"2024-03-27T06:42:20.721047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:25.224220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654549+0000","last_change":"2024-03-27T06:41:28.402259+0000","last_active":"2024-03-27T06:42:20.654549+0000","last_peered":"2024-03-27T06:42:20.654549+0000","last_clean":"2024-03-27T06:42:20.654549+0000","last_became_active":"2024-03-27T06:41:28.401922+0000","last_became_peered":"2024-03-27T06:41:28.401922+0000","last_unstale":"2024-03-27T06:42:20.654549+0000","last_undegraded":"2024-03-27T06:42:20.654549+0000","last_fullsized":"2024-03-27T06:42:20.654549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:57:33.245183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658242+0000","last_change":"2024-03-27T06:41:29.409034+0000","last_active":"2024-03-27T06:42:20.658242+0000","last_peered":"2024-03-27T06:42:20.658242+0000","last_clean":"2024-03-27T06:42:20.658242+0000","last_became_active":"2024-03-27T06:41:29.408666+0000","last_became_peered":"2024-03-27T06:41:29.408666+0000","last_unstale":"2024-03-27T06:42:20.658242+0000","last_undegraded":"2024-03-27T06:42:20.658242+0000","last_fullsized":"2024-03-27T06:42:20.658242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:09:55.532059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.720843+0000","last_change":"2024-03-27T06:41:28.393802+0000","last_active":"2024-03-27T06:42:20.720843+0000","last_peered":"2024-03-27T06:42:20.720843+0000","last_clean":"2024-03-27T06:42:20.720843+0000","last_became_active":"2024-03-27T06:41:28.393503+0000","last_became_peered":"2024-03-27T06:41:28.393503+0000","last_unstale":"2024-03-27T06:42:20.720843+0000","last_undegraded":"2024-03-27T06:42:20.720843+0000","last_fullsized":"2024-03-27T06:42:20.720843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:17.765709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650454+0000","last_change":"2024-03-27T06:41:29.416557+0000","last_active":"2024-03-27T06:42:20.650454+0000","last_peered":"2024-03-27T06:42:20.650454+0000","last_clean":"2024-03-27T06:42:20.650454+0000","last_became_active":"2024-03-27T06:41:29.416223+0000","last_became_peered":"2024-03-27T06:41:29.416223+0000","last_unstale":"2024-03-27T06:42:20.650454+0000","last_undegraded":"2024-03-27T06:42:20.650454+0000","last_fullsized":"2024-03-27T06:42:20.650454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:54:20.929325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":159,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:42.544335+0000","last_change":"2024-03-27T06:41:28.396014+0000","last_active":"2024-03-27T06:42:42.544335+0000","last_peered":"2024-03-27T06:42:42.544335+0000","last_clean":"2024-03-27T06:42:42.544335+0000","last_became_active":"2024-03-27T06:41:28.395699+0000","last_became_peered":"2024-03-27T06:41:28.395699+0000","last_unstale":"2024-03-27T06:42:42.544335+0000","last_undegraded":"2024-03-27T06:42:42.544335+0000","last_fullsized":"2024-03-27T06:42:42.544335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:00:59.679843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.656463+0000","last_change":"2024-03-27T06:41:29.401668+0000","last_active":"2024-03-27T06:42:20.656463+0000","last_peered":"2024-03-27T06:42:20.656463+0000","last_clean":"2024-03-27T06:42:20.656463+0000","last_became_active":"2024-03-27T06:41:29.401328+0000","last_became_peered":"2024-03-27T06:41:29.401328+0000","last_unstale":"2024-03-27T06:42:20.656463+0000","last_undegraded":"2024-03-27T06:42:20.656463+0000","last_fullsized":"2024-03-27T06:42:20.656463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:32:54.273194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.011596+0000","last_change":"2024-03-27T06:42:21.011596+0000","last_active":"2024-03-27T06:42:21.011596+0000","last_peered":"2024-03-27T06:42:21.011596+0000","last_clean":"2024-03-27T06:42:21.011596+0000","last_became_active":"2024-03-27T06:42:21.005035+0000","last_became_peered":"2024-03-27T06:42:21.005035+0000","last_unstale":"2024-03-27T06:42:21.011596+0000","last_undegraded":"2024-03-27T06:42:21.011596+0000","last_fullsized":"2024-03-27T06:42:21.011596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:51:35.652266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653185+0000","last_change":"2024-03-27T06:41:29.413919+0000","last_active":"2024-03-27T06:42:20.653185+0000","last_peered":"2024-03-27T06:42:20.653185+0000","last_clean":"2024-03-27T06:42:20.653185+0000","last_became_active":"2024-03-27T06:41:29.413566+0000","last_became_peered":"2024-03-27T06:41:29.413566+0000","last_unstale":"2024-03-27T06:42:20.653185+0000","last_undegraded":"2024-03-27T06:42:20.653185+0000","last_fullsized":"2024-03-27T06:42:20.653185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:45:03.705372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.721086+0000","last_change":"2024-03-27T06:41:28.391661+0000","last_active":"2024-03-27T06:42:20.721086+0000","last_peered":"2024-03-27T06:42:20.721086+0000","last_clean":"2024-03-27T06:42:20.721086+0000","last_became_active":"2024-03-27T06:41:28.391230+0000","last_became_peered":"2024-03-27T06:41:28.391230+0000","last_unstale":"2024-03-27T06:42:20.721086+0000","last_undegraded":"2024-03-27T06:42:20.721086+0000","last_fullsized":"2024-03-27T06:42:20.721086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:41:22.789267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.656132+0000","last_change":"2024-03-27T06:41:29.405727+0000","last_active":"2024-03-27T06:42:20.656132+0000","last_peered":"2024-03-27T06:42:20.656132+0000","last_clean":"2024-03-27T06:42:20.656132+0000","last_became_active":"2024-03-27T06:41:29.405413+0000","last_became_peered":"2024-03-27T06:41:29.405413+0000","last_unstale":"2024-03-27T06:42:20.656132+0000","last_undegraded":"2024-03-27T06:42:20.656132+0000","last_fullsized":"2024-03-27T06:42:20.656132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:35:21.575729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'31","reported_seq":192,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:42.544323+0000","last_change":"2024-03-27T06:41:28.391155+0000","last_active":"2024-03-27T06:42:42.544323+0000","last_peered":"2024-03-27T06:42:42.544323+0000","last_clean":"2024-03-27T06:42:42.544323+0000","last_became_active":"2024-03-27T06:41:28.390828+0000","last_became_peered":"2024-03-27T06:41:28.390828+0000","last_unstale":"2024-03-27T06:42:42.544323+0000","last_undegraded":"2024-03-27T06:42:42.544323+0000","last_fullsized":"2024-03-27T06:42:42.544323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:40.192560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":142,"num_read_kb":129,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654043+0000","last_change":"2024-03-27T06:41:29.413283+0000","last_active":"2024-03-27T06:42:20.654043+0000","last_peered":"2024-03-27T06:42:20.654043+0000","last_clean":"2024-03-27T06:42:20.654043+0000","last_became_active":"2024-03-27T06:41:29.412247+0000","last_became_peered":"2024-03-27T06:41:29.412247+0000","last_unstale":"2024-03-27T06:42:20.654043+0000","last_undegraded":"2024-03-27T06:42:20.654043+0000","last_fullsized":"2024-03-27T06:42:20.654043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:50:15.336003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653342+0000","last_change":"2024-03-27T06:41:28.395089+0000","last_active":"2024-03-27T06:42:20.653342+0000","last_peered":"2024-03-27T06:42:20.653342+0000","last_clean":"2024-03-27T06:42:20.653342+0000","last_became_active":"2024-03-27T06:41:28.394145+0000","last_became_peered":"2024-03-27T06:41:28.394145+0000","last_unstale":"2024-03-27T06:42:20.653342+0000","last_undegraded":"2024-03-27T06:42:20.653342+0000","last_fullsized":"2024-03-27T06:42:20.653342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:40.858404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654325+0000","last_change":"2024-03-27T06:41:29.416955+0000","last_active":"2024-03-27T06:42:20.654325+0000","last_peered":"2024-03-27T06:42:20.654325+0000","last_clean":"2024-03-27T06:42:20.654325+0000","last_became_active":"2024-03-27T06:41:29.416616+0000","last_became_peered":"2024-03-27T06:41:29.416616+0000","last_unstale":"2024-03-27T06:42:20.654325+0000","last_undegraded":"2024-03-27T06:42:20.654325+0000","last_fullsized":"2024-03-27T06:42:20.654325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:38:18.791602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.720820+0000","last_change":"2024-03-27T06:41:28.393876+0000","last_active":"2024-03-27T06:42:20.720820+0000","last_peered":"2024-03-27T06:42:20.720820+0000","last_clean":"2024-03-27T06:42:20.720820+0000","last_became_active":"2024-03-27T06:41:28.393557+0000","last_became_peered":"2024-03-27T06:41:28.393557+0000","last_unstale":"2024-03-27T06:42:20.720820+0000","last_undegraded":"2024-03-27T06:42:20.720820+0000","last_fullsized":"2024-03-27T06:42:20.720820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:30.831039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.660026+0000","last_change":"2024-03-27T06:42:20.660026+0000","last_active":"2024-03-27T06:42:20.660026+0000","last_peered":"2024-03-27T06:42:20.660026+0000","last_clean":"2024-03-27T06:42:20.660026+0000","last_became_active":"2024-03-27T06:42:20.659628+0000","last_became_peered":"2024-03-27T06:42:20.659628+0000","last_unstale":"2024-03-27T06:42:20.660026+0000","last_undegraded":"2024-03-27T06:42:20.660026+0000","last_fullsized":"2024-03-27T06:42:20.660026+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:18:59.396472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:45.552846+0000","last_change":"2024-03-27T06:41:28.393849+0000","last_active":"2024-03-27T06:42:45.552846+0000","last_peered":"2024-03-27T06:42:45.552846+0000","last_clean":"2024-03-27T06:42:45.552846+0000","last_became_active":"2024-03-27T06:41:28.393529+0000","last_became_peered":"2024-03-27T06:41:28.393529+0000","last_unstale":"2024-03-27T06:42:45.552846+0000","last_undegraded":"2024-03-27T06:42:45.552846+0000","last_fullsized":"2024-03-27T06:42:45.552846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:24:23.251723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653685+0000","last_change":"2024-03-27T06:41:29.411273+0000","last_active":"2024-03-27T06:42:20.653685+0000","last_peered":"2024-03-27T06:42:20.653685+0000","last_clean":"2024-03-27T06:42:20.653685+0000","last_became_active":"2024-03-27T06:41:29.410936+0000","last_became_peered":"2024-03-27T06:41:29.410936+0000","last_unstale":"2024-03-27T06:42:20.653685+0000","last_undegraded":"2024-03-27T06:42:20.653685+0000","last_fullsized":"2024-03-27T06:42:20.653685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:41.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.674113+0000","last_change":"2024-03-27T06:42:20.673735+0000","last_active":"2024-03-27T06:42:20.674113+0000","last_peered":"2024-03-27T06:42:20.674113+0000","last_clean":"2024-03-27T06:42:20.674113+0000","last_became_active":"2024-03-27T06:42:20.655144+0000","last_became_peered":"2024-03-27T06:42:20.655144+0000","last_unstale":"2024-03-27T06:42:20.674113+0000","last_undegraded":"2024-03-27T06:42:20.674113+0000","last_fullsized":"2024-03-27T06:42:20.674113+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:05:39.059633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.720745+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:42:20.720745+0000","last_peered":"2024-03-27T06:42:20.720745+0000","last_clean":"2024-03-27T06:42:20.720745+0000","last_became_active":"2024-03-27T06:41:29.408815+0000","last_became_peered":"2024-03-27T06:41:29.408815+0000","last_unstale":"2024-03-27T06:42:20.720745+0000","last_undegraded":"2024-03-27T06:42:20.720745+0000","last_fullsized":"2024-03-27T06:42:20.720745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:45.547747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654026+0000","last_change":"2024-03-27T06:41:28.398249+0000","last_active":"2024-03-27T06:42:20.654026+0000","last_peered":"2024-03-27T06:42:20.654026+0000","last_clean":"2024-03-27T06:42:20.654026+0000","last_became_active":"2024-03-27T06:41:28.397176+0000","last_became_peered":"2024-03-27T06:41:28.397176+0000","last_unstale":"2024-03-27T06:42:20.654026+0000","last_undegraded":"2024-03-27T06:42:20.654026+0000","last_fullsized":"2024-03-27T06:42:20.654026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:32.614558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651362+0000","last_change":"2024-03-27T06:41:29.412928+0000","last_active":"2024-03-27T06:42:20.651362+0000","last_peered":"2024-03-27T06:42:20.651362+0000","last_clean":"2024-03-27T06:42:20.651362+0000","last_became_active":"2024-03-27T06:41:29.412576+0000","last_became_peered":"2024-03-27T06:41:29.412576+0000","last_unstale":"2024-03-27T06:42:20.651362+0000","last_undegraded":"2024-03-27T06:42:20.651362+0000","last_fullsized":"2024-03-27T06:42:20.651362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:01:22.389424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650603+0000","last_change":"2024-03-27T06:41:28.397135+0000","last_active":"2024-03-27T06:42:20.650603+0000","last_peered":"2024-03-27T06:42:20.650603+0000","last_clean":"2024-03-27T06:42:20.650603+0000","last_became_active":"2024-03-27T06:41:28.396483+0000","last_became_peered":"2024-03-27T06:41:28.396483+0000","last_unstale":"2024-03-27T06:42:20.650603+0000","last_undegraded":"2024-03-27T06:42:20.650603+0000","last_fullsized":"2024-03-27T06:42:20.650603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:05:02.038024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651233+0000","last_change":"2024-03-27T06:41:29.418629+0000","last_active":"2024-03-27T06:42:20.651233+0000","last_peered":"2024-03-27T06:42:20.651233+0000","last_clean":"2024-03-27T06:42:20.651233+0000","last_became_active":"2024-03-27T06:41:29.418278+0000","last_became_peered":"2024-03-27T06:41:29.418278+0000","last_unstale":"2024-03-27T06:42:20.651233+0000","last_undegraded":"2024-03-27T06:42:20.651233+0000","last_fullsized":"2024-03-27T06:42:20.651233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:33:54.461349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:43.550542+0000","last_change":"2024-03-27T06:41:28.400611+0000","last_active":"2024-03-27T06:42:43.550542+0000","last_peered":"2024-03-27T06:42:43.550542+0000","last_clean":"2024-03-27T06:42:43.550542+0000","last_became_active":"2024-03-27T06:41:28.400211+0000","last_became_peered":"2024-03-27T06:41:28.400211+0000","last_unstale":"2024-03-27T06:42:43.550542+0000","last_undegraded":"2024-03-27T06:42:43.550542+0000","last_fullsized":"2024-03-27T06:42:43.550542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:46:35.491071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651106+0000","last_change":"2024-03-27T06:41:29.413246+0000","last_active":"2024-03-27T06:42:20.651106+0000","last_peered":"2024-03-27T06:42:20.651106+0000","last_clean":"2024-03-27T06:42:20.651106+0000","last_became_active":"2024-03-27T06:41:29.412946+0000","last_became_peered":"2024-03-27T06:41:29.412946+0000","last_unstale":"2024-03-27T06:42:20.651106+0000","last_undegraded":"2024-03-27T06:42:20.651106+0000","last_fullsized":"2024-03-27T06:42:20.651106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:41:51.441233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.721062+0000","last_change":"2024-03-27T06:41:29.402160+0000","last_active":"2024-03-27T06:42:20.721062+0000","last_peered":"2024-03-27T06:42:20.721062+0000","last_clean":"2024-03-27T06:42:20.721062+0000","last_became_active":"2024-03-27T06:41:29.401789+0000","last_became_peered":"2024-03-27T06:41:29.401789+0000","last_unstale":"2024-03-27T06:42:20.721062+0000","last_undegraded":"2024-03-27T06:42:20.721062+0000","last_fullsized":"2024-03-27T06:42:20.721062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:31:27.821966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:42.549150+0000","last_change":"2024-03-27T06:41:28.395978+0000","last_active":"2024-03-27T06:42:42.549150+0000","last_peered":"2024-03-27T06:42:42.549150+0000","last_clean":"2024-03-27T06:42:42.549150+0000","last_became_active":"2024-03-27T06:41:28.395650+0000","last_became_peered":"2024-03-27T06:41:28.395650+0000","last_unstale":"2024-03-27T06:42:42.549150+0000","last_undegraded":"2024-03-27T06:42:42.549150+0000","last_fullsized":"2024-03-27T06:42:42.549150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:01:15.841344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658935+0000","last_change":"2024-03-27T06:41:29.423303+0000","last_active":"2024-03-27T06:42:20.658935+0000","last_peered":"2024-03-27T06:42:20.658935+0000","last_clean":"2024-03-27T06:42:20.658935+0000","last_became_active":"2024-03-27T06:41:29.422977+0000","last_became_peered":"2024-03-27T06:41:29.422977+0000","last_unstale":"2024-03-27T06:42:20.658935+0000","last_undegraded":"2024-03-27T06:42:20.658935+0000","last_fullsized":"2024-03-27T06:42:20.658935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:14:01.877801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.004415+0000","last_change":"2024-03-27T06:41:28.395141+0000","last_active":"2024-03-27T06:42:21.004415+0000","last_peered":"2024-03-27T06:42:21.004415+0000","last_clean":"2024-03-27T06:42:21.004415+0000","last_became_active":"2024-03-27T06:41:28.394826+0000","last_became_peered":"2024-03-27T06:41:28.394826+0000","last_unstale":"2024-03-27T06:42:21.004415+0000","last_undegraded":"2024-03-27T06:42:21.004415+0000","last_fullsized":"2024-03-27T06:42:21.004415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:29:49.997090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653806+0000","last_change":"2024-03-27T06:40:16.460939+0000","last_active":"2024-03-27T06:42:20.653806+0000","last_peered":"2024-03-27T06:42:20.653806+0000","last_clean":"2024-03-27T06:42:20.653806+0000","last_became_active":"2024-03-27T06:40:16.460453+0000","last_became_peered":"2024-03-27T06:40:16.460453+0000","last_unstale":"2024-03-27T06:42:20.653806+0000","last_undegraded":"2024-03-27T06:42:20.653806+0000","last_fullsized":"2024-03-27T06:42:20.653806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_clean_scrub_stamp":"2024-03-27T06:36:48.325777+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:28.241688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658690+0000","last_change":"2024-03-27T06:42:20.658690+0000","last_active":"2024-03-27T06:42:20.658690+0000","last_peered":"2024-03-27T06:42:20.658690+0000","last_clean":"2024-03-27T06:42:20.658690+0000","last_became_active":"2024-03-27T06:42:20.658348+0000","last_became_peered":"2024-03-27T06:42:20.658348+0000","last_unstale":"2024-03-27T06:42:20.658690+0000","last_undegraded":"2024-03-27T06:42:20.658690+0000","last_fullsized":"2024-03-27T06:42:20.658690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:12:29.985173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653871+0000","last_change":"2024-03-27T06:41:28.399205+0000","last_active":"2024-03-27T06:42:20.653871+0000","last_peered":"2024-03-27T06:42:20.653871+0000","last_clean":"2024-03-27T06:42:20.653871+0000","last_became_active":"2024-03-27T06:41:28.398889+0000","last_became_peered":"2024-03-27T06:41:28.398889+0000","last_unstale":"2024-03-27T06:42:20.653871+0000","last_undegraded":"2024-03-27T06:42:20.653871+0000","last_fullsized":"2024-03-27T06:42:20.653871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:28:08.769187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.663743+0000","last_change":"2024-03-27T06:42:20.663370+0000","last_active":"2024-03-27T06:42:20.663743+0000","last_peered":"2024-03-27T06:42:20.663743+0000","last_clean":"2024-03-27T06:42:20.663743+0000","last_became_active":"2024-03-27T06:42:20.654243+0000","last_became_peered":"2024-03-27T06:42:20.654243+0000","last_unstale":"2024-03-27T06:42:20.663743+0000","last_undegraded":"2024-03-27T06:42:20.663743+0000","last_fullsized":"2024-03-27T06:42:20.663743+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:43:46.566952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654169+0000","last_change":"2024-03-27T06:41:29.395466+0000","last_active":"2024-03-27T06:42:20.654169+0000","last_peered":"2024-03-27T06:42:20.654169+0000","last_clean":"2024-03-27T06:42:20.654169+0000","last_became_active":"2024-03-27T06:41:29.395094+0000","last_became_peered":"2024-03-27T06:41:29.395094+0000","last_unstale":"2024-03-27T06:42:20.654169+0000","last_undegraded":"2024-03-27T06:42:20.654169+0000","last_fullsized":"2024-03-27T06:42:20.654169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:29:01.746809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650682+0000","last_change":"2024-03-27T06:41:28.400420+0000","last_active":"2024-03-27T06:42:20.650682+0000","last_peered":"2024-03-27T06:42:20.650682+0000","last_clean":"2024-03-27T06:42:20.650682+0000","last_became_active":"2024-03-27T06:41:28.400115+0000","last_became_peered":"2024-03-27T06:41:28.400115+0000","last_unstale":"2024-03-27T06:42:20.650682+0000","last_undegraded":"2024-03-27T06:42:20.650682+0000","last_fullsized":"2024-03-27T06:42:20.650682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:37:40.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657748+0000","last_change":"2024-03-27T06:41:29.411428+0000","last_active":"2024-03-27T06:42:20.657748+0000","last_peered":"2024-03-27T06:42:20.657748+0000","last_clean":"2024-03-27T06:42:20.657748+0000","last_became_active":"2024-03-27T06:41:29.411075+0000","last_became_peered":"2024-03-27T06:41:29.411075+0000","last_unstale":"2024-03-27T06:42:20.657748+0000","last_undegraded":"2024-03-27T06:42:20.657748+0000","last_fullsized":"2024-03-27T06:42:20.657748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:55.764459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.004153+0000","last_change":"2024-03-27T06:41:28.399799+0000","last_active":"2024-03-27T06:42:21.004153+0000","last_peered":"2024-03-27T06:42:21.004153+0000","last_clean":"2024-03-27T06:42:21.004153+0000","last_became_active":"2024-03-27T06:41:28.399385+0000","last_became_peered":"2024-03-27T06:41:28.399385+0000","last_unstale":"2024-03-27T06:42:21.004153+0000","last_undegraded":"2024-03-27T06:42:21.004153+0000","last_fullsized":"2024-03-27T06:42:21.004153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:16:18.005519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653268+0000","last_change":"2024-03-27T06:41:29.418991+0000","last_active":"2024-03-27T06:42:20.653268+0000","last_peered":"2024-03-27T06:42:20.653268+0000","last_clean":"2024-03-27T06:42:20.653268+0000","last_became_active":"2024-03-27T06:41:29.418527+0000","last_became_peered":"2024-03-27T06:41:29.418527+0000","last_unstale":"2024-03-27T06:42:20.653268+0000","last_undegraded":"2024-03-27T06:42:20.653268+0000","last_fullsized":"2024-03-27T06:42:20.653268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:49:55.913276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.655725+0000","last_change":"2024-03-27T06:41:28.397771+0000","last_active":"2024-03-27T06:42:20.655725+0000","last_peered":"2024-03-27T06:42:20.655725+0000","last_clean":"2024-03-27T06:42:20.655725+0000","last_became_active":"2024-03-27T06:41:28.397470+0000","last_became_peered":"2024-03-27T06:41:28.397470+0000","last_unstale":"2024-03-27T06:42:20.655725+0000","last_undegraded":"2024-03-27T06:42:20.655725+0000","last_fullsized":"2024-03-27T06:42:20.655725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:37:21.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.652813+0000","last_change":"2024-03-27T06:41:29.412280+0000","last_active":"2024-03-27T06:42:20.652813+0000","last_peered":"2024-03-27T06:42:20.652813+0000","last_clean":"2024-03-27T06:42:20.652813+0000","last_became_active":"2024-03-27T06:41:29.411336+0000","last_became_peered":"2024-03-27T06:41:29.411336+0000","last_unstale":"2024-03-27T06:42:20.652813+0000","last_undegraded":"2024-03-27T06:42:20.652813+0000","last_fullsized":"2024-03-27T06:42:20.652813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:13:03.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658518+0000","last_change":"2024-03-27T06:41:28.403529+0000","last_active":"2024-03-27T06:42:20.658518+0000","last_peered":"2024-03-27T06:42:20.658518+0000","last_clean":"2024-03-27T06:42:20.658518+0000","last_became_active":"2024-03-27T06:41:28.403155+0000","last_became_peered":"2024-03-27T06:41:28.403155+0000","last_unstale":"2024-03-27T06:42:20.658518+0000","last_undegraded":"2024-03-27T06:42:20.658518+0000","last_fullsized":"2024-03-27T06:42:20.658518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:23:53.971374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650720+0000","last_change":"2024-03-27T06:41:29.404929+0000","last_active":"2024-03-27T06:42:20.650720+0000","last_peered":"2024-03-27T06:42:20.650720+0000","last_clean":"2024-03-27T06:42:20.650720+0000","last_became_active":"2024-03-27T06:41:29.404602+0000","last_became_peered":"2024-03-27T06:41:29.404602+0000","last_unstale":"2024-03-27T06:42:20.650720+0000","last_undegraded":"2024-03-27T06:42:20.650720+0000","last_fullsized":"2024-03-27T06:42:20.650720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:19:07.379046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.652812+0000","last_change":"2024-03-27T06:41:28.398602+0000","last_active":"2024-03-27T06:42:20.652812+0000","last_peered":"2024-03-27T06:42:20.652812+0000","last_clean":"2024-03-27T06:42:20.652812+0000","last_became_active":"2024-03-27T06:41:28.398068+0000","last_became_peered":"2024-03-27T06:41:28.398068+0000","last_unstale":"2024-03-27T06:42:20.652812+0000","last_undegraded":"2024-03-27T06:42:20.652812+0000","last_fullsized":"2024-03-27T06:42:20.652812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:10:53.312044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657304+0000","last_change":"2024-03-27T06:41:29.420245+0000","last_active":"2024-03-27T06:42:20.657304+0000","last_peered":"2024-03-27T06:42:20.657304+0000","last_clean":"2024-03-27T06:42:20.657304+0000","last_became_active":"2024-03-27T06:41:29.419912+0000","last_became_peered":"2024-03-27T06:41:29.419912+0000","last_unstale":"2024-03-27T06:42:20.657304+0000","last_undegraded":"2024-03-27T06:42:20.657304+0000","last_fullsized":"2024-03-27T06:42:20.657304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:46:58.203083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653250+0000","last_change":"2024-03-27T06:41:28.398595+0000","last_active":"2024-03-27T06:42:20.653250+0000","last_peered":"2024-03-27T06:42:20.653250+0000","last_clean":"2024-03-27T06:42:20.653250+0000","last_became_active":"2024-03-27T06:41:28.398036+0000","last_became_peered":"2024-03-27T06:41:28.398036+0000","last_unstale":"2024-03-27T06:42:20.653250+0000","last_undegraded":"2024-03-27T06:42:20.653250+0000","last_fullsized":"2024-03-27T06:42:20.653250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:47:15.769760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654057+0000","last_change":"2024-03-27T06:41:29.406576+0000","last_active":"2024-03-27T06:42:20.654057+0000","last_peered":"2024-03-27T06:42:20.654057+0000","last_clean":"2024-03-27T06:42:20.654057+0000","last_became_active":"2024-03-27T06:41:29.406222+0000","last_became_peered":"2024-03-27T06:41:29.406222+0000","last_unstale":"2024-03-27T06:42:20.654057+0000","last_undegraded":"2024-03-27T06:42:20.654057+0000","last_fullsized":"2024-03-27T06:42:20.654057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:05:12.446154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:45.552696+0000","last_change":"2024-03-27T06:41:28.399361+0000","last_active":"2024-03-27T06:42:45.552696+0000","last_peered":"2024-03-27T06:42:45.552696+0000","last_clean":"2024-03-27T06:42:45.552696+0000","last_became_active":"2024-03-27T06:41:28.398021+0000","last_became_peered":"2024-03-27T06:41:28.398021+0000","last_unstale":"2024-03-27T06:42:45.552696+0000","last_undegraded":"2024-03-27T06:42:45.552696+0000","last_fullsized":"2024-03-27T06:42:45.552696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:51:34.106340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654197+0000","last_change":"2024-03-27T06:41:29.405696+0000","last_active":"2024-03-27T06:42:20.654197+0000","last_peered":"2024-03-27T06:42:20.654197+0000","last_clean":"2024-03-27T06:42:20.654197+0000","last_became_active":"2024-03-27T06:41:29.405273+0000","last_became_peered":"2024-03-27T06:41:29.405273+0000","last_unstale":"2024-03-27T06:42:20.654197+0000","last_undegraded":"2024-03-27T06:42:20.654197+0000","last_fullsized":"2024-03-27T06:42:20.654197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:02:58.778464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.720949+0000","last_change":"2024-03-27T06:41:28.391406+0000","last_active":"2024-03-27T06:42:20.720949+0000","last_peered":"2024-03-27T06:42:20.720949+0000","last_clean":"2024-03-27T06:42:20.720949+0000","last_became_active":"2024-03-27T06:41:28.390950+0000","last_became_peered":"2024-03-27T06:41:28.390950+0000","last_unstale":"2024-03-27T06:42:20.720949+0000","last_undegraded":"2024-03-27T06:42:20.720949+0000","last_fullsized":"2024-03-27T06:42:20.720949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:10:48.347139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653937+0000","last_change":"2024-03-27T06:41:29.410422+0000","last_active":"2024-03-27T06:42:20.653937+0000","last_peered":"2024-03-27T06:42:20.653937+0000","last_clean":"2024-03-27T06:42:20.653937+0000","last_became_active":"2024-03-27T06:41:29.409738+0000","last_became_peered":"2024-03-27T06:41:29.409738+0000","last_unstale":"2024-03-27T06:42:20.653937+0000","last_undegraded":"2024-03-27T06:42:20.653937+0000","last_fullsized":"2024-03-27T06:42:20.653937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:38:17.546504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.008586+0000","last_change":"2024-03-27T06:42:21.008586+0000","last_active":"2024-03-27T06:42:21.008586+0000","last_peered":"2024-03-27T06:42:21.008586+0000","last_clean":"2024-03-27T06:42:21.008586+0000","last_became_active":"2024-03-27T06:42:21.004964+0000","last_became_peered":"2024-03-27T06:42:21.004964+0000","last_unstale":"2024-03-27T06:42:21.008586+0000","last_undegraded":"2024-03-27T06:42:21.008586+0000","last_fullsized":"2024-03-27T06:42:21.008586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:11:13.246994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654160+0000","last_change":"2024-03-27T06:41:29.408809+0000","last_active":"2024-03-27T06:42:20.654160+0000","last_peered":"2024-03-27T06:42:20.654160+0000","last_clean":"2024-03-27T06:42:20.654160+0000","last_became_active":"2024-03-27T06:41:29.408447+0000","last_became_peered":"2024-03-27T06:41:29.408447+0000","last_unstale":"2024-03-27T06:42:20.654160+0000","last_undegraded":"2024-03-27T06:42:20.654160+0000","last_fullsized":"2024-03-27T06:42:20.654160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:04:55.269490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.655859+0000","last_change":"2024-03-27T06:42:20.655859+0000","last_active":"2024-03-27T06:42:20.655859+0000","last_peered":"2024-03-27T06:42:20.655859+0000","last_clean":"2024-03-27T06:42:20.655859+0000","last_became_active":"2024-03-27T06:42:20.655446+0000","last_became_peered":"2024-03-27T06:42:20.655446+0000","last_unstale":"2024-03-27T06:42:20.655859+0000","last_undegraded":"2024-03-27T06:42:20.655859+0000","last_fullsized":"2024-03-27T06:42:20.655859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:43:07.920040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653684+0000","last_change":"2024-03-27T06:41:29.409896+0000","last_active":"2024-03-27T06:42:20.653684+0000","last_peered":"2024-03-27T06:42:20.653684+0000","last_clean":"2024-03-27T06:42:20.653684+0000","last_became_active":"2024-03-27T06:41:29.408279+0000","last_became_peered":"2024-03-27T06:41:29.408279+0000","last_unstale":"2024-03-27T06:42:20.653684+0000","last_undegraded":"2024-03-27T06:42:20.653684+0000","last_fullsized":"2024-03-27T06:42:20.653684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:49:03.257543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.652699+0000","last_change":"2024-03-27T06:41:29.412526+0000","last_active":"2024-03-27T06:42:20.652699+0000","last_peered":"2024-03-27T06:42:20.652699+0000","last_clean":"2024-03-27T06:42:20.652699+0000","last_became_active":"2024-03-27T06:41:29.412180+0000","last_became_peered":"2024-03-27T06:41:29.412180+0000","last_unstale":"2024-03-27T06:42:20.652699+0000","last_undegraded":"2024-03-27T06:42:20.652699+0000","last_fullsized":"2024-03-27T06:42:20.652699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:47.461457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657735+0000","last_change":"2024-03-27T06:42:20.657735+0000","last_active":"2024-03-27T06:42:20.657735+0000","last_peered":"2024-03-27T06:42:20.657735+0000","last_clean":"2024-03-27T06:42:20.657735+0000","last_became_active":"2024-03-27T06:42:20.657378+0000","last_became_peered":"2024-03-27T06:42:20.657378+0000","last_unstale":"2024-03-27T06:42:20.657735+0000","last_undegraded":"2024-03-27T06:42:20.657735+0000","last_fullsized":"2024-03-27T06:42:20.657735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:13:56.368229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654275+0000","last_change":"2024-03-27T06:41:29.402442+0000","last_active":"2024-03-27T06:42:20.654275+0000","last_peered":"2024-03-27T06:42:20.654275+0000","last_clean":"2024-03-27T06:42:20.654275+0000","last_became_active":"2024-03-27T06:41:29.402078+0000","last_became_peered":"2024-03-27T06:41:29.402078+0000","last_unstale":"2024-03-27T06:42:20.654275+0000","last_undegraded":"2024-03-27T06:42:20.654275+0000","last_fullsized":"2024-03-27T06:42:20.654275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:07:35.344881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650986+0000","last_change":"2024-03-27T06:41:29.418563+0000","last_active":"2024-03-27T06:42:20.650986+0000","last_peered":"2024-03-27T06:42:20.650986+0000","last_clean":"2024-03-27T06:42:20.650986+0000","last_became_active":"2024-03-27T06:41:29.418209+0000","last_became_peered":"2024-03-27T06:41:29.418209+0000","last_unstale":"2024-03-27T06:42:20.650986+0000","last_undegraded":"2024-03-27T06:42:20.650986+0000","last_fullsized":"2024-03-27T06:42:20.650986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:53:58.536148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653731+0000","last_change":"2024-03-27T06:41:29.400414+0000","last_active":"2024-03-27T06:42:20.653731+0000","last_peered":"2024-03-27T06:42:20.653731+0000","last_clean":"2024-03-27T06:42:20.653731+0000","last_became_active":"2024-03-27T06:41:29.397865+0000","last_became_peered":"2024-03-27T06:41:29.397865+0000","last_unstale":"2024-03-27T06:42:20.653731+0000","last_undegraded":"2024-03-27T06:42:20.653731+0000","last_fullsized":"2024-03-27T06:42:20.653731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:35:28.405347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657648+0000","last_change":"2024-03-27T06:41:29.416740+0000","last_active":"2024-03-27T06:42:20.657648+0000","last_peered":"2024-03-27T06:42:20.657648+0000","last_clean":"2024-03-27T06:42:20.657648+0000","last_became_active":"2024-03-27T06:41:29.416417+0000","last_became_peered":"2024-03-27T06:41:29.416417+0000","last_unstale":"2024-03-27T06:42:20.657648+0000","last_undegraded":"2024-03-27T06:42:20.657648+0000","last_fullsized":"2024-03-27T06:42:20.657648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:08:34.186527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.720948+0000","last_change":"2024-03-27T06:41:29.418797+0000","last_active":"2024-03-27T06:42:20.720948+0000","last_peered":"2024-03-27T06:42:20.720948+0000","last_clean":"2024-03-27T06:42:20.720948+0000","last_became_active":"2024-03-27T06:41:29.418472+0000","last_became_peered":"2024-03-27T06:41:29.418472+0000","last_unstale":"2024-03-27T06:42:20.720948+0000","last_undegraded":"2024-03-27T06:42:20.720948+0000","last_fullsized":"2024-03-27T06:42:20.720948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:30:55.540353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658425+0000","last_change":"2024-03-27T06:41:29.416124+0000","last_active":"2024-03-27T06:42:20.658425+0000","last_peered":"2024-03-27T06:42:20.658425+0000","last_clean":"2024-03-27T06:42:20.658425+0000","last_became_active":"2024-03-27T06:41:29.415665+0000","last_became_peered":"2024-03-27T06:41:29.415665+0000","last_unstale":"2024-03-27T06:42:20.658425+0000","last_undegraded":"2024-03-27T06:42:20.658425+0000","last_fullsized":"2024-03-27T06:42:20.658425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:46:04.160705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.655988+0000","last_change":"2024-03-27T06:41:29.417165+0000","last_active":"2024-03-27T06:42:20.655988+0000","last_peered":"2024-03-27T06:42:20.655988+0000","last_clean":"2024-03-27T06:42:20.655988+0000","last_became_active":"2024-03-27T06:41:29.416856+0000","last_became_peered":"2024-03-27T06:41:29.416856+0000","last_unstale":"2024-03-27T06:42:20.655988+0000","last_undegraded":"2024-03-27T06:42:20.655988+0000","last_fullsized":"2024-03-27T06:42:20.655988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:55:20.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654515+0000","last_change":"2024-03-27T06:41:29.412377+0000","last_active":"2024-03-27T06:42:20.654515+0000","last_peered":"2024-03-27T06:42:20.654515+0000","last_clean":"2024-03-27T06:42:20.654515+0000","last_became_active":"2024-03-27T06:41:29.411761+0000","last_became_peered":"2024-03-27T06:41:29.411761+0000","last_unstale":"2024-03-27T06:42:20.654515+0000","last_undegraded":"2024-03-27T06:42:20.654515+0000","last_fullsized":"2024-03-27T06:42:20.654515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:16:56.844342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650715+0000","last_change":"2024-03-27T06:41:29.413660+0000","last_active":"2024-03-27T06:42:20.650715+0000","last_peered":"2024-03-27T06:42:20.650715+0000","last_clean":"2024-03-27T06:42:20.650715+0000","last_became_active":"2024-03-27T06:41:29.413201+0000","last_became_peered":"2024-03-27T06:41:29.413201+0000","last_unstale":"2024-03-27T06:42:20.650715+0000","last_undegraded":"2024-03-27T06:42:20.650715+0000","last_fullsized":"2024-03-27T06:42:20.650715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:14:46.446595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":278864,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67227,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":733,"num_read_kb":434,"num_write":238,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":460074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":1111293,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":459780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":459934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1053067,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":459497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1028096,"data_stored":520433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":460836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":521044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":459322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":459749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1052354,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T06:42:48.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-27T06:42:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:49.573 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:49 smithi016 ceph-mon[31908]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:49.573 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:49 smithi016 ceph-mon[31908]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:42:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:49 smithi156 ceph-mon[36261]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:49 smithi156 ceph-mon[36261]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:42:49.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:49 smithi170 ceph-mon[36438]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:42:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:49 smithi170 ceph-mon[36438]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:42:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.600+0000 7f99a345c700 1 -- 172.21.15.16:0/2890443332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c103080 msgr2=0x7f999c10f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.600+0000 7f99a345c700 1 --2- 172.21.15.16:0/2890443332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c103080 0x7f999c10f4f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f999400b0d0 tx=0x7f999400b3e0 comp rx=0 tx=0).stop 2024-03-27T06:42:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.600+0000 7f99a345c700 1 -- 172.21.15.16:0/2890443332 shutdown_connections 2024-03-27T06:42:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.600+0000 7f99a345c700 1 --2- 172.21.15.16:0/2890443332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c103080 0x7f999c10f4f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.600+0000 7f99a345c700 1 --2- 172.21.15.16:0/2890443332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f999c107510 0x7f999c1078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.600+0000 7f99a345c700 1 --2- 172.21.15.16:0/2890443332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c1026c0 0x7f999c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.600+0000 7f99a345c700 1 -- 172.21.15.16:0/2890443332 >> 172.21.15.16:0/2890443332 conn(0x7f999c0797a0 msgr2=0x7f999c079bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a345c700 1 -- 172.21.15.16:0/2890443332 shutdown_connections 2024-03-27T06:42:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a345c700 1 -- 172.21.15.16:0/2890443332 wait complete. 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a345c700 1 Processor -- start 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a345c700 1 -- start start 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a345c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f999c1026c0 0x7f999c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a345c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c103080 0x7f999c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a345c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c107510 0x7f999c19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a345c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f999c112ab0 con 0x7f999c103080 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a345c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f999c112930 con 0x7f999c1026c0 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a345c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f999c112c30 con 0x7f999c107510 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a09f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c103080 0x7f999c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.601+0000 7f99a09f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c103080 0x7f999c199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43054/0 (socket says 172.21.15.16:43054) 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99a09f7700 1 -- 172.21.15.16:0/1778686753 learned_addr learned my addr 172.21.15.16:0/1778686753 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99a11f8700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f999c1026c0 0x7f999c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99a19f9700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c107510 0x7f999c19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99a09f7700 1 -- 172.21.15.16:0/1778686753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c107510 msgr2=0x7f999c19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99a09f7700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c107510 0x7f999c19d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99a09f7700 1 -- 172.21.15.16:0/1778686753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f999c1026c0 msgr2=0x7f999c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99a09f7700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f999c1026c0 0x7f999c198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99a09f7700 1 -- 172.21.15.16:0/1778686753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c19df30 con 0x7f999c103080 2024-03-27T06:42:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99a09f7700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c103080 0x7f999c199460 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f998c00e9c0 tx=0x7f998c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99927fc700 1 -- 172.21.15.16:0/1778686753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c00cbd0 con 0x7f999c103080 2024-03-27T06:42:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99927fc700 1 -- 172.21.15.16:0/1778686753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f998c00ee60 con 0x7f999c103080 2024-03-27T06:42:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99a345c700 1 -- 172.21.15.16:0/1778686753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c19e220 con 0x7f999c103080 2024-03-27T06:42:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99927fc700 1 -- 172.21.15.16:0/1778686753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c010430 con 0x7f999c103080 2024-03-27T06:42:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.602+0000 7f99a345c700 1 -- 172.21.15.16:0/1778686753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c1a5aa0 con 0x7f999c103080 2024-03-27T06:42:50.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.603+0000 7f99a345c700 1 -- 172.21.15.16:0/1778686753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999c069300 con 0x7f999c103080 2024-03-27T06:42:50.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.604+0000 7f99927fc700 1 -- 172.21.15.16:0/1778686753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138421+0+0 (secure 0 0 0) 0x7f998c004750 con 0x7f999c103080 2024-03-27T06:42:50.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.605+0000 7f99927fc700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99880a3730 0x7f99880a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:50.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.605+0000 7f99a11f8700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99880a3730 0x7f99880a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:50.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.605+0000 7f99927fc700 1 -- 172.21.15.16:0/1778686753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f998c0d0100 con 0x7f999c103080 2024-03-27T06:42:50.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.606+0000 7f99a11f8700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99880a3730 0x7f99880a5bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9998009be0 tx=0x7f9998009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:50.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.610+0000 7f99927fc700 1 -- 172.21.15.16:0/1778686753 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f998c09cb80 con 0x7f999c103080 2024-03-27T06:42:50.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.811+0000 7f99a345c700 1 -- 172.21.15.16:0/1778686753 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f999c19a280 con 0x7f99880a3730 2024-03-27T06:42:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.818+0000 7f99927fc700 1 -- 172.21.15.16:0/1778686753 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258934 (secure 0 0 0) 0x7f999c19a280 con 0x7f99880a3730 2024-03-27T06:42:50.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 -- 172.21.15.16:0/1778686753 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99880a3730 msgr2=0x7f99880a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99880a3730 0x7f99880a5bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9998009be0 tx=0x7f9998009260 comp rx=0 tx=0).stop 2024-03-27T06:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 -- 172.21.15.16:0/1778686753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c103080 msgr2=0x7f999c199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c103080 0x7f999c199460 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f998c00e9c0 tx=0x7f998c00ecd0 comp rx=0 tx=0).stop 2024-03-27T06:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 -- 172.21.15.16:0/1778686753 shutdown_connections 2024-03-27T06:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f99880a3730 0x7f99880a5bf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c107510 0x7f999c19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f999c1026c0 0x7f999c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 --2- 172.21.15.16:0/1778686753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c103080 0x7f999c199460 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 -- 172.21.15.16:0/1778686753 >> 172.21.15.16:0/1778686753 conn(0x7f999c0797a0 msgr2=0x7f999c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 -- 172.21.15.16:0/1778686753 shutdown_connections 2024-03-27T06:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:50.821+0000 7f99a345c700 1 -- 172.21.15.16:0/1778686753 wait complete. 2024-03-27T06:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-03-27T06:42:51.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:51 smithi016 ceph-mon[31908]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:42:51.662 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-03-27T06:42:49.581059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657829,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":894,"num_read_kb":573,"num_write":421,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1180960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365180,"kb_used_data":11808,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576644,"statfs":{"total":1151940427776,"available":1151566483456,"internally_reserved":0,"allocated":12091392,"data_stored":7477383,"data_compressed":106419,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003109"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651071+0000","last_change":"2024-03-27T06:41:29.408953+0000","last_active":"2024-03-27T06:42:20.651071+0000","last_peered":"2024-03-27T06:42:20.651071+0000","last_clean":"2024-03-27T06:42:20.651071+0000","last_became_active":"2024-03-27T06:41:29.408143+0000","last_became_peered":"2024-03-27T06:41:29.408143+0000","last_unstale":"2024-03-27T06:42:20.651071+0000","last_undegraded":"2024-03-27T06:42:20.651071+0000","last_fullsized":"2024-03-27T06:42:20.651071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:22.486951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653219+0000","last_change":"2024-03-27T06:41:29.405080+0000","last_active":"2024-03-27T06:42:20.653219+0000","last_peered":"2024-03-27T06:42:20.653219+0000","last_clean":"2024-03-27T06:42:20.653219+0000","last_became_active":"2024-03-27T06:41:29.404743+0000","last_became_peered":"2024-03-27T06:41:29.404743+0000","last_unstale":"2024-03-27T06:42:20.653219+0000","last_undegraded":"2024-03-27T06:42:20.653219+0000","last_fullsized":"2024-03-27T06:42:20.653219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:24:30.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653238+0000","last_change":"2024-03-27T06:41:29.409346+0000","last_active":"2024-03-27T06:42:20.653238+0000","last_peered":"2024-03-27T06:42:20.653238+0000","last_clean":"2024-03-27T06:42:20.653238+0000","last_became_active":"2024-03-27T06:41:29.408837+0000","last_became_peered":"2024-03-27T06:41:29.408837+0000","last_unstale":"2024-03-27T06:42:20.653238+0000","last_undegraded":"2024-03-27T06:42:20.653238+0000","last_fullsized":"2024-03-27T06:42:20.653238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:11:12.274869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.003961+0000","last_change":"2024-03-27T06:41:29.412115+0000","last_active":"2024-03-27T06:42:21.003961+0000","last_peered":"2024-03-27T06:42:21.003961+0000","last_clean":"2024-03-27T06:42:21.003961+0000","last_became_active":"2024-03-27T06:41:29.411702+0000","last_became_peered":"2024-03-27T06:41:29.411702+0000","last_unstale":"2024-03-27T06:42:21.003961+0000","last_undegraded":"2024-03-27T06:42:21.003961+0000","last_fullsized":"2024-03-27T06:42:21.003961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:04:42.347229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653364+0000","last_change":"2024-03-27T06:41:29.409888+0000","last_active":"2024-03-27T06:42:20.653364+0000","last_peered":"2024-03-27T06:42:20.653364+0000","last_clean":"2024-03-27T06:42:20.653364+0000","last_became_active":"2024-03-27T06:41:29.409558+0000","last_became_peered":"2024-03-27T06:41:29.409558+0000","last_unstale":"2024-03-27T06:42:20.653364+0000","last_undegraded":"2024-03-27T06:42:20.653364+0000","last_fullsized":"2024-03-27T06:42:20.653364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:38:51.452793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650936+0000","last_change":"2024-03-27T06:41:29.404088+0000","last_active":"2024-03-27T06:42:20.650936+0000","last_peered":"2024-03-27T06:42:20.650936+0000","last_clean":"2024-03-27T06:42:20.650936+0000","last_became_active":"2024-03-27T06:41:29.403698+0000","last_became_peered":"2024-03-27T06:41:29.403698+0000","last_unstale":"2024-03-27T06:42:20.650936+0000","last_undegraded":"2024-03-27T06:42:20.650936+0000","last_fullsized":"2024-03-27T06:42:20.650936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:14:31.296523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653732+0000","last_change":"2024-03-27T06:41:29.413157+0000","last_active":"2024-03-27T06:42:20.653732+0000","last_peered":"2024-03-27T06:42:20.653732+0000","last_clean":"2024-03-27T06:42:20.653732+0000","last_became_active":"2024-03-27T06:41:29.412669+0000","last_became_peered":"2024-03-27T06:41:29.412669+0000","last_unstale":"2024-03-27T06:42:20.653732+0000","last_undegraded":"2024-03-27T06:42:20.653732+0000","last_fullsized":"2024-03-27T06:42:20.653732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:28:20.823770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658144+0000","last_change":"2024-03-27T06:41:29.417207+0000","last_active":"2024-03-27T06:42:20.658144+0000","last_peered":"2024-03-27T06:42:20.658144+0000","last_clean":"2024-03-27T06:42:20.658144+0000","last_became_active":"2024-03-27T06:41:29.415556+0000","last_became_peered":"2024-03-27T06:41:29.415556+0000","last_unstale":"2024-03-27T06:42:20.658144+0000","last_undegraded":"2024-03-27T06:42:20.658144+0000","last_fullsized":"2024-03-27T06:42:20.658144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:26:04.040296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.656886+0000","last_change":"2024-03-27T06:41:29.400032+0000","last_active":"2024-03-27T06:42:20.656886+0000","last_peered":"2024-03-27T06:42:20.656886+0000","last_clean":"2024-03-27T06:42:20.656886+0000","last_became_active":"2024-03-27T06:41:29.399274+0000","last_became_peered":"2024-03-27T06:41:29.399274+0000","last_unstale":"2024-03-27T06:42:20.656886+0000","last_undegraded":"2024-03-27T06:42:20.656886+0000","last_fullsized":"2024-03-27T06:42:20.656886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:55.306309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651336+0000","last_change":"2024-03-27T06:41:29.412471+0000","last_active":"2024-03-27T06:42:20.651336+0000","last_peered":"2024-03-27T06:42:20.651336+0000","last_clean":"2024-03-27T06:42:20.651336+0000","last_became_active":"2024-03-27T06:41:29.412134+0000","last_became_peered":"2024-03-27T06:41:29.412134+0000","last_unstale":"2024-03-27T06:42:20.651336+0000","last_undegraded":"2024-03-27T06:42:20.651336+0000","last_fullsized":"2024-03-27T06:42:20.651336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:35:57.146944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654091+0000","last_change":"2024-03-27T06:41:29.407915+0000","last_active":"2024-03-27T06:42:20.654091+0000","last_peered":"2024-03-27T06:42:20.654091+0000","last_clean":"2024-03-27T06:42:20.654091+0000","last_became_active":"2024-03-27T06:41:29.407563+0000","last_became_peered":"2024-03-27T06:41:29.407563+0000","last_unstale":"2024-03-27T06:42:20.654091+0000","last_undegraded":"2024-03-27T06:42:20.654091+0000","last_fullsized":"2024-03-27T06:42:20.654091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:09:05.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.003952+0000","last_change":"2024-03-27T06:41:29.413374+0000","last_active":"2024-03-27T06:42:21.003952+0000","last_peered":"2024-03-27T06:42:21.003952+0000","last_clean":"2024-03-27T06:42:21.003952+0000","last_became_active":"2024-03-27T06:41:29.413010+0000","last_became_peered":"2024-03-27T06:41:29.413010+0000","last_unstale":"2024-03-27T06:42:21.003952+0000","last_undegraded":"2024-03-27T06:42:21.003952+0000","last_fullsized":"2024-03-27T06:42:21.003952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:48:52.329611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654602+0000","last_change":"2024-03-27T06:41:29.408640+0000","last_active":"2024-03-27T06:42:20.654602+0000","last_peered":"2024-03-27T06:42:20.654602+0000","last_clean":"2024-03-27T06:42:20.654602+0000","last_became_active":"2024-03-27T06:41:29.408268+0000","last_became_peered":"2024-03-27T06:41:29.408268+0000","last_unstale":"2024-03-27T06:42:20.654602+0000","last_undegraded":"2024-03-27T06:42:20.654602+0000","last_fullsized":"2024-03-27T06:42:20.654602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:51:59.803596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653170+0000","last_change":"2024-03-27T06:41:29.420436+0000","last_active":"2024-03-27T06:42:20.653170+0000","last_peered":"2024-03-27T06:42:20.653170+0000","last_clean":"2024-03-27T06:42:20.653170+0000","last_became_active":"2024-03-27T06:41:29.419426+0000","last_became_peered":"2024-03-27T06:41:29.419426+0000","last_unstale":"2024-03-27T06:42:20.653170+0000","last_undegraded":"2024-03-27T06:42:20.653170+0000","last_fullsized":"2024-03-27T06:42:20.653170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:38:15.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653035+0000","last_change":"2024-03-27T06:41:29.415673+0000","last_active":"2024-03-27T06:42:20.653035+0000","last_peered":"2024-03-27T06:42:20.653035+0000","last_clean":"2024-03-27T06:42:20.653035+0000","last_became_active":"2024-03-27T06:41:29.414741+0000","last_became_peered":"2024-03-27T06:41:29.414741+0000","last_unstale":"2024-03-27T06:42:20.653035+0000","last_undegraded":"2024-03-27T06:42:20.653035+0000","last_fullsized":"2024-03-27T06:42:20.653035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:38:43.183517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.655655+0000","last_change":"2024-03-27T06:41:29.408609+0000","last_active":"2024-03-27T06:42:20.655655+0000","last_peered":"2024-03-27T06:42:20.655655+0000","last_clean":"2024-03-27T06:42:20.655655+0000","last_became_active":"2024-03-27T06:41:29.408252+0000","last_became_peered":"2024-03-27T06:41:29.408252+0000","last_unstale":"2024-03-27T06:42:20.655655+0000","last_undegraded":"2024-03-27T06:42:20.655655+0000","last_fullsized":"2024-03-27T06:42:20.655655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:14:13.609113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651294+0000","last_change":"2024-03-27T06:41:29.409221+0000","last_active":"2024-03-27T06:42:20.651294+0000","last_peered":"2024-03-27T06:42:20.651294+0000","last_clean":"2024-03-27T06:42:20.651294+0000","last_became_active":"2024-03-27T06:41:29.408598+0000","last_became_peered":"2024-03-27T06:41:29.408598+0000","last_unstale":"2024-03-27T06:42:20.651294+0000","last_undegraded":"2024-03-27T06:42:20.651294+0000","last_fullsized":"2024-03-27T06:42:20.651294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:42:16.251149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650671+0000","last_change":"2024-03-27T06:41:29.406143+0000","last_active":"2024-03-27T06:42:20.650671+0000","last_peered":"2024-03-27T06:42:20.650671+0000","last_clean":"2024-03-27T06:42:20.650671+0000","last_became_active":"2024-03-27T06:41:29.405754+0000","last_became_peered":"2024-03-27T06:41:29.405754+0000","last_unstale":"2024-03-27T06:42:20.650671+0000","last_undegraded":"2024-03-27T06:42:20.650671+0000","last_fullsized":"2024-03-27T06:42:20.650671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:42:47.787099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653263+0000","last_change":"2024-03-27T06:41:29.402471+0000","last_active":"2024-03-27T06:42:20.653263+0000","last_peered":"2024-03-27T06:42:20.653263+0000","last_clean":"2024-03-27T06:42:20.653263+0000","last_became_active":"2024-03-27T06:41:29.402113+0000","last_became_peered":"2024-03-27T06:41:29.402113+0000","last_unstale":"2024-03-27T06:42:20.653263+0000","last_undegraded":"2024-03-27T06:42:20.653263+0000","last_fullsized":"2024-03-27T06:42:20.653263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:34:16.408058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:45.552237+0000","last_change":"2024-03-27T06:41:28.397815+0000","last_active":"2024-03-27T06:42:45.552237+0000","last_peered":"2024-03-27T06:42:45.552237+0000","last_clean":"2024-03-27T06:42:45.552237+0000","last_became_active":"2024-03-27T06:41:28.397503+0000","last_became_peered":"2024-03-27T06:41:28.397503+0000","last_unstale":"2024-03-27T06:42:45.552237+0000","last_undegraded":"2024-03-27T06:42:45.552237+0000","last_fullsized":"2024-03-27T06:42:45.552237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:15:53.751809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657518+0000","last_change":"2024-03-27T06:41:29.419528+0000","last_active":"2024-03-27T06:42:20.657518+0000","last_peered":"2024-03-27T06:42:20.657518+0000","last_clean":"2024-03-27T06:42:20.657518+0000","last_became_active":"2024-03-27T06:41:29.417091+0000","last_became_peered":"2024-03-27T06:41:29.417091+0000","last_unstale":"2024-03-27T06:42:20.657518+0000","last_undegraded":"2024-03-27T06:42:20.657518+0000","last_fullsized":"2024-03-27T06:42:20.657518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:13:14.629211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657772+0000","last_change":"2024-03-27T06:41:28.400468+0000","last_active":"2024-03-27T06:42:20.657772+0000","last_peered":"2024-03-27T06:42:20.657772+0000","last_clean":"2024-03-27T06:42:20.657772+0000","last_became_active":"2024-03-27T06:41:28.400190+0000","last_became_peered":"2024-03-27T06:41:28.400190+0000","last_unstale":"2024-03-27T06:42:20.657772+0000","last_undegraded":"2024-03-27T06:42:20.657772+0000","last_fullsized":"2024-03-27T06:42:20.657772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:24:17.059874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653289+0000","last_change":"2024-03-27T06:41:29.409364+0000","last_active":"2024-03-27T06:42:20.653289+0000","last_peered":"2024-03-27T06:42:20.653289+0000","last_clean":"2024-03-27T06:42:20.653289+0000","last_became_active":"2024-03-27T06:41:29.408946+0000","last_became_peered":"2024-03-27T06:41:29.408946+0000","last_unstale":"2024-03-27T06:42:20.653289+0000","last_undegraded":"2024-03-27T06:42:20.653289+0000","last_fullsized":"2024-03-27T06:42:20.653289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:46:17.132071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654300+0000","last_change":"2024-03-27T06:41:28.401327+0000","last_active":"2024-03-27T06:42:20.654300+0000","last_peered":"2024-03-27T06:42:20.654300+0000","last_clean":"2024-03-27T06:42:20.654300+0000","last_became_active":"2024-03-27T06:41:28.400524+0000","last_became_peered":"2024-03-27T06:41:28.400524+0000","last_unstale":"2024-03-27T06:42:20.654300+0000","last_undegraded":"2024-03-27T06:42:20.654300+0000","last_fullsized":"2024-03-27T06:42:20.654300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:11:52.608690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.656060+0000","last_change":"2024-03-27T06:41:29.403042+0000","last_active":"2024-03-27T06:42:20.656060+0000","last_peered":"2024-03-27T06:42:20.656060+0000","last_clean":"2024-03-27T06:42:20.656060+0000","last_became_active":"2024-03-27T06:41:29.402516+0000","last_became_peered":"2024-03-27T06:41:29.402516+0000","last_unstale":"2024-03-27T06:42:20.656060+0000","last_undegraded":"2024-03-27T06:42:20.656060+0000","last_fullsized":"2024-03-27T06:42:20.656060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:20:55.777498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653689+0000","last_change":"2024-03-27T06:41:28.395155+0000","last_active":"2024-03-27T06:42:20.653689+0000","last_peered":"2024-03-27T06:42:20.653689+0000","last_clean":"2024-03-27T06:42:20.653689+0000","last_became_active":"2024-03-27T06:41:28.394760+0000","last_became_peered":"2024-03-27T06:41:28.394760+0000","last_unstale":"2024-03-27T06:42:20.653689+0000","last_undegraded":"2024-03-27T06:42:20.653689+0000","last_fullsized":"2024-03-27T06:42:20.653689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:15:28.182581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.721029+0000","last_change":"2024-03-27T06:41:29.409747+0000","last_active":"2024-03-27T06:42:20.721029+0000","last_peered":"2024-03-27T06:42:20.721029+0000","last_clean":"2024-03-27T06:42:20.721029+0000","last_became_active":"2024-03-27T06:41:29.409425+0000","last_became_peered":"2024-03-27T06:41:29.409425+0000","last_unstale":"2024-03-27T06:42:20.721029+0000","last_undegraded":"2024-03-27T06:42:20.721029+0000","last_fullsized":"2024-03-27T06:42:20.721029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:36.181279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:45.547898+0000","last_change":"2024-03-27T06:41:28.394726+0000","last_active":"2024-03-27T06:42:45.547898+0000","last_peered":"2024-03-27T06:42:45.547898+0000","last_clean":"2024-03-27T06:42:45.547898+0000","last_became_active":"2024-03-27T06:41:28.394378+0000","last_became_peered":"2024-03-27T06:41:28.394378+0000","last_unstale":"2024-03-27T06:42:45.547898+0000","last_undegraded":"2024-03-27T06:42:45.547898+0000","last_fullsized":"2024-03-27T06:42:45.547898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:30:22.194211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.652987+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:42:20.652987+0000","last_peered":"2024-03-27T06:42:20.652987+0000","last_clean":"2024-03-27T06:42:20.652987+0000","last_became_active":"2024-03-27T06:41:29.408826+0000","last_became_peered":"2024-03-27T06:41:29.408826+0000","last_unstale":"2024-03-27T06:42:20.652987+0000","last_undegraded":"2024-03-27T06:42:20.652987+0000","last_fullsized":"2024-03-27T06:42:20.652987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:50:53.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650936+0000","last_change":"2024-03-27T06:41:28.393655+0000","last_active":"2024-03-27T06:42:20.650936+0000","last_peered":"2024-03-27T06:42:20.650936+0000","last_clean":"2024-03-27T06:42:20.650936+0000","last_became_active":"2024-03-27T06:41:28.393245+0000","last_became_peered":"2024-03-27T06:41:28.393245+0000","last_unstale":"2024-03-27T06:42:20.650936+0000","last_undegraded":"2024-03-27T06:42:20.650936+0000","last_fullsized":"2024-03-27T06:42:20.650936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:58:21.169444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.721047+0000","last_change":"2024-03-27T06:41:29.417730+0000","last_active":"2024-03-27T06:42:20.721047+0000","last_peered":"2024-03-27T06:42:20.721047+0000","last_clean":"2024-03-27T06:42:20.721047+0000","last_became_active":"2024-03-27T06:41:29.417319+0000","last_became_peered":"2024-03-27T06:41:29.417319+0000","last_unstale":"2024-03-27T06:42:20.721047+0000","last_undegraded":"2024-03-27T06:42:20.721047+0000","last_fullsized":"2024-03-27T06:42:20.721047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:25.224220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654549+0000","last_change":"2024-03-27T06:41:28.402259+0000","last_active":"2024-03-27T06:42:20.654549+0000","last_peered":"2024-03-27T06:42:20.654549+0000","last_clean":"2024-03-27T06:42:20.654549+0000","last_became_active":"2024-03-27T06:41:28.401922+0000","last_became_peered":"2024-03-27T06:41:28.401922+0000","last_unstale":"2024-03-27T06:42:20.654549+0000","last_undegraded":"2024-03-27T06:42:20.654549+0000","last_fullsized":"2024-03-27T06:42:20.654549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:57:33.245183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658242+0000","last_change":"2024-03-27T06:41:29.409034+0000","last_active":"2024-03-27T06:42:20.658242+0000","last_peered":"2024-03-27T06:42:20.658242+0000","last_clean":"2024-03-27T06:42:20.658242+0000","last_became_active":"2024-03-27T06:41:29.408666+0000","last_became_peered":"2024-03-27T06:41:29.408666+0000","last_unstale":"2024-03-27T06:42:20.658242+0000","last_undegraded":"2024-03-27T06:42:20.658242+0000","last_fullsized":"2024-03-27T06:42:20.658242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:09:55.532059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.720843+0000","last_change":"2024-03-27T06:41:28.393802+0000","last_active":"2024-03-27T06:42:20.720843+0000","last_peered":"2024-03-27T06:42:20.720843+0000","last_clean":"2024-03-27T06:42:20.720843+0000","last_became_active":"2024-03-27T06:41:28.393503+0000","last_became_peered":"2024-03-27T06:41:28.393503+0000","last_unstale":"2024-03-27T06:42:20.720843+0000","last_undegraded":"2024-03-27T06:42:20.720843+0000","last_fullsized":"2024-03-27T06:42:20.720843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:17.765709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650454+0000","last_change":"2024-03-27T06:41:29.416557+0000","last_active":"2024-03-27T06:42:20.650454+0000","last_peered":"2024-03-27T06:42:20.650454+0000","last_clean":"2024-03-27T06:42:20.650454+0000","last_became_active":"2024-03-27T06:41:29.416223+0000","last_became_peered":"2024-03-27T06:41:29.416223+0000","last_unstale":"2024-03-27T06:42:20.650454+0000","last_undegraded":"2024-03-27T06:42:20.650454+0000","last_fullsized":"2024-03-27T06:42:20.650454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:54:20.929325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":169,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:47.549532+0000","last_change":"2024-03-27T06:41:28.396014+0000","last_active":"2024-03-27T06:42:47.549532+0000","last_peered":"2024-03-27T06:42:47.549532+0000","last_clean":"2024-03-27T06:42:47.549532+0000","last_became_active":"2024-03-27T06:41:28.395699+0000","last_became_peered":"2024-03-27T06:41:28.395699+0000","last_unstale":"2024-03-27T06:42:47.549532+0000","last_undegraded":"2024-03-27T06:42:47.549532+0000","last_fullsized":"2024-03-27T06:42:47.549532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:00:59.679843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.656463+0000","last_change":"2024-03-27T06:41:29.401668+0000","last_active":"2024-03-27T06:42:20.656463+0000","last_peered":"2024-03-27T06:42:20.656463+0000","last_clean":"2024-03-27T06:42:20.656463+0000","last_became_active":"2024-03-27T06:41:29.401328+0000","last_became_peered":"2024-03-27T06:41:29.401328+0000","last_unstale":"2024-03-27T06:42:20.656463+0000","last_undegraded":"2024-03-27T06:42:20.656463+0000","last_fullsized":"2024-03-27T06:42:20.656463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:32:54.273194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.011596+0000","last_change":"2024-03-27T06:42:21.011596+0000","last_active":"2024-03-27T06:42:21.011596+0000","last_peered":"2024-03-27T06:42:21.011596+0000","last_clean":"2024-03-27T06:42:21.011596+0000","last_became_active":"2024-03-27T06:42:21.005035+0000","last_became_peered":"2024-03-27T06:42:21.005035+0000","last_unstale":"2024-03-27T06:42:21.011596+0000","last_undegraded":"2024-03-27T06:42:21.011596+0000","last_fullsized":"2024-03-27T06:42:21.011596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:51:35.652266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653185+0000","last_change":"2024-03-27T06:41:29.413919+0000","last_active":"2024-03-27T06:42:20.653185+0000","last_peered":"2024-03-27T06:42:20.653185+0000","last_clean":"2024-03-27T06:42:20.653185+0000","last_became_active":"2024-03-27T06:41:29.413566+0000","last_became_peered":"2024-03-27T06:41:29.413566+0000","last_unstale":"2024-03-27T06:42:20.653185+0000","last_undegraded":"2024-03-27T06:42:20.653185+0000","last_fullsized":"2024-03-27T06:42:20.653185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:45:03.705372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.721086+0000","last_change":"2024-03-27T06:41:28.391661+0000","last_active":"2024-03-27T06:42:20.721086+0000","last_peered":"2024-03-27T06:42:20.721086+0000","last_clean":"2024-03-27T06:42:20.721086+0000","last_became_active":"2024-03-27T06:41:28.391230+0000","last_became_peered":"2024-03-27T06:41:28.391230+0000","last_unstale":"2024-03-27T06:42:20.721086+0000","last_undegraded":"2024-03-27T06:42:20.721086+0000","last_fullsized":"2024-03-27T06:42:20.721086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:41:22.789267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.656132+0000","last_change":"2024-03-27T06:41:29.405727+0000","last_active":"2024-03-27T06:42:20.656132+0000","last_peered":"2024-03-27T06:42:20.656132+0000","last_clean":"2024-03-27T06:42:20.656132+0000","last_became_active":"2024-03-27T06:41:29.405413+0000","last_became_peered":"2024-03-27T06:41:29.405413+0000","last_unstale":"2024-03-27T06:42:20.656132+0000","last_undegraded":"2024-03-27T06:42:20.656132+0000","last_fullsized":"2024-03-27T06:42:20.656132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:35:21.575729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'31","reported_seq":202,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:47.549557+0000","last_change":"2024-03-27T06:41:28.391155+0000","last_active":"2024-03-27T06:42:47.549557+0000","last_peered":"2024-03-27T06:42:47.549557+0000","last_clean":"2024-03-27T06:42:47.549557+0000","last_became_active":"2024-03-27T06:41:28.390828+0000","last_became_peered":"2024-03-27T06:41:28.390828+0000","last_unstale":"2024-03-27T06:42:47.549557+0000","last_undegraded":"2024-03-27T06:42:47.549557+0000","last_fullsized":"2024-03-27T06:42:47.549557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:40.192560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":152,"num_read_kb":134,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654043+0000","last_change":"2024-03-27T06:41:29.413283+0000","last_active":"2024-03-27T06:42:20.654043+0000","last_peered":"2024-03-27T06:42:20.654043+0000","last_clean":"2024-03-27T06:42:20.654043+0000","last_became_active":"2024-03-27T06:41:29.412247+0000","last_became_peered":"2024-03-27T06:41:29.412247+0000","last_unstale":"2024-03-27T06:42:20.654043+0000","last_undegraded":"2024-03-27T06:42:20.654043+0000","last_fullsized":"2024-03-27T06:42:20.654043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:50:15.336003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653342+0000","last_change":"2024-03-27T06:41:28.395089+0000","last_active":"2024-03-27T06:42:20.653342+0000","last_peered":"2024-03-27T06:42:20.653342+0000","last_clean":"2024-03-27T06:42:20.653342+0000","last_became_active":"2024-03-27T06:41:28.394145+0000","last_became_peered":"2024-03-27T06:41:28.394145+0000","last_unstale":"2024-03-27T06:42:20.653342+0000","last_undegraded":"2024-03-27T06:42:20.653342+0000","last_fullsized":"2024-03-27T06:42:20.653342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:40.858404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654325+0000","last_change":"2024-03-27T06:41:29.416955+0000","last_active":"2024-03-27T06:42:20.654325+0000","last_peered":"2024-03-27T06:42:20.654325+0000","last_clean":"2024-03-27T06:42:20.654325+0000","last_became_active":"2024-03-27T06:41:29.416616+0000","last_became_peered":"2024-03-27T06:41:29.416616+0000","last_unstale":"2024-03-27T06:42:20.654325+0000","last_undegraded":"2024-03-27T06:42:20.654325+0000","last_fullsized":"2024-03-27T06:42:20.654325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:38:18.791602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.720820+0000","last_change":"2024-03-27T06:41:28.393876+0000","last_active":"2024-03-27T06:42:20.720820+0000","last_peered":"2024-03-27T06:42:20.720820+0000","last_clean":"2024-03-27T06:42:20.720820+0000","last_became_active":"2024-03-27T06:41:28.393557+0000","last_became_peered":"2024-03-27T06:41:28.393557+0000","last_unstale":"2024-03-27T06:42:20.720820+0000","last_undegraded":"2024-03-27T06:42:20.720820+0000","last_fullsized":"2024-03-27T06:42:20.720820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:30.831039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.660026+0000","last_change":"2024-03-27T06:42:20.660026+0000","last_active":"2024-03-27T06:42:20.660026+0000","last_peered":"2024-03-27T06:42:20.660026+0000","last_clean":"2024-03-27T06:42:20.660026+0000","last_became_active":"2024-03-27T06:42:20.659628+0000","last_became_peered":"2024-03-27T06:42:20.659628+0000","last_unstale":"2024-03-27T06:42:20.660026+0000","last_undegraded":"2024-03-27T06:42:20.660026+0000","last_fullsized":"2024-03-27T06:42:20.660026+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:18:59.396472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:45.552846+0000","last_change":"2024-03-27T06:41:28.393849+0000","last_active":"2024-03-27T06:42:45.552846+0000","last_peered":"2024-03-27T06:42:45.552846+0000","last_clean":"2024-03-27T06:42:45.552846+0000","last_became_active":"2024-03-27T06:41:28.393529+0000","last_became_peered":"2024-03-27T06:41:28.393529+0000","last_unstale":"2024-03-27T06:42:45.552846+0000","last_undegraded":"2024-03-27T06:42:45.552846+0000","last_fullsized":"2024-03-27T06:42:45.552846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:24:23.251723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653685+0000","last_change":"2024-03-27T06:41:29.411273+0000","last_active":"2024-03-27T06:42:20.653685+0000","last_peered":"2024-03-27T06:42:20.653685+0000","last_clean":"2024-03-27T06:42:20.653685+0000","last_became_active":"2024-03-27T06:41:29.410936+0000","last_became_peered":"2024-03-27T06:41:29.410936+0000","last_unstale":"2024-03-27T06:42:20.653685+0000","last_undegraded":"2024-03-27T06:42:20.653685+0000","last_fullsized":"2024-03-27T06:42:20.653685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:41.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.674113+0000","last_change":"2024-03-27T06:42:20.673735+0000","last_active":"2024-03-27T06:42:20.674113+0000","last_peered":"2024-03-27T06:42:20.674113+0000","last_clean":"2024-03-27T06:42:20.674113+0000","last_became_active":"2024-03-27T06:42:20.655144+0000","last_became_peered":"2024-03-27T06:42:20.655144+0000","last_unstale":"2024-03-27T06:42:20.674113+0000","last_undegraded":"2024-03-27T06:42:20.674113+0000","last_fullsized":"2024-03-27T06:42:20.674113+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:05:39.059633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.720745+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:42:20.720745+0000","last_peered":"2024-03-27T06:42:20.720745+0000","last_clean":"2024-03-27T06:42:20.720745+0000","last_became_active":"2024-03-27T06:41:29.408815+0000","last_became_peered":"2024-03-27T06:41:29.408815+0000","last_unstale":"2024-03-27T06:42:20.720745+0000","last_undegraded":"2024-03-27T06:42:20.720745+0000","last_fullsized":"2024-03-27T06:42:20.720745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:45.547747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654026+0000","last_change":"2024-03-27T06:41:28.398249+0000","last_active":"2024-03-27T06:42:20.654026+0000","last_peered":"2024-03-27T06:42:20.654026+0000","last_clean":"2024-03-27T06:42:20.654026+0000","last_became_active":"2024-03-27T06:41:28.397176+0000","last_became_peered":"2024-03-27T06:41:28.397176+0000","last_unstale":"2024-03-27T06:42:20.654026+0000","last_undegraded":"2024-03-27T06:42:20.654026+0000","last_fullsized":"2024-03-27T06:42:20.654026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:32.614558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651362+0000","last_change":"2024-03-27T06:41:29.412928+0000","last_active":"2024-03-27T06:42:20.651362+0000","last_peered":"2024-03-27T06:42:20.651362+0000","last_clean":"2024-03-27T06:42:20.651362+0000","last_became_active":"2024-03-27T06:41:29.412576+0000","last_became_peered":"2024-03-27T06:41:29.412576+0000","last_unstale":"2024-03-27T06:42:20.651362+0000","last_undegraded":"2024-03-27T06:42:20.651362+0000","last_fullsized":"2024-03-27T06:42:20.651362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:01:22.389424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650603+0000","last_change":"2024-03-27T06:41:28.397135+0000","last_active":"2024-03-27T06:42:20.650603+0000","last_peered":"2024-03-27T06:42:20.650603+0000","last_clean":"2024-03-27T06:42:20.650603+0000","last_became_active":"2024-03-27T06:41:28.396483+0000","last_became_peered":"2024-03-27T06:41:28.396483+0000","last_unstale":"2024-03-27T06:42:20.650603+0000","last_undegraded":"2024-03-27T06:42:20.650603+0000","last_fullsized":"2024-03-27T06:42:20.650603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:05:02.038024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651233+0000","last_change":"2024-03-27T06:41:29.418629+0000","last_active":"2024-03-27T06:42:20.651233+0000","last_peered":"2024-03-27T06:42:20.651233+0000","last_clean":"2024-03-27T06:42:20.651233+0000","last_became_active":"2024-03-27T06:41:29.418278+0000","last_became_peered":"2024-03-27T06:41:29.418278+0000","last_unstale":"2024-03-27T06:42:20.651233+0000","last_undegraded":"2024-03-27T06:42:20.651233+0000","last_fullsized":"2024-03-27T06:42:20.651233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:33:54.461349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:43.550542+0000","last_change":"2024-03-27T06:41:28.400611+0000","last_active":"2024-03-27T06:42:43.550542+0000","last_peered":"2024-03-27T06:42:43.550542+0000","last_clean":"2024-03-27T06:42:43.550542+0000","last_became_active":"2024-03-27T06:41:28.400211+0000","last_became_peered":"2024-03-27T06:41:28.400211+0000","last_unstale":"2024-03-27T06:42:43.550542+0000","last_undegraded":"2024-03-27T06:42:43.550542+0000","last_fullsized":"2024-03-27T06:42:43.550542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:46:35.491071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.651106+0000","last_change":"2024-03-27T06:41:29.413246+0000","last_active":"2024-03-27T06:42:20.651106+0000","last_peered":"2024-03-27T06:42:20.651106+0000","last_clean":"2024-03-27T06:42:20.651106+0000","last_became_active":"2024-03-27T06:41:29.412946+0000","last_became_peered":"2024-03-27T06:41:29.412946+0000","last_unstale":"2024-03-27T06:42:20.651106+0000","last_undegraded":"2024-03-27T06:42:20.651106+0000","last_fullsized":"2024-03-27T06:42:20.651106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:41:51.441233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.721062+0000","last_change":"2024-03-27T06:41:29.402160+0000","last_active":"2024-03-27T06:42:20.721062+0000","last_peered":"2024-03-27T06:42:20.721062+0000","last_clean":"2024-03-27T06:42:20.721062+0000","last_became_active":"2024-03-27T06:41:29.401789+0000","last_became_peered":"2024-03-27T06:41:29.401789+0000","last_unstale":"2024-03-27T06:42:20.721062+0000","last_undegraded":"2024-03-27T06:42:20.721062+0000","last_fullsized":"2024-03-27T06:42:20.721062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:31:27.821966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:47.554363+0000","last_change":"2024-03-27T06:41:28.395978+0000","last_active":"2024-03-27T06:42:47.554363+0000","last_peered":"2024-03-27T06:42:47.554363+0000","last_clean":"2024-03-27T06:42:47.554363+0000","last_became_active":"2024-03-27T06:41:28.395650+0000","last_became_peered":"2024-03-27T06:41:28.395650+0000","last_unstale":"2024-03-27T06:42:47.554363+0000","last_undegraded":"2024-03-27T06:42:47.554363+0000","last_fullsized":"2024-03-27T06:42:47.554363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:01:15.841344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658935+0000","last_change":"2024-03-27T06:41:29.423303+0000","last_active":"2024-03-27T06:42:20.658935+0000","last_peered":"2024-03-27T06:42:20.658935+0000","last_clean":"2024-03-27T06:42:20.658935+0000","last_became_active":"2024-03-27T06:41:29.422977+0000","last_became_peered":"2024-03-27T06:41:29.422977+0000","last_unstale":"2024-03-27T06:42:20.658935+0000","last_undegraded":"2024-03-27T06:42:20.658935+0000","last_fullsized":"2024-03-27T06:42:20.658935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:14:01.877801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.004415+0000","last_change":"2024-03-27T06:41:28.395141+0000","last_active":"2024-03-27T06:42:21.004415+0000","last_peered":"2024-03-27T06:42:21.004415+0000","last_clean":"2024-03-27T06:42:21.004415+0000","last_became_active":"2024-03-27T06:41:28.394826+0000","last_became_peered":"2024-03-27T06:41:28.394826+0000","last_unstale":"2024-03-27T06:42:21.004415+0000","last_undegraded":"2024-03-27T06:42:21.004415+0000","last_fullsized":"2024-03-27T06:42:21.004415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:29:49.997090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653806+0000","last_change":"2024-03-27T06:40:16.460939+0000","last_active":"2024-03-27T06:42:20.653806+0000","last_peered":"2024-03-27T06:42:20.653806+0000","last_clean":"2024-03-27T06:42:20.653806+0000","last_became_active":"2024-03-27T06:40:16.460453+0000","last_became_peered":"2024-03-27T06:40:16.460453+0000","last_unstale":"2024-03-27T06:42:20.653806+0000","last_undegraded":"2024-03-27T06:42:20.653806+0000","last_fullsized":"2024-03-27T06:42:20.653806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_clean_scrub_stamp":"2024-03-27T06:36:48.325777+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:28.241688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658690+0000","last_change":"2024-03-27T06:42:20.658690+0000","last_active":"2024-03-27T06:42:20.658690+0000","last_peered":"2024-03-27T06:42:20.658690+0000","last_clean":"2024-03-27T06:42:20.658690+0000","last_became_active":"2024-03-27T06:42:20.658348+0000","last_became_peered":"2024-03-27T06:42:20.658348+0000","last_unstale":"2024-03-27T06:42:20.658690+0000","last_undegraded":"2024-03-27T06:42:20.658690+0000","last_fullsized":"2024-03-27T06:42:20.658690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:12:29.985173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653871+0000","last_change":"2024-03-27T06:41:28.399205+0000","last_active":"2024-03-27T06:42:20.653871+0000","last_peered":"2024-03-27T06:42:20.653871+0000","last_clean":"2024-03-27T06:42:20.653871+0000","last_became_active":"2024-03-27T06:41:28.398889+0000","last_became_peered":"2024-03-27T06:41:28.398889+0000","last_unstale":"2024-03-27T06:42:20.653871+0000","last_undegraded":"2024-03-27T06:42:20.653871+0000","last_fullsized":"2024-03-27T06:42:20.653871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:28:08.769187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.663743+0000","last_change":"2024-03-27T06:42:20.663370+0000","last_active":"2024-03-27T06:42:20.663743+0000","last_peered":"2024-03-27T06:42:20.663743+0000","last_clean":"2024-03-27T06:42:20.663743+0000","last_became_active":"2024-03-27T06:42:20.654243+0000","last_became_peered":"2024-03-27T06:42:20.654243+0000","last_unstale":"2024-03-27T06:42:20.663743+0000","last_undegraded":"2024-03-27T06:42:20.663743+0000","last_fullsized":"2024-03-27T06:42:20.663743+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:43:46.566952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654169+0000","last_change":"2024-03-27T06:41:29.395466+0000","last_active":"2024-03-27T06:42:20.654169+0000","last_peered":"2024-03-27T06:42:20.654169+0000","last_clean":"2024-03-27T06:42:20.654169+0000","last_became_active":"2024-03-27T06:41:29.395094+0000","last_became_peered":"2024-03-27T06:41:29.395094+0000","last_unstale":"2024-03-27T06:42:20.654169+0000","last_undegraded":"2024-03-27T06:42:20.654169+0000","last_fullsized":"2024-03-27T06:42:20.654169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:29:01.746809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650682+0000","last_change":"2024-03-27T06:41:28.400420+0000","last_active":"2024-03-27T06:42:20.650682+0000","last_peered":"2024-03-27T06:42:20.650682+0000","last_clean":"2024-03-27T06:42:20.650682+0000","last_became_active":"2024-03-27T06:41:28.400115+0000","last_became_peered":"2024-03-27T06:41:28.400115+0000","last_unstale":"2024-03-27T06:42:20.650682+0000","last_undegraded":"2024-03-27T06:42:20.650682+0000","last_fullsized":"2024-03-27T06:42:20.650682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:37:40.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657748+0000","last_change":"2024-03-27T06:41:29.411428+0000","last_active":"2024-03-27T06:42:20.657748+0000","last_peered":"2024-03-27T06:42:20.657748+0000","last_clean":"2024-03-27T06:42:20.657748+0000","last_became_active":"2024-03-27T06:41:29.411075+0000","last_became_peered":"2024-03-27T06:41:29.411075+0000","last_unstale":"2024-03-27T06:42:20.657748+0000","last_undegraded":"2024-03-27T06:42:20.657748+0000","last_fullsized":"2024-03-27T06:42:20.657748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:55.764459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.004153+0000","last_change":"2024-03-27T06:41:28.399799+0000","last_active":"2024-03-27T06:42:21.004153+0000","last_peered":"2024-03-27T06:42:21.004153+0000","last_clean":"2024-03-27T06:42:21.004153+0000","last_became_active":"2024-03-27T06:41:28.399385+0000","last_became_peered":"2024-03-27T06:41:28.399385+0000","last_unstale":"2024-03-27T06:42:21.004153+0000","last_undegraded":"2024-03-27T06:42:21.004153+0000","last_fullsized":"2024-03-27T06:42:21.004153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:16:18.005519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653268+0000","last_change":"2024-03-27T06:41:29.418991+0000","last_active":"2024-03-27T06:42:20.653268+0000","last_peered":"2024-03-27T06:42:20.653268+0000","last_clean":"2024-03-27T06:42:20.653268+0000","last_became_active":"2024-03-27T06:41:29.418527+0000","last_became_peered":"2024-03-27T06:41:29.418527+0000","last_unstale":"2024-03-27T06:42:20.653268+0000","last_undegraded":"2024-03-27T06:42:20.653268+0000","last_fullsized":"2024-03-27T06:42:20.653268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:49:55.913276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.655725+0000","last_change":"2024-03-27T06:41:28.397771+0000","last_active":"2024-03-27T06:42:20.655725+0000","last_peered":"2024-03-27T06:42:20.655725+0000","last_clean":"2024-03-27T06:42:20.655725+0000","last_became_active":"2024-03-27T06:41:28.397470+0000","last_became_peered":"2024-03-27T06:41:28.397470+0000","last_unstale":"2024-03-27T06:42:20.655725+0000","last_undegraded":"2024-03-27T06:42:20.655725+0000","last_fullsized":"2024-03-27T06:42:20.655725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:37:21.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.652813+0000","last_change":"2024-03-27T06:41:29.412280+0000","last_active":"2024-03-27T06:42:20.652813+0000","last_peered":"2024-03-27T06:42:20.652813+0000","last_clean":"2024-03-27T06:42:20.652813+0000","last_became_active":"2024-03-27T06:41:29.411336+0000","last_became_peered":"2024-03-27T06:41:29.411336+0000","last_unstale":"2024-03-27T06:42:20.652813+0000","last_undegraded":"2024-03-27T06:42:20.652813+0000","last_fullsized":"2024-03-27T06:42:20.652813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:13:03.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658518+0000","last_change":"2024-03-27T06:41:28.403529+0000","last_active":"2024-03-27T06:42:20.658518+0000","last_peered":"2024-03-27T06:42:20.658518+0000","last_clean":"2024-03-27T06:42:20.658518+0000","last_became_active":"2024-03-27T06:41:28.403155+0000","last_became_peered":"2024-03-27T06:41:28.403155+0000","last_unstale":"2024-03-27T06:42:20.658518+0000","last_undegraded":"2024-03-27T06:42:20.658518+0000","last_fullsized":"2024-03-27T06:42:20.658518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:23:53.971374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650720+0000","last_change":"2024-03-27T06:41:29.404929+0000","last_active":"2024-03-27T06:42:20.650720+0000","last_peered":"2024-03-27T06:42:20.650720+0000","last_clean":"2024-03-27T06:42:20.650720+0000","last_became_active":"2024-03-27T06:41:29.404602+0000","last_became_peered":"2024-03-27T06:41:29.404602+0000","last_unstale":"2024-03-27T06:42:20.650720+0000","last_undegraded":"2024-03-27T06:42:20.650720+0000","last_fullsized":"2024-03-27T06:42:20.650720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:19:07.379046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.652812+0000","last_change":"2024-03-27T06:41:28.398602+0000","last_active":"2024-03-27T06:42:20.652812+0000","last_peered":"2024-03-27T06:42:20.652812+0000","last_clean":"2024-03-27T06:42:20.652812+0000","last_became_active":"2024-03-27T06:41:28.398068+0000","last_became_peered":"2024-03-27T06:41:28.398068+0000","last_unstale":"2024-03-27T06:42:20.652812+0000","last_undegraded":"2024-03-27T06:42:20.652812+0000","last_fullsized":"2024-03-27T06:42:20.652812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:10:53.312044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657304+0000","last_change":"2024-03-27T06:41:29.420245+0000","last_active":"2024-03-27T06:42:20.657304+0000","last_peered":"2024-03-27T06:42:20.657304+0000","last_clean":"2024-03-27T06:42:20.657304+0000","last_became_active":"2024-03-27T06:41:29.419912+0000","last_became_peered":"2024-03-27T06:41:29.419912+0000","last_unstale":"2024-03-27T06:42:20.657304+0000","last_undegraded":"2024-03-27T06:42:20.657304+0000","last_fullsized":"2024-03-27T06:42:20.657304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:46:58.203083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653250+0000","last_change":"2024-03-27T06:41:28.398595+0000","last_active":"2024-03-27T06:42:20.653250+0000","last_peered":"2024-03-27T06:42:20.653250+0000","last_clean":"2024-03-27T06:42:20.653250+0000","last_became_active":"2024-03-27T06:41:28.398036+0000","last_became_peered":"2024-03-27T06:41:28.398036+0000","last_unstale":"2024-03-27T06:42:20.653250+0000","last_undegraded":"2024-03-27T06:42:20.653250+0000","last_fullsized":"2024-03-27T06:42:20.653250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:47:15.769760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654057+0000","last_change":"2024-03-27T06:41:29.406576+0000","last_active":"2024-03-27T06:42:20.654057+0000","last_peered":"2024-03-27T06:42:20.654057+0000","last_clean":"2024-03-27T06:42:20.654057+0000","last_became_active":"2024-03-27T06:41:29.406222+0000","last_became_peered":"2024-03-27T06:41:29.406222+0000","last_unstale":"2024-03-27T06:42:20.654057+0000","last_undegraded":"2024-03-27T06:42:20.654057+0000","last_fullsized":"2024-03-27T06:42:20.654057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:05:12.446154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:45.552696+0000","last_change":"2024-03-27T06:41:28.399361+0000","last_active":"2024-03-27T06:42:45.552696+0000","last_peered":"2024-03-27T06:42:45.552696+0000","last_clean":"2024-03-27T06:42:45.552696+0000","last_became_active":"2024-03-27T06:41:28.398021+0000","last_became_peered":"2024-03-27T06:41:28.398021+0000","last_unstale":"2024-03-27T06:42:45.552696+0000","last_undegraded":"2024-03-27T06:42:45.552696+0000","last_fullsized":"2024-03-27T06:42:45.552696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:51:34.106340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654197+0000","last_change":"2024-03-27T06:41:29.405696+0000","last_active":"2024-03-27T06:42:20.654197+0000","last_peered":"2024-03-27T06:42:20.654197+0000","last_clean":"2024-03-27T06:42:20.654197+0000","last_became_active":"2024-03-27T06:41:29.405273+0000","last_became_peered":"2024-03-27T06:41:29.405273+0000","last_unstale":"2024-03-27T06:42:20.654197+0000","last_undegraded":"2024-03-27T06:42:20.654197+0000","last_fullsized":"2024-03-27T06:42:20.654197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:02:58.778464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.720949+0000","last_change":"2024-03-27T06:41:28.391406+0000","last_active":"2024-03-27T06:42:20.720949+0000","last_peered":"2024-03-27T06:42:20.720949+0000","last_clean":"2024-03-27T06:42:20.720949+0000","last_became_active":"2024-03-27T06:41:28.390950+0000","last_became_peered":"2024-03-27T06:41:28.390950+0000","last_unstale":"2024-03-27T06:42:20.720949+0000","last_undegraded":"2024-03-27T06:42:20.720949+0000","last_fullsized":"2024-03-27T06:42:20.720949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:10:48.347139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653937+0000","last_change":"2024-03-27T06:41:29.410422+0000","last_active":"2024-03-27T06:42:20.653937+0000","last_peered":"2024-03-27T06:42:20.653937+0000","last_clean":"2024-03-27T06:42:20.653937+0000","last_became_active":"2024-03-27T06:41:29.409738+0000","last_became_peered":"2024-03-27T06:41:29.409738+0000","last_unstale":"2024-03-27T06:42:20.653937+0000","last_undegraded":"2024-03-27T06:42:20.653937+0000","last_fullsized":"2024-03-27T06:42:20.653937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:38:17.546504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:21.008586+0000","last_change":"2024-03-27T06:42:21.008586+0000","last_active":"2024-03-27T06:42:21.008586+0000","last_peered":"2024-03-27T06:42:21.008586+0000","last_clean":"2024-03-27T06:42:21.008586+0000","last_became_active":"2024-03-27T06:42:21.004964+0000","last_became_peered":"2024-03-27T06:42:21.004964+0000","last_unstale":"2024-03-27T06:42:21.008586+0000","last_undegraded":"2024-03-27T06:42:21.008586+0000","last_fullsized":"2024-03-27T06:42:21.008586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:11:13.246994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654160+0000","last_change":"2024-03-27T06:41:29.408809+0000","last_active":"2024-03-27T06:42:20.654160+0000","last_peered":"2024-03-27T06:42:20.654160+0000","last_clean":"2024-03-27T06:42:20.654160+0000","last_became_active":"2024-03-27T06:41:29.408447+0000","last_became_peered":"2024-03-27T06:41:29.408447+0000","last_unstale":"2024-03-27T06:42:20.654160+0000","last_undegraded":"2024-03-27T06:42:20.654160+0000","last_fullsized":"2024-03-27T06:42:20.654160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:04:55.269490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.655859+0000","last_change":"2024-03-27T06:42:20.655859+0000","last_active":"2024-03-27T06:42:20.655859+0000","last_peered":"2024-03-27T06:42:20.655859+0000","last_clean":"2024-03-27T06:42:20.655859+0000","last_became_active":"2024-03-27T06:42:20.655446+0000","last_became_peered":"2024-03-27T06:42:20.655446+0000","last_unstale":"2024-03-27T06:42:20.655859+0000","last_undegraded":"2024-03-27T06:42:20.655859+0000","last_fullsized":"2024-03-27T06:42:20.655859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:43:07.920040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653684+0000","last_change":"2024-03-27T06:41:29.409896+0000","last_active":"2024-03-27T06:42:20.653684+0000","last_peered":"2024-03-27T06:42:20.653684+0000","last_clean":"2024-03-27T06:42:20.653684+0000","last_became_active":"2024-03-27T06:41:29.408279+0000","last_became_peered":"2024-03-27T06:41:29.408279+0000","last_unstale":"2024-03-27T06:42:20.653684+0000","last_undegraded":"2024-03-27T06:42:20.653684+0000","last_fullsized":"2024-03-27T06:42:20.653684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:49:03.257543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.652699+0000","last_change":"2024-03-27T06:41:29.412526+0000","last_active":"2024-03-27T06:42:20.652699+0000","last_peered":"2024-03-27T06:42:20.652699+0000","last_clean":"2024-03-27T06:42:20.652699+0000","last_became_active":"2024-03-27T06:41:29.412180+0000","last_became_peered":"2024-03-27T06:41:29.412180+0000","last_unstale":"2024-03-27T06:42:20.652699+0000","last_undegraded":"2024-03-27T06:42:20.652699+0000","last_fullsized":"2024-03-27T06:42:20.652699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:47.461457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657735+0000","last_change":"2024-03-27T06:42:20.657735+0000","last_active":"2024-03-27T06:42:20.657735+0000","last_peered":"2024-03-27T06:42:20.657735+0000","last_clean":"2024-03-27T06:42:20.657735+0000","last_became_active":"2024-03-27T06:42:20.657378+0000","last_became_peered":"2024-03-27T06:42:20.657378+0000","last_unstale":"2024-03-27T06:42:20.657735+0000","last_undegraded":"2024-03-27T06:42:20.657735+0000","last_fullsized":"2024-03-27T06:42:20.657735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:13:56.368229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654275+0000","last_change":"2024-03-27T06:41:29.402442+0000","last_active":"2024-03-27T06:42:20.654275+0000","last_peered":"2024-03-27T06:42:20.654275+0000","last_clean":"2024-03-27T06:42:20.654275+0000","last_became_active":"2024-03-27T06:41:29.402078+0000","last_became_peered":"2024-03-27T06:41:29.402078+0000","last_unstale":"2024-03-27T06:42:20.654275+0000","last_undegraded":"2024-03-27T06:42:20.654275+0000","last_fullsized":"2024-03-27T06:42:20.654275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:07:35.344881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650986+0000","last_change":"2024-03-27T06:41:29.418563+0000","last_active":"2024-03-27T06:42:20.650986+0000","last_peered":"2024-03-27T06:42:20.650986+0000","last_clean":"2024-03-27T06:42:20.650986+0000","last_became_active":"2024-03-27T06:41:29.418209+0000","last_became_peered":"2024-03-27T06:41:29.418209+0000","last_unstale":"2024-03-27T06:42:20.650986+0000","last_undegraded":"2024-03-27T06:42:20.650986+0000","last_fullsized":"2024-03-27T06:42:20.650986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:53:58.536148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.653731+0000","last_change":"2024-03-27T06:41:29.400414+0000","last_active":"2024-03-27T06:42:20.653731+0000","last_peered":"2024-03-27T06:42:20.653731+0000","last_clean":"2024-03-27T06:42:20.653731+0000","last_became_active":"2024-03-27T06:41:29.397865+0000","last_became_peered":"2024-03-27T06:41:29.397865+0000","last_unstale":"2024-03-27T06:42:20.653731+0000","last_undegraded":"2024-03-27T06:42:20.653731+0000","last_fullsized":"2024-03-27T06:42:20.653731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:35:28.405347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.657648+0000","last_change":"2024-03-27T06:41:29.416740+0000","last_active":"2024-03-27T06:42:20.657648+0000","last_peered":"2024-03-27T06:42:20.657648+0000","last_clean":"2024-03-27T06:42:20.657648+0000","last_became_active":"2024-03-27T06:41:29.416417+0000","last_became_peered":"2024-03-27T06:41:29.416417+0000","last_unstale":"2024-03-27T06:42:20.657648+0000","last_undegraded":"2024-03-27T06:42:20.657648+0000","last_fullsized":"2024-03-27T06:42:20.657648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:08:34.186527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.720948+0000","last_change":"2024-03-27T06:41:29.418797+0000","last_active":"2024-03-27T06:42:20.720948+0000","last_peered":"2024-03-27T06:42:20.720948+0000","last_clean":"2024-03-27T06:42:20.720948+0000","last_became_active":"2024-03-27T06:41:29.418472+0000","last_became_peered":"2024-03-27T06:41:29.418472+0000","last_unstale":"2024-03-27T06:42:20.720948+0000","last_undegraded":"2024-03-27T06:42:20.720948+0000","last_fullsized":"2024-03-27T06:42:20.720948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:30:55.540353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.658425+0000","last_change":"2024-03-27T06:41:29.416124+0000","last_active":"2024-03-27T06:42:20.658425+0000","last_peered":"2024-03-27T06:42:20.658425+0000","last_clean":"2024-03-27T06:42:20.658425+0000","last_became_active":"2024-03-27T06:41:29.415665+0000","last_became_peered":"2024-03-27T06:41:29.415665+0000","last_unstale":"2024-03-27T06:42:20.658425+0000","last_undegraded":"2024-03-27T06:42:20.658425+0000","last_fullsized":"2024-03-27T06:42:20.658425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:46:04.160705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.655988+0000","last_change":"2024-03-27T06:41:29.417165+0000","last_active":"2024-03-27T06:42:20.655988+0000","last_peered":"2024-03-27T06:42:20.655988+0000","last_clean":"2024-03-27T06:42:20.655988+0000","last_became_active":"2024-03-27T06:41:29.416856+0000","last_became_peered":"2024-03-27T06:41:29.416856+0000","last_unstale":"2024-03-27T06:42:20.655988+0000","last_undegraded":"2024-03-27T06:42:20.655988+0000","last_fullsized":"2024-03-27T06:42:20.655988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:55:20.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.654515+0000","last_change":"2024-03-27T06:41:29.412377+0000","last_active":"2024-03-27T06:42:20.654515+0000","last_peered":"2024-03-27T06:42:20.654515+0000","last_clean":"2024-03-27T06:42:20.654515+0000","last_became_active":"2024-03-27T06:41:29.411761+0000","last_became_peered":"2024-03-27T06:41:29.411761+0000","last_unstale":"2024-03-27T06:42:20.654515+0000","last_undegraded":"2024-03-27T06:42:20.654515+0000","last_fullsized":"2024-03-27T06:42:20.654515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:16:56.844342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-27T06:42:20.650715+0000","last_change":"2024-03-27T06:41:29.413660+0000","last_active":"2024-03-27T06:42:20.650715+0000","last_peered":"2024-03-27T06:42:20.650715+0000","last_clean":"2024-03-27T06:42:20.650715+0000","last_became_active":"2024-03-27T06:41:29.413201+0000","last_became_peered":"2024-03-27T06:41:29.413201+0000","last_unstale":"2024-03-27T06:42:20.650715+0000","last_undegraded":"2024-03-27T06:42:20.650715+0000","last_fullsized":"2024-03-27T06:42:20.650715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:14:46.446595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":278864,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67227,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":758,"num_read_kb":454,"num_write":238,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":460074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1114112,"data_stored":1111293,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":459780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":459934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1053067,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":459497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1028096,"data_stored":520433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":460836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":521044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":459322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":459749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1052354,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33833,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T06:42:51.666 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-27T06:42:51.666 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-27T06:42:51.666 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-27T06:42:51.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-27T06:42:51.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:51 smithi156 ceph-mon[36261]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:42:51.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:51 smithi170 ceph-mon[36438]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:42:51.916 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:52.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:52 smithi016 ceph-mon[31908]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:42:52.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:52 smithi016 ceph-mon[31908]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-27T06:42:52.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:52 smithi156 ceph-mon[36261]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:42:52.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:52 smithi156 ceph-mon[36261]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-27T06:42:52.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:52 smithi170 ceph-mon[36438]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:42:52.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:52 smithi170 ceph-mon[36438]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-27T06:42:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.627+0000 7f469383a700 1 -- 172.21.15.16:0/871829848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c102fe0 msgr2=0x7f468c10f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.627+0000 7f469383a700 1 --2- 172.21.15.16:0/871829848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c102fe0 0x7f468c10f4f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f468400b0a0 tx=0x7f468400b3b0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.627+0000 7f469383a700 1 -- 172.21.15.16:0/871829848 shutdown_connections 2024-03-27T06:42:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.627+0000 7f469383a700 1 --2- 172.21.15.16:0/871829848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c102fe0 0x7f468c10f4f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.627+0000 7f469383a700 1 --2- 172.21.15.16:0/871829848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c108620 0x7f468c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.627+0000 7f469383a700 1 --2- 172.21.15.16:0/871829848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102620 0x7f468c102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.627+0000 7f469383a700 1 -- 172.21.15.16:0/871829848 >> 172.21.15.16:0/871829848 conn(0x7f468c0fe0f0 msgr2=0x7f468c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.627+0000 7f469383a700 1 -- 172.21.15.16:0/871829848 shutdown_connections 2024-03-27T06:42:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.627+0000 7f469383a700 1 -- 172.21.15.16:0/871829848 wait complete. 2024-03-27T06:42:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.628+0000 7f469383a700 1 Processor -- start 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.628+0000 7f469383a700 1 -- start start 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.628+0000 7f469383a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102620 0x7f468c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.628+0000 7f469383a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c102fe0 0x7f468c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f469383a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c108620 0x7f468c19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f469383a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f468c112a90 con 0x7f468c102fe0 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f469383a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f468c112910 con 0x7f468c108620 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f469383a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f468c112c10 con 0x7f468c102620 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f4690dd5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c102fe0 0x7f468c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f4690dd5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c102fe0 0x7f468c199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43080/0 (socket says 172.21.15.16:43080) 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f4690dd5700 1 -- 172.21.15.16:0/1174452220 learned_addr learned my addr 172.21.15.16:0/1174452220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f46915d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102620 0x7f468c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f4691dd7700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c108620 0x7f468c19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f4690dd5700 1 -- 172.21.15.16:0/1174452220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102620 msgr2=0x7f468c198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f4690dd5700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102620 0x7f468c198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f4690dd5700 1 -- 172.21.15.16:0/1174452220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c108620 msgr2=0x7f468c19d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f4690dd5700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c108620 0x7f468c19d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f4690dd5700 1 -- 172.21.15.16:0/1174452220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c19de50 con 0x7f468c102fe0 2024-03-27T06:42:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f4690dd5700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c102fe0 0x7f468c199380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f467c00e9c0 tx=0x7f467c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f46827fc700 1 -- 172.21.15.16:0/1174452220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c00cbd0 con 0x7f468c102fe0 2024-03-27T06:42:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f46827fc700 1 -- 172.21.15.16:0/1174452220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f467c00ee60 con 0x7f468c102fe0 2024-03-27T06:42:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.629+0000 7f469383a700 1 -- 172.21.15.16:0/1174452220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468c19e140 con 0x7f468c102fe0 2024-03-27T06:42:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.630+0000 7f46827fc700 1 -- 172.21.15.16:0/1174452220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c010430 con 0x7f468c102fe0 2024-03-27T06:42:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.630+0000 7f469383a700 1 -- 172.21.15.16:0/1174452220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468c1a59c0 con 0x7f468c102fe0 2024-03-27T06:42:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.630+0000 7f469383a700 1 -- 172.21.15.16:0/1174452220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f468c069300 con 0x7f468c102fe0 2024-03-27T06:42:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.631+0000 7f46827fc700 1 -- 172.21.15.16:0/1174452220 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f467c004750 con 0x7f468c102fe0 2024-03-27T06:42:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.632+0000 7f46827fc700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f46780a3720 0x7f46780a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.632+0000 7f46915d6700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f46780a3720 0x7f46780a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.632+0000 7f46827fc700 1 -- 172.21.15.16:0/1174452220 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f467c0cff10 con 0x7f468c102fe0 2024-03-27T06:42:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.632+0000 7f46915d6700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f46780a3720 0x7f46780a5be0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4688007b60 tx=0x7f46880073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.637+0000 7f46827fc700 1 -- 172.21.15.16:0/1174452220 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f467c09caf0 con 0x7f468c102fe0 2024-03-27T06:42:53.632 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:53 smithi016 ceph-mon[31908]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:42:53.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:53 smithi156 ceph-mon[36261]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:42:53.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:53 smithi170 ceph-mon[36438]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:42:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.893+0000 7f469383a700 1 -- 172.21.15.16:0/1174452220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f468c1a5c50 con 0x7f468c102fe0 2024-03-27T06:42:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.893+0000 7f46827fc700 1 -- 172.21.15.16:0/1174452220 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f467c09c3f0 con 0x7f468c102fe0 2024-03-27T06:42:53.890 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:42:53.890 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-27T06:42:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 -- 172.21.15.16:0/1174452220 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f46780a3720 msgr2=0x7f46780a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f46780a3720 0x7f46780a5be0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4688007b60 tx=0x7f46880073d0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 -- 172.21.15.16:0/1174452220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c102fe0 msgr2=0x7f468c199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c102fe0 0x7f468c199380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f467c00e9c0 tx=0x7f467c00ecd0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 -- 172.21.15.16:0/1174452220 shutdown_connections 2024-03-27T06:42:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c108620 0x7f468c19d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f46780a3720 0x7f46780a5be0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102620 0x7f468c198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 --2- 172.21.15.16:0/1174452220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c102fe0 0x7f468c199380 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 -- 172.21.15.16:0/1174452220 >> 172.21.15.16:0/1174452220 conn(0x7f468c0fe0f0 msgr2=0x7f468c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 -- 172.21.15.16:0/1174452220 shutdown_connections 2024-03-27T06:42:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:53.895+0000 7f469383a700 1 -- 172.21.15.16:0/1174452220 wait complete. 2024-03-27T06:42:54.563 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:54 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1174452220' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-27T06:42:54.595 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-27T06:42:54.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:42:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.710+0000 7f0447fe2640 1 -- 172.21.15.16:0/3017893770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440101580 msgr2=0x7f0440101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.710+0000 7f0447fe2640 1 --2- 172.21.15.16:0/3017893770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440101580 0x7f0440101e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0430009ca0 tx=0x7f043002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:42:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.710+0000 7f0447fe2640 1 -- 172.21.15.16:0/3017893770 shutdown_connections 2024-03-27T06:42:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.710+0000 7f0447fe2640 1 --2- 172.21.15.16:0/3017893770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440101580 0x7f0440101e00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.710+0000 7f0447fe2640 1 --2- 172.21.15.16:0/3017893770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0440100bc0 0x7f0440101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.710+0000 7f0447fe2640 1 --2- 172.21.15.16:0/3017893770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04400ffe60 0x7f0440100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.710+0000 7f0447fe2640 1 -- 172.21.15.16:0/3017893770 >> 172.21.15.16:0/3017893770 conn(0x7f04400fb790 msgr2=0x7f04400fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.710+0000 7f0447fe2640 1 -- 172.21.15.16:0/3017893770 shutdown_connections 2024-03-27T06:42:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.710+0000 7f0447fe2640 1 -- 172.21.15.16:0/3017893770 wait complete. 2024-03-27T06:42:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.710+0000 7f0447fe2640 1 Processor -- start 2024-03-27T06:42:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f0447fe2640 1 -- start start 2024-03-27T06:42:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f0447fe2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04400ffe60 0x7f044019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f0447fe2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440100bc0 0x7f044019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f0447fe2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0440101580 0x7f04401a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f0447fe2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f044010fba0 con 0x7f0440100bc0 2024-03-27T06:42:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f0447fe2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f044010fa20 con 0x7f0440101580 2024-03-27T06:42:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f0447fe2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f044010fd20 con 0x7f04400ffe60 2024-03-27T06:42:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f044557d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440100bc0 0x7f044019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f044557d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440100bc0 0x7f044019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43088/0 (socket says 172.21.15.16:43088) 2024-03-27T06:42:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f044557d640 1 -- 172.21.15.16:0/2500567616 learned_addr learned my addr 172.21.15.16:0/2500567616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f044657f640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0440101580 0x7f04401a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f0445d7e640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04400ffe60 0x7f044019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f044557d640 1 -- 172.21.15.16:0/2500567616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04400ffe60 msgr2=0x7f044019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f044557d640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04400ffe60 0x7f044019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f044557d640 1 -- 172.21.15.16:0/2500567616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0440101580 msgr2=0x7f04401a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f044557d640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0440101580 0x7f04401a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f044557d640 1 -- 172.21.15.16:0/2500567616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04401a1d50 con 0x7f0440100bc0 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f044557d640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440100bc0 0x7f044019a670 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f043400e9c0 tx=0x7f043400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f043effd640 1 -- 172.21.15.16:0/2500567616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043400cd80 con 0x7f0440100bc0 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f043effd640 1 -- 172.21.15.16:0/2500567616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f04340035f0 con 0x7f0440100bc0 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f0447fe2640 1 -- 172.21.15.16:0/2500567616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04401a1fe0 con 0x7f0440100bc0 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f043effd640 1 -- 172.21.15.16:0/2500567616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434010670 con 0x7f0440100bc0 2024-03-27T06:42:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.711+0000 7f0447fe2640 1 -- 172.21.15.16:0/2500567616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04401a24b0 con 0x7f0440100bc0 2024-03-27T06:42:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.712+0000 7f0447fe2640 1 -- 172.21.15.16:0/2500567616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f044010ba40 con 0x7f0440100bc0 2024-03-27T06:42:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.713+0000 7f043effd640 1 -- 172.21.15.16:0/2500567616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f04340037a0 con 0x7f0440100bc0 2024-03-27T06:42:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.714+0000 7f043effd640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f04280a3610 0x7f04280a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.714+0000 7f0445d7e640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f04280a3610 0x7f04280a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.714+0000 7f043effd640 1 -- 172.21.15.16:0/2500567616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f04340d0090 con 0x7f0440100bc0 2024-03-27T06:42:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.714+0000 7f0445d7e640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f04280a3610 0x7f04280a5ad0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0438009bd0 tx=0x7f0438005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.718+0000 7f043effd640 1 -- 172.21.15.16:0/2500567616 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f043409cc10 con 0x7f0440100bc0 2024-03-27T06:42:54.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:54 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1174452220' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-27T06:42:54.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:54 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1174452220' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-27T06:42:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.904+0000 7f0447fe2640 1 -- 172.21.15.16:0/2500567616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0440062750 con 0x7f0440100bc0 2024-03-27T06:42:54.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.906+0000 7f043effd640 1 -- 172.21.15.16:0/2500567616 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7f043409c510 con 0x7f0440100bc0 2024-03-27T06:42:54.902 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:42:54.902 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:42:20.647546+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:42:54.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 -- 172.21.15.16:0/2500567616 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f04280a3610 msgr2=0x7f04280a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:54.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f04280a3610 0x7f04280a5ad0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0438009bd0 tx=0x7f0438005f90 comp rx=0 tx=0).stop 2024-03-27T06:42:54.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 -- 172.21.15.16:0/2500567616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440100bc0 msgr2=0x7f044019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440100bc0 0x7f044019a670 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f043400e9c0 tx=0x7f043400ee80 comp rx=0 tx=0).stop 2024-03-27T06:42:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 -- 172.21.15.16:0/2500567616 shutdown_connections 2024-03-27T06:42:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f04280a3610 0x7f04280a5ad0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0440101580 0x7f04401a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440100bc0 0x7f044019a670 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 --2- 172.21.15.16:0/2500567616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04400ffe60 0x7f044019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 -- 172.21.15.16:0/2500567616 >> 172.21.15.16:0/2500567616 conn(0x7f04400fb790 msgr2=0x7f04400fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 -- 172.21.15.16:0/2500567616 shutdown_connections 2024-03-27T06:42:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:54.908+0000 7f0447fe2640 1 -- 172.21.15.16:0/2500567616 wait complete. 2024-03-27T06:42:54.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T06:41:26.393978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T06:41:27.707469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:42:54.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T06:42:55.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.025+0000 7f60a3f5c640 1 -- 172.21.15.16:0/1346997535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609c100a70 msgr2=0x7f609c100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.025+0000 7f60a3f5c640 1 --2- 172.21.15.16:0/1346997535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609c100a70 0x7f609c100e90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6094009860 tx=0x7f609402bb70 comp rx=0 tx=0).stop 2024-03-27T06:42:55.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.025+0000 7f60a3f5c640 1 -- 172.21.15.16:0/1346997535 shutdown_connections 2024-03-27T06:42:55.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.025+0000 7f60a3f5c640 1 --2- 172.21.15.16:0/1346997535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609c102620 0x7f609c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.025+0000 7f60a3f5c640 1 --2- 172.21.15.16:0/1346997535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f609c101c60 0x7f609c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.025+0000 7f60a3f5c640 1 --2- 172.21.15.16:0/1346997535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609c100a70 0x7f609c100e90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.025+0000 7f60a3f5c640 1 -- 172.21.15.16:0/1346997535 >> 172.21.15.16:0/1346997535 conn(0x7f609c0fc000 msgr2=0x7f609c0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.025+0000 7f60a3f5c640 1 -- 172.21.15.16:0/1346997535 shutdown_connections 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.025+0000 7f60a3f5c640 1 -- 172.21.15.16:0/1346997535 wait complete. 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a3f5c640 1 Processor -- start 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a3f5c640 1 -- start start 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a3f5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609c100a70 0x7f609c104a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a3f5c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f609c101c60 0x7f609c103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a3f5c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609c102620 0x7f609c103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a3f5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f609c10c810 con 0x7f609c100a70 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a3f5c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f609c10c690 con 0x7f609c101c60 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a3f5c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f609c10c990 con 0x7f609c102620 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a1cf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609c100a70 0x7f609c104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a1cf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609c100a70 0x7f609c104a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43104/0 (socket says 172.21.15.16:43104) 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a1cf8640 1 -- 172.21.15.16:0/2448890047 learned_addr learned my addr 172.21.15.16:0/2448890047 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a24f9640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609c102620 0x7f609c103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.026+0000 7f60a14f7640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f609c101c60 0x7f609c103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.027+0000 7f60a1cf8640 1 -- 172.21.15.16:0/2448890047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609c102620 msgr2=0x7f609c103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.027+0000 7f60a1cf8640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609c102620 0x7f609c103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.027+0000 7f60a1cf8640 1 -- 172.21.15.16:0/2448890047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f609c101c60 msgr2=0x7f609c103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.027+0000 7f60a1cf8640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f609c101c60 0x7f609c103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.027+0000 7f60a1cf8640 1 -- 172.21.15.16:0/2448890047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c103e10 con 0x7f609c100a70 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.027+0000 7f60a1cf8640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609c100a70 0x7f609c104a00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6094034a80 tx=0x7f6094035f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.027+0000 7f609affd640 1 -- 172.21.15.16:0/2448890047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094041020 con 0x7f609c100a70 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.027+0000 7f60a3f5c640 1 -- 172.21.15.16:0/2448890047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609c1a20a0 con 0x7f609c100a70 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.027+0000 7f609affd640 1 -- 172.21.15.16:0/2448890047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f609402ca10 con 0x7f609c100a70 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.027+0000 7f609affd640 1 -- 172.21.15.16:0/2448890047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094039040 con 0x7f609c100a70 2024-03-27T06:42:55.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.027+0000 7f60a3f5c640 1 -- 172.21.15.16:0/2448890047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609c1a25f0 con 0x7f609c100a70 2024-03-27T06:42:55.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.028+0000 7f60a3f5c640 1 -- 172.21.15.16:0/2448890047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f609c1a2850 con 0x7f609c100a70 2024-03-27T06:42:55.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.029+0000 7f609affd640 1 -- 172.21.15.16:0/2448890047 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f6094044020 con 0x7f609c100a70 2024-03-27T06:42:55.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.030+0000 7f609affd640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f60840a3610 0x7f60840a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.030+0000 7f60a14f7640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f60840a3610 0x7f60840a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.030+0000 7f609affd640 1 -- 172.21.15.16:0/2448890047 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f60940f1590 con 0x7f609c100a70 2024-03-27T06:42:55.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.030+0000 7f60a14f7640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f60840a3610 0x7f60840a5ad0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f609c104770 tx=0x7f6090005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.034+0000 7f609affd640 1 -- 172.21.15.16:0/2448890047 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f609404b020 con 0x7f609c100a70 2024-03-27T06:42:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.218+0000 7f60a3f5c640 1 -- 172.21.15.16:0/2448890047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f609c1a2b40 con 0x7f609c100a70 2024-03-27T06:42:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.219+0000 7f609affd640 1 -- 172.21.15.16:0/2448890047 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f60940be110 con 0x7f609c100a70 2024-03-27T06:42:55.215 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 -- 172.21.15.16:0/2448890047 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f60840a3610 msgr2=0x7f60840a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f60840a3610 0x7f60840a5ad0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f609c104770 tx=0x7f6090005f90 comp rx=0 tx=0).stop 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 -- 172.21.15.16:0/2448890047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609c100a70 msgr2=0x7f609c104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609c100a70 0x7f609c104a00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6094034a80 tx=0x7f6094035f10 comp rx=0 tx=0).stop 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 -- 172.21.15.16:0/2448890047 shutdown_connections 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f60840a3610 0x7f60840a5ad0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f609c102620 0x7f609c103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f609c101c60 0x7f609c103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 --2- 172.21.15.16:0/2448890047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609c100a70 0x7f609c104a00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 -- 172.21.15.16:0/2448890047 >> 172.21.15.16:0/2448890047 conn(0x7f609c0fc000 msgr2=0x7f609c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 -- 172.21.15.16:0/2448890047 shutdown_connections 2024-03-27T06:42:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.221+0000 7f60a3f5c640 1 -- 172.21.15.16:0/2448890047 wait complete. 2024-03-27T06:42:55.228 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T06:42:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.338+0000 7f4a7853a640 1 -- 172.21.15.16:0/213798548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a70104fb0 msgr2=0x7f4a701073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.338+0000 7f4a7853a640 1 --2- 172.21.15.16:0/213798548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a70104fb0 0x7f4a701073a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009900 tx=0x7f4a6002bc30 comp rx=0 tx=0).stop 2024-03-27T06:42:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.338+0000 7f4a7853a640 1 -- 172.21.15.16:0/213798548 shutdown_connections 2024-03-27T06:42:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.338+0000 7f4a7853a640 1 --2- 172.21.15.16:0/213798548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a701078e0 0x7f4a70109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.338+0000 7f4a7853a640 1 --2- 172.21.15.16:0/213798548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a70104fb0 0x7f4a701073a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.338+0000 7f4a7853a640 1 --2- 172.21.15.16:0/213798548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a70102680 0x7f4a70104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.338+0000 7f4a7853a640 1 -- 172.21.15.16:0/213798548 >> 172.21.15.16:0/213798548 conn(0x7f4a700fbff0 msgr2=0x7f4a700fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.338+0000 7f4a7853a640 1 -- 172.21.15.16:0/213798548 shutdown_connections 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.338+0000 7f4a7853a640 1 -- 172.21.15.16:0/213798548 wait complete. 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a7853a640 1 Processor -- start 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a7853a640 1 -- start start 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a7853a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a70102680 0x7f4a70197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a7853a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a70104fb0 0x7f4a70198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a7853a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a701078e0 0x7f4a7019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a7853a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010d8e0 con 0x7f4a70102680 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a7853a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010d760 con 0x7f4a70104fb0 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a7853a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010da60 con 0x7f4a701078e0 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a762d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a70102680 0x7f4a70197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a762d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a70102680 0x7f4a70197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43122/0 (socket says 172.21.15.16:43122) 2024-03-27T06:42:55.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a762d6640 1 -- 172.21.15.16:0/2508037074 learned_addr learned my addr 172.21.15.16:0/2508037074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a76ad7640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a701078e0 0x7f4a7019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a75ad5640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a70104fb0 0x7f4a70198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a762d6640 1 -- 172.21.15.16:0/2508037074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a701078e0 msgr2=0x7f4a7019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a762d6640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a701078e0 0x7f4a7019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a762d6640 1 -- 172.21.15.16:0/2508037074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a70104fb0 msgr2=0x7f4a70198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a762d6640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a70104fb0 0x7f4a70198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a762d6640 1 -- 172.21.15.16:0/2508037074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7019fc10 con 0x7f4a70102680 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a762d6640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a70102680 0x7f4a70197ff0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00e9f0 tx=0x7f4a6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a677fe640 1 -- 172.21.15.16:0/2508037074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00cd80 con 0x7f4a70102680 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a677fe640 1 -- 172.21.15.16:0/2508037074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4a6c005520 con 0x7f4a70102680 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.339+0000 7f4a677fe640 1 -- 172.21.15.16:0/2508037074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c010690 con 0x7f4a70102680 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.340+0000 7f4a7853a640 1 -- 172.21.15.16:0/2508037074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7019ff00 con 0x7f4a70102680 2024-03-27T06:42:55.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.340+0000 7f4a7853a640 1 -- 172.21.15.16:0/2508037074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a701a0360 con 0x7f4a70102680 2024-03-27T06:42:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.341+0000 7f4a7853a640 1 -- 172.21.15.16:0/2508037074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a700fdae0 con 0x7f4a70102680 2024-03-27T06:42:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.341+0000 7f4a677fe640 1 -- 172.21.15.16:0/2508037074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f4a6c0046c0 con 0x7f4a70102680 2024-03-27T06:42:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.342+0000 7f4a677fe640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a5c0a3690 0x7f4a5c0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.342+0000 7f4a75ad5640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a5c0a3690 0x7f4a5c0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.342+0000 7f4a677fe640 1 -- 172.21.15.16:0/2508037074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4a6c0cffd0 con 0x7f4a70102680 2024-03-27T06:42:55.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.342+0000 7f4a75ad5640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a5c0a3690 0x7f4a5c0a5b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4a60000c00 tx=0x7f4a60005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.347+0000 7f4a677fe640 1 -- 172.21.15.16:0/2508037074 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f4a6c09cc50 con 0x7f4a70102680 2024-03-27T06:42:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.532+0000 7f4a7853a640 1 -- 172.21.15.16:0/2508037074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4a7002d540 con 0x7f4a70102680 2024-03-27T06:42:55.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.532+0000 7f4a677fe640 1 -- 172.21.15.16:0/2508037074 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f4a6c09c550 con 0x7f4a70102680 2024-03-27T06:42:55.528 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-03-27T06:42:55.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 -- 172.21.15.16:0/2508037074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a5c0a3690 msgr2=0x7f4a5c0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a5c0a3690 0x7f4a5c0a5b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4a60000c00 tx=0x7f4a60005fd0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 -- 172.21.15.16:0/2508037074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a70102680 msgr2=0x7f4a70197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a70102680 0x7f4a70197ff0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00e9f0 tx=0x7f4a6c00edf0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 -- 172.21.15.16:0/2508037074 shutdown_connections 2024-03-27T06:42:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f4a5c0a3690 0x7f4a5c0a5b50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a701078e0 0x7f4a7019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a70104fb0 0x7f4a70198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 --2- 172.21.15.16:0/2508037074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a70102680 0x7f4a70197ff0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 -- 172.21.15.16:0/2508037074 >> 172.21.15.16:0/2508037074 conn(0x7f4a700fbff0 msgr2=0x7f4a70103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 -- 172.21.15.16:0/2508037074 shutdown_connections 2024-03-27T06:42:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.534+0000 7f4a7853a640 1 -- 172.21.15.16:0/2508037074 wait complete. 2024-03-27T06:42:55.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:55 smithi016 ceph-mon[31908]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-27T06:42:55.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:55 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2500567616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:42:55.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:55 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2448890047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:42:55.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T06:42:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 -- 172.21.15.16:0/2414949131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d8101c50 msgr2=0x7f10d81020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 --2- 172.21.15.16:0/2414949131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d8101c50 0x7f10d81020d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009900 tx=0x7f10cc02bc30 comp rx=0 tx=0).stop 2024-03-27T06:42:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 -- 172.21.15.16:0/2414949131 shutdown_connections 2024-03-27T06:42:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 --2- 172.21.15.16:0/2414949131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10d8102610 0x7f10d8108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 --2- 172.21.15.16:0/2414949131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d8101c50 0x7f10d81020d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 --2- 172.21.15.16:0/2414949131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8100a60 0x7f10d8100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 -- 172.21.15.16:0/2414949131 >> 172.21.15.16:0/2414949131 conn(0x7f10d80fbff0 msgr2=0x7f10d80fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 -- 172.21.15.16:0/2414949131 shutdown_connections 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 -- 172.21.15.16:0/2414949131 wait complete. 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 Processor -- start 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 -- start start 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8100a60 0x7f10d806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d8101c50 0x7f10d819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10d8102610 0x7f10d819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f10d810c7c0 con 0x7f10d8101c50 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f10d810c640 con 0x7f10d8100a60 2024-03-27T06:42:55.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.652+0000 7f10dd993640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f10d810c940 con 0x7f10d8102610 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d8101c50 0x7f10d819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8100a60 0x7f10d806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d8101c50 0x7f10d819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43152/0 (socket says 172.21.15.16:43152) 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d67fc640 1 -- 172.21.15.16:0/2510468363 learned_addr learned my addr 172.21.15.16:0/2510468363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d77fe640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10d8102610 0x7f10d819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d6ffd640 1 -- 172.21.15.16:0/2510468363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10d8102610 msgr2=0x7f10d819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d6ffd640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10d8102610 0x7f10d819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d6ffd640 1 -- 172.21.15.16:0/2510468363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d8101c50 msgr2=0x7f10d819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d6ffd640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d8101c50 0x7f10d819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d6ffd640 1 -- 172.21.15.16:0/2510468363 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d819fba0 con 0x7f10d8100a60 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d67fc640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d8101c50 0x7f10d819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10d6ffd640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8100a60 0x7f10d806ac90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10c800e9c0 tx=0x7f10c800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10dc991640 1 -- 172.21.15.16:0/2510468363 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c800ccc0 con 0x7f10d8100a60 2024-03-27T06:42:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10dd993640 1 -- 172.21.15.16:0/2510468363 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d819fe30 con 0x7f10d8100a60 2024-03-27T06:42:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10dc991640 1 -- 172.21.15.16:0/2510468363 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f10c80035f0 con 0x7f10d8100a60 2024-03-27T06:42:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10dc991640 1 -- 172.21.15.16:0/2510468363 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c800ce60 con 0x7f10d8100a60 2024-03-27T06:42:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.653+0000 7f10dd993640 1 -- 172.21.15.16:0/2510468363 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d81a02a0 con 0x7f10d8100a60 2024-03-27T06:42:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.654+0000 7f10dd993640 1 -- 172.21.15.16:0/2510468363 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d8107e00 con 0x7f10d8100a60 2024-03-27T06:42:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.656+0000 7f10dc991640 1 -- 172.21.15.16:0/2510468363 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f10c80037a0 con 0x7f10d8100a60 2024-03-27T06:42:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.656+0000 7f10dc991640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f10c00a3610 0x7f10c00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.656+0000 7f10d67fc640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f10c00a3610 0x7f10c00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.656+0000 7f10dc991640 1 -- 172.21.15.16:0/2510468363 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f10c80d03e0 con 0x7f10d8100a60 2024-03-27T06:42:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.657+0000 7f10d67fc640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f10c00a3610 0x7f10c00a5ad0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f10cc000c00 tx=0x7f10cc02c9d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:55.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.664+0000 7f10dc991640 1 -- 172.21.15.16:0/2510468363 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f10c809cfc0 con 0x7f10d8100a60 2024-03-27T06:42:55.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:55 smithi156 ceph-mon[36261]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-27T06:42:55.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:55 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2500567616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:42:55.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:55 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2448890047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:42:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.847+0000 7f10dd993640 1 -- 172.21.15.16:0/2510468363 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10d802d540 con 0x7f10d8100a60 2024-03-27T06:42:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.847+0000 7f10dc991640 1 -- 172.21.15.16:0/2510468363 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f10c809c8c0 con 0x7f10d8100a60 2024-03-27T06:42:55.843 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-03-27T06:42:55.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:55 smithi170 ceph-mon[36438]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-27T06:42:55.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:55 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2500567616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:42:55.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:55 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2448890047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:42:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 -- 172.21.15.16:0/2510468363 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f10c00a3610 msgr2=0x7f10c00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f10c00a3610 0x7f10c00a5ad0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f10cc000c00 tx=0x7f10cc02c9d0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 -- 172.21.15.16:0/2510468363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8100a60 msgr2=0x7f10d806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8100a60 0x7f10d806ac90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10c800e9c0 tx=0x7f10c800ee80 comp rx=0 tx=0).stop 2024-03-27T06:42:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 -- 172.21.15.16:0/2510468363 shutdown_connections 2024-03-27T06:42:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f10c00a3610 0x7f10c00a5ad0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10d8102610 0x7f10d819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d8101c50 0x7f10d819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 --2- 172.21.15.16:0/2510468363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10d8100a60 0x7f10d806ac90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 -- 172.21.15.16:0/2510468363 >> 172.21.15.16:0/2510468363 conn(0x7f10d80fbff0 msgr2=0x7f10d8104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 -- 172.21.15.16:0/2510468363 shutdown_connections 2024-03-27T06:42:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.849+0000 7f10dd993640 1 -- 172.21.15.16:0/2510468363 wait complete. 2024-03-27T06:42:55.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:42:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.963+0000 7f539d99e640 1 -- 172.21.15.16:0/3899357446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53981008a0 msgr2=0x7f5398100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.963+0000 7f539d99e640 1 --2- 172.21.15.16:0/3899357446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53981008a0 0x7f5398100cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5390009be0 tx=0x7f539002bef0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.963+0000 7f539d99e640 1 -- 172.21.15.16:0/3899357446 shutdown_connections 2024-03-27T06:42:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.963+0000 7f539d99e640 1 --2- 172.21.15.16:0/3899357446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5398102450 0x7f5398108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.963+0000 7f539d99e640 1 --2- 172.21.15.16:0/3899357446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5398101a90 0x7f5398101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.963+0000 7f539d99e640 1 --2- 172.21.15.16:0/3899357446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53981008a0 0x7f5398100cc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539d99e640 1 -- 172.21.15.16:0/3899357446 >> 172.21.15.16:0/3899357446 conn(0x7f53980fbe30 msgr2=0x7f53980fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539d99e640 1 -- 172.21.15.16:0/3899357446 shutdown_connections 2024-03-27T06:42:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539d99e640 1 -- 172.21.15.16:0/3899357446 wait complete. 2024-03-27T06:42:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539d99e640 1 Processor -- start 2024-03-27T06:42:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539d99e640 1 -- start start 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539d99e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53981008a0 0x7f5398197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539d99e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5398101a90 0x7f53981982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539d99e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5398102450 0x7f539819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539d99e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f539810d760 con 0x7f53981008a0 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539d99e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f539810d5e0 con 0x7f5398101a90 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539d99e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f539810d8e0 con 0x7f5398102450 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539c99c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53981008a0 0x7f5398197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539c99c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53981008a0 0x7f5398197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48452/0 (socket says 172.21.15.16:48452) 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.964+0000 7f539c99c640 1 -- 172.21.15.16:0/2681663174 learned_addr learned my addr 172.21.15.16:0/2681663174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f5397fff640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5398101a90 0x7f53981982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f539d19d640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5398102450 0x7f539819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f539c99c640 1 -- 172.21.15.16:0/2681663174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5398102450 msgr2=0x7f539819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f539c99c640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5398102450 0x7f539819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f539c99c640 1 -- 172.21.15.16:0/2681663174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5398101a90 msgr2=0x7f53981982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f539c99c640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5398101a90 0x7f53981982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f539c99c640 1 -- 172.21.15.16:0/2681663174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539819f980 con 0x7f53981008a0 2024-03-27T06:42:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f539c99c640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53981008a0 0x7f5398197d60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5390000f80 tx=0x7f53900338a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f5395ffb640 1 -- 172.21.15.16:0/2681663174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390043020 con 0x7f53981008a0 2024-03-27T06:42:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f5395ffb640 1 -- 172.21.15.16:0/2681663174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f539002ca80 con 0x7f53981008a0 2024-03-27T06:42:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f539d99e640 1 -- 172.21.15.16:0/2681663174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539819fc10 con 0x7f53981008a0 2024-03-27T06:42:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f5395ffb640 1 -- 172.21.15.16:0/2681663174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539003e740 con 0x7f53981008a0 2024-03-27T06:42:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.965+0000 7f539d99e640 1 -- 172.21.15.16:0/2681663174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53981a0160 con 0x7f53981008a0 2024-03-27T06:42:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.966+0000 7f539d99e640 1 -- 172.21.15.16:0/2681663174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5398107b60 con 0x7f53981008a0 2024-03-27T06:42:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.967+0000 7f5395ffb640 1 -- 172.21.15.16:0/2681663174 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5390046020 con 0x7f53981008a0 2024-03-27T06:42:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.968+0000 7f5395ffb640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f53800a3610 0x7f53800a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.968+0000 7f5397fff640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f53800a3610 0x7f53800a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.968+0000 7f5395ffb640 1 -- 172.21.15.16:0/2681663174 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f53900f2410 con 0x7f53981008a0 2024-03-27T06:42:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.969+0000 7f5397fff640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f53800a3610 0x7f53800a5ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5398199270 tx=0x7f538c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:55.973+0000 7f5395ffb640 1 -- 172.21.15.16:0/2681663174 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f539004d020 con 0x7f53981008a0 2024-03-27T06:42:56.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.199+0000 7f539d99e640 1 -- 172.21.15.16:0/2681663174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5398062750 con 0x7f53981008a0 2024-03-27T06:42:56.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.200+0000 7f5395ffb640 1 -- 172.21.15.16:0/2681663174 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7f53900beff0 con 0x7f53981008a0 2024-03-27T06:42:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:42:56.196 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:41:35.466274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24472,24493,24500,24527,24467]},"id":1}]} 2024-03-27T06:42:56.196 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-03-27T06:42:56.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 -- 172.21.15.16:0/2681663174 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f53800a3610 msgr2=0x7f53800a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f53800a3610 0x7f53800a5ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5398199270 tx=0x7f538c009260 comp rx=0 tx=0).stop 2024-03-27T06:42:56.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 -- 172.21.15.16:0/2681663174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53981008a0 msgr2=0x7f5398197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53981008a0 0x7f5398197d60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5390000f80 tx=0x7f53900338a0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 -- 172.21.15.16:0/2681663174 shutdown_connections 2024-03-27T06:42:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f53800a3610 0x7f53800a5ad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5398102450 0x7f539819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5398101a90 0x7f53981982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 --2- 172.21.15.16:0/2681663174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53981008a0 0x7f5398197d60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 -- 172.21.15.16:0/2681663174 >> 172.21.15.16:0/2681663174 conn(0x7f53980fbe30 msgr2=0x7f5398104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 -- 172.21.15.16:0/2681663174 shutdown_connections 2024-03-27T06:42:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.202+0000 7f539d99e640 1 -- 172.21.15.16:0/2681663174 wait complete. 2024-03-27T06:42:56.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:42:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 -- 172.21.15.16:0/3735421296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7101026a0 msgr2=0x7fc710108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 --2- 172.21.15.16:0/3735421296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7101026a0 0x7fc710108f30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc700009ca0 tx=0x7fc70002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 -- 172.21.15.16:0/3735421296 shutdown_connections 2024-03-27T06:42:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 --2- 172.21.15.16:0/3735421296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7101026a0 0x7fc710108f30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 --2- 172.21.15.16:0/3735421296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc710101ce0 0x7fc710102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 --2- 172.21.15.16:0/3735421296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc710100af0 0x7fc710100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 -- 172.21.15.16:0/3735421296 >> 172.21.15.16:0/3735421296 conn(0x7fc7100fc080 msgr2=0x7fc7100fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 -- 172.21.15.16:0/3735421296 shutdown_connections 2024-03-27T06:42:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 -- 172.21.15.16:0/3735421296 wait complete. 2024-03-27T06:42:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 Processor -- start 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 -- start start 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.317+0000 7fc71797a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc710100af0 0x7fc710196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc71797a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710101ce0 0x7fc710197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc71797a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7101026a0 0x7fc71019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc71797a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010c820 con 0x7fc710101ce0 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc71797a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010c6a0 con 0x7fc710100af0 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc71797a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010c9a0 con 0x7fc7101026a0 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc714f15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710101ce0 0x7fc710197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc714f15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710101ce0 0x7fc710197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48464/0 (socket says 172.21.15.16:48464) 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc714f15640 1 -- 172.21.15.16:0/1307884822 learned_addr learned my addr 172.21.15.16:0/1307884822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc715716640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc710100af0 0x7fc710196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc715f17640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7101026a0 0x7fc71019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc714f15640 1 -- 172.21.15.16:0/1307884822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7101026a0 msgr2=0x7fc71019e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc714f15640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7101026a0 0x7fc71019e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc714f15640 1 -- 172.21.15.16:0/1307884822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc710100af0 msgr2=0x7fc710196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc714f15640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc710100af0 0x7fc710196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc714f15640 1 -- 172.21.15.16:0/1307884822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71019ea00 con 0x7fc710101ce0 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc714f15640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710101ce0 0x7fc710197320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc70400ea60 tx=0x7fc70400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc70e7fc640 1 -- 172.21.15.16:0/1307884822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70400cde0 con 0x7fc710101ce0 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc70e7fc640 1 -- 172.21.15.16:0/1307884822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc7040035f0 con 0x7fc710101ce0 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc71797a640 1 -- 172.21.15.16:0/1307884822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71019ecf0 con 0x7fc710101ce0 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc70e7fc640 1 -- 172.21.15.16:0/1307884822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704010720 con 0x7fc710101ce0 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.318+0000 7fc71797a640 1 -- 172.21.15.16:0/1307884822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71019f230 con 0x7fc710101ce0 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.319+0000 7fc71797a640 1 -- 172.21.15.16:0/1307884822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc710107d40 con 0x7fc710101ce0 2024-03-27T06:42:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.320+0000 7fc70e7fc640 1 -- 172.21.15.16:0/1307884822 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc7040037a0 con 0x7fc710101ce0 2024-03-27T06:42:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.321+0000 7fc70e7fc640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc6f80a3690 0x7fc6f80a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.321+0000 7fc715716640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc6f80a3690 0x7fc6f80a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.321+0000 7fc70e7fc640 1 -- 172.21.15.16:0/1307884822 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc7040cfff0 con 0x7fc710101ce0 2024-03-27T06:42:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.321+0000 7fc715716640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc6f80a3690 0x7fc6f80a5b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc710101b40 tx=0x7fc708005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:56.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.325+0000 7fc70e7fc640 1 -- 172.21.15.16:0/1307884822 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc70409cbd0 con 0x7fc710101ce0 2024-03-27T06:42:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.516+0000 7fc71797a640 1 -- 172.21.15.16:0/1307884822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7100627f0 con 0x7fc710101ce0 2024-03-27T06:42:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.517+0000 7fc70e7fc640 1 -- 172.21.15.16:0/1307884822 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7fc70409c4d0 con 0x7fc710101ce0 2024-03-27T06:42:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:42:56.513 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:42:20.647546+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:42:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 -- 172.21.15.16:0/1307884822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc6f80a3690 msgr2=0x7fc6f80a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc6f80a3690 0x7fc6f80a5b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc710101b40 tx=0x7fc708005f90 comp rx=0 tx=0).stop 2024-03-27T06:42:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 -- 172.21.15.16:0/1307884822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710101ce0 msgr2=0x7fc710197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710101ce0 0x7fc710197320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc70400ea60 tx=0x7fc70400ef20 comp rx=0 tx=0).stop 2024-03-27T06:42:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 -- 172.21.15.16:0/1307884822 shutdown_connections 2024-03-27T06:42:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc6f80a3690 0x7fc6f80a5b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7101026a0 0x7fc71019e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710101ce0 0x7fc710197320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 --2- 172.21.15.16:0/1307884822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc710100af0 0x7fc710196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 -- 172.21.15.16:0/1307884822 >> 172.21.15.16:0/1307884822 conn(0x7fc7100fc080 msgr2=0x7fc710104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 -- 172.21.15.16:0/1307884822 shutdown_connections 2024-03-27T06:42:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.520+0000 7fc71797a640 1 -- 172.21.15.16:0/1307884822 wait complete. 2024-03-27T06:42:56.525 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:56 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2508037074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:42:56.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:56 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2510468363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:42:56.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:56 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2681663174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:42:56.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T06:41:26.393978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T06:41:27.707469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:42:56.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T06:42:56.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.638+0000 7faf194d4640 1 -- 172.21.15.16:0/940537673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14107830 msgr2=0x7faf14109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.638+0000 7faf194d4640 1 --2- 172.21.15.16:0/940537673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14107830 0x7faf14109c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faf04009ca0 tx=0x7faf0402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.638+0000 7faf194d4640 1 -- 172.21.15.16:0/940537673 shutdown_connections 2024-03-27T06:42:56.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.638+0000 7faf194d4640 1 --2- 172.21.15.16:0/940537673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14107830 0x7faf14109c80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.638+0000 7faf194d4640 1 --2- 172.21.15.16:0/940537673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf14104f00 0x7faf141072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.638+0000 7faf194d4640 1 --2- 172.21.15.16:0/940537673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf141025d0 0x7faf141049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.638+0000 7faf194d4640 1 -- 172.21.15.16:0/940537673 >> 172.21.15.16:0/940537673 conn(0x7faf140fbfe0 msgr2=0x7faf140fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:56.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.638+0000 7faf194d4640 1 -- 172.21.15.16:0/940537673 shutdown_connections 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.638+0000 7faf194d4640 1 -- 172.21.15.16:0/940537673 wait complete. 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf194d4640 1 Processor -- start 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf194d4640 1 -- start start 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf194d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf141025d0 0x7faf1419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf194d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf14104f00 0x7faf1419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf194d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf14107830 0x7faf141a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf194d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf1410d920 con 0x7faf141025d0 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf194d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7faf1410d7a0 con 0x7faf14107830 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf194d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faf1410daa0 con 0x7faf14104f00 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf141025d0 0x7faf1419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf141025d0 0x7faf1419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48482/0 (socket says 172.21.15.16:48482) 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf12ffd640 1 -- 172.21.15.16:0/4101908252 learned_addr learned my addr 172.21.15.16:0/4101908252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf127fc640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf14104f00 0x7faf1419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.639+0000 7faf137fe640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf14107830 0x7faf141a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.640+0000 7faf12ffd640 1 -- 172.21.15.16:0/4101908252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf14104f00 msgr2=0x7faf1419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.640+0000 7faf12ffd640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf14104f00 0x7faf1419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.640+0000 7faf12ffd640 1 -- 172.21.15.16:0/4101908252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf14107830 msgr2=0x7faf141a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.640+0000 7faf12ffd640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf14107830 0x7faf141a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.640+0000 7faf12ffd640 1 -- 172.21.15.16:0/4101908252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf141a1d90 con 0x7faf141025d0 2024-03-27T06:42:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.640+0000 7faf12ffd640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf141025d0 0x7faf1419a170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00e9f0 tx=0x7faf0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.640+0000 7faf0bfff640 1 -- 172.21.15.16:0/4101908252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00cd80 con 0x7faf141025d0 2024-03-27T06:42:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.640+0000 7faf0bfff640 1 -- 172.21.15.16:0/4101908252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faf0c010930 con 0x7faf141025d0 2024-03-27T06:42:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.640+0000 7faf194d4640 1 -- 172.21.15.16:0/4101908252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf141a2080 con 0x7faf141025d0 2024-03-27T06:42:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.640+0000 7faf0bfff640 1 -- 172.21.15.16:0/4101908252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c018b00 con 0x7faf141025d0 2024-03-27T06:42:56.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.640+0000 7faf194d4640 1 -- 172.21.15.16:0/4101908252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141a2620 con 0x7faf141025d0 2024-03-27T06:42:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.641+0000 7faf194d4640 1 -- 172.21.15.16:0/4101908252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf14109320 con 0x7faf141025d0 2024-03-27T06:42:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.642+0000 7faf0bfff640 1 -- 172.21.15.16:0/4101908252 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7faf0c018ca0 con 0x7faf141025d0 2024-03-27T06:42:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.642+0000 7faf0bfff640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf000a3690 0x7faf000a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.643+0000 7faf127fc640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf000a3690 0x7faf000a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.643+0000 7faf0bfff640 1 -- 172.21.15.16:0/4101908252 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7faf0c0d03b0 con 0x7faf141025d0 2024-03-27T06:42:56.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.643+0000 7faf127fc640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf000a3690 0x7faf000a5b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf1419b680 tx=0x7faefc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:56.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.646+0000 7faf0bfff640 1 -- 172.21.15.16:0/4101908252 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7faf0c09c8e0 con 0x7faf141025d0 2024-03-27T06:42:56.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:56 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2508037074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:42:56.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:56 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2510468363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:42:56.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:56 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2681663174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:42:56.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.830+0000 7faf194d4640 1 -- 172.21.15.16:0/4101908252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf14062750 con 0x7faf141025d0 2024-03-27T06:42:56.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.831+0000 7faf0bfff640 1 -- 172.21.15.16:0/4101908252 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7faf0c0a0be0 con 0x7faf141025d0 2024-03-27T06:42:56.826 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:42:56.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 -- 172.21.15.16:0/4101908252 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf000a3690 msgr2=0x7faf000a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf000a3690 0x7faf000a5b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf1419b680 tx=0x7faefc005f90 comp rx=0 tx=0).stop 2024-03-27T06:42:56.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 -- 172.21.15.16:0/4101908252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf141025d0 msgr2=0x7faf1419a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf141025d0 0x7faf1419a170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00e9f0 tx=0x7faf0c00eeb0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 -- 172.21.15.16:0/4101908252 shutdown_connections 2024-03-27T06:42:56.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf000a3690 0x7faf000a5b50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf14107830 0x7faf141a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf14104f00 0x7faf1419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 --2- 172.21.15.16:0/4101908252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf141025d0 0x7faf1419a170 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 -- 172.21.15.16:0/4101908252 >> 172.21.15.16:0/4101908252 conn(0x7faf140fbfe0 msgr2=0x7faf14101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:56.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 -- 172.21.15.16:0/4101908252 shutdown_connections 2024-03-27T06:42:56.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.833+0000 7faf194d4640 1 -- 172.21.15.16:0/4101908252 wait complete. 2024-03-27T06:42:56.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T06:42:56.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:56 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2508037074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:42:56.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:56 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2510468363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:42:56.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:56 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2681663174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:42:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.951+0000 7f077a0dd640 1 -- 172.21.15.16:0/1024865041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0774102610 msgr2=0x7f0774108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.951+0000 7f077a0dd640 1 --2- 172.21.15.16:0/1024865041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0774102610 0x7f0774108ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f076c009ca0 tx=0x7f076c02c0f0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.951+0000 7f077a0dd640 1 -- 172.21.15.16:0/1024865041 shutdown_connections 2024-03-27T06:42:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.951+0000 7f077a0dd640 1 --2- 172.21.15.16:0/1024865041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0774102610 0x7f0774108ea0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.951+0000 7f077a0dd640 1 --2- 172.21.15.16:0/1024865041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774101c50 0x7f07741020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.951+0000 7f077a0dd640 1 --2- 172.21.15.16:0/1024865041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0774100a60 0x7f0774100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.951+0000 7f077a0dd640 1 -- 172.21.15.16:0/1024865041 >> 172.21.15.16:0/1024865041 conn(0x7f07740fbff0 msgr2=0x7f07740fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.951+0000 7f077a0dd640 1 -- 172.21.15.16:0/1024865041 shutdown_connections 2024-03-27T06:42:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.951+0000 7f077a0dd640 1 -- 172.21.15.16:0/1024865041 wait complete. 2024-03-27T06:42:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.951+0000 7f077a0dd640 1 Processor -- start 2024-03-27T06:42:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f077a0dd640 1 -- start start 2024-03-27T06:42:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f077a0dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774100a60 0x7f077406acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f077a0dd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0774101c50 0x7f077419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f077a0dd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0774102610 0x7f077419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f077a0dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f077410c810 con 0x7f0774100a60 2024-03-27T06:42:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f077a0dd640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f077410c690 con 0x7f0774101c50 2024-03-27T06:42:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f077a0dd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f077410c990 con 0x7f0774102610 2024-03-27T06:42:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774100a60 0x7f077406acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774100a60 0x7f077406acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48502/0 (socket says 172.21.15.16:48502) 2024-03-27T06:42:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f07737fe640 1 -- 172.21.15.16:0/3770177309 learned_addr learned my addr 172.21.15.16:0/3770177309 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f0772ffd640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0774101c50 0x7f077419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f0773fff640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0774102610 0x7f077419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f07737fe640 1 -- 172.21.15.16:0/3770177309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0774102610 msgr2=0x7f077419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f07737fe640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0774102610 0x7f077419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f07737fe640 1 -- 172.21.15.16:0/3770177309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0774101c50 msgr2=0x7f077419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f07737fe640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0774101c50 0x7f077419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f07737fe640 1 -- 172.21.15.16:0/3770177309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077419fb30 con 0x7f0774100a60 2024-03-27T06:42:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.952+0000 7f07737fe640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774100a60 0x7f077406acf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9c0 tx=0x7f076400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.953+0000 7f0770ff9640 1 -- 172.21.15.16:0/3770177309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076400cd80 con 0x7f0774100a60 2024-03-27T06:42:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.953+0000 7f077a0dd640 1 -- 172.21.15.16:0/3770177309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077419fe20 con 0x7f0774100a60 2024-03-27T06:42:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.953+0000 7f0770ff9640 1 -- 172.21.15.16:0/3770177309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07640035f0 con 0x7f0774100a60 2024-03-27T06:42:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.953+0000 7f0770ff9640 1 -- 172.21.15.16:0/3770177309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764010690 con 0x7f0774100a60 2024-03-27T06:42:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.953+0000 7f077a0dd640 1 -- 172.21.15.16:0/3770177309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07741a02e0 con 0x7f0774100a60 2024-03-27T06:42:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.954+0000 7f077a0dd640 1 -- 172.21.15.16:0/3770177309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0774107d80 con 0x7f0774100a60 2024-03-27T06:42:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.955+0000 7f0770ff9640 1 -- 172.21.15.16:0/3770177309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f07640037a0 con 0x7f0774100a60 2024-03-27T06:42:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.956+0000 7f0770ff9640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f075c0a3610 0x7f075c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.956+0000 7f0772ffd640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f075c0a3610 0x7f075c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.956+0000 7f0770ff9640 1 -- 172.21.15.16:0/3770177309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f07640d01b0 con 0x7f0774100a60 2024-03-27T06:42:56.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.956+0000 7f0772ffd640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f075c0a3610 0x7f075c0a5ad0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f077406aa60 tx=0x7f0768005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:56.960+0000 7f0770ff9640 1 -- 172.21.15.16:0/3770177309 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f076409cd90 con 0x7f0774100a60 2024-03-27T06:42:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.144+0000 7f077a0dd640 1 -- 172.21.15.16:0/3770177309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f07740627f0 con 0x7f0774100a60 2024-03-27T06:42:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.144+0000 7f0770ff9640 1 -- 172.21.15.16:0/3770177309 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f076409c690 con 0x7f0774100a60 2024-03-27T06:42:57.140 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-03-27T06:42:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 -- 172.21.15.16:0/3770177309 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f075c0a3610 msgr2=0x7f075c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f075c0a3610 0x7f075c0a5ad0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f077406aa60 tx=0x7f0768005f90 comp rx=0 tx=0).stop 2024-03-27T06:42:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 -- 172.21.15.16:0/3770177309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774100a60 msgr2=0x7f077406acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774100a60 0x7f077406acf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9c0 tx=0x7f076400ee80 comp rx=0 tx=0).stop 2024-03-27T06:42:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 -- 172.21.15.16:0/3770177309 shutdown_connections 2024-03-27T06:42:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f075c0a3610 0x7f075c0a5ad0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0774102610 0x7f077419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0774101c50 0x7f077419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 --2- 172.21.15.16:0/3770177309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774100a60 0x7f077406acf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 -- 172.21.15.16:0/3770177309 >> 172.21.15.16:0/3770177309 conn(0x7f07740fbff0 msgr2=0x7f0774104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 -- 172.21.15.16:0/3770177309 shutdown_connections 2024-03-27T06:42:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.146+0000 7f077a0dd640 1 -- 172.21.15.16:0/3770177309 wait complete. 2024-03-27T06:42:57.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T06:42:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.263+0000 7f61f5eb5640 1 -- 172.21.15.16:0/3708156769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f0102d50 msgr2=0x7f61f01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.263+0000 7f61f5eb5640 1 --2- 172.21.15.16:0/3708156769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f0102d50 0x7f61f01031d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009900 tx=0x7f61e402bc30 comp rx=0 tx=0).stop 2024-03-27T06:42:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.263+0000 7f61f5eb5640 1 -- 172.21.15.16:0/3708156769 shutdown_connections 2024-03-27T06:42:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.263+0000 7f61f5eb5640 1 --2- 172.21.15.16:0/3708156769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61f0103710 0x7f61f0109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.263+0000 7f61f5eb5640 1 --2- 172.21.15.16:0/3708156769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f0102d50 0x7f61f01031d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.263+0000 7f61f5eb5640 1 --2- 172.21.15.16:0/3708156769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61f0101b60 0x7f61f0101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.263+0000 7f61f5eb5640 1 -- 172.21.15.16:0/3708156769 >> 172.21.15.16:0/3708156769 conn(0x7f61f00fd0f0 msgr2=0x7f61f00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.263+0000 7f61f5eb5640 1 -- 172.21.15.16:0/3708156769 shutdown_connections 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.263+0000 7f61f5eb5640 1 -- 172.21.15.16:0/3708156769 wait complete. 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61f5eb5640 1 Processor -- start 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61f5eb5640 1 -- start start 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61f5eb5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61f0101b60 0x7f61f0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61f5eb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f0102d50 0x7f61f01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61f5eb5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61f0103710 0x7f61f019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61f5eb5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61f010d8c0 con 0x7f61f0102d50 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61f5eb5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f61f010d740 con 0x7f61f0103710 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61f5eb5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f61f010da40 con 0x7f61f0101b60 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f0102d50 0x7f61f01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f0102d50 0x7f61f01984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48516/0 (socket says 172.21.15.16:48516) 2024-03-27T06:42:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61eeffd640 1 -- 172.21.15.16:0/1934240989 learned_addr learned my addr 172.21.15.16:0/1934240989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61ef7fe640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61f0101b60 0x7f61f0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61effff640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61f0103710 0x7f61f019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61eeffd640 1 -- 172.21.15.16:0/1934240989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61f0101b60 msgr2=0x7f61f0197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61eeffd640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61f0101b60 0x7f61f0197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61eeffd640 1 -- 172.21.15.16:0/1934240989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61f0103710 msgr2=0x7f61f019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61eeffd640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61f0103710 0x7f61f019f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.264+0000 7f61eeffd640 1 -- 172.21.15.16:0/1934240989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f019fbc0 con 0x7f61f0102d50 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.265+0000 7f61eeffd640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f0102d50 0x7f61f01984e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009230 tx=0x7f61e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.265+0000 7f61ecff9640 1 -- 172.21.15.16:0/1934240989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4043020 con 0x7f61f0102d50 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.265+0000 7f61ecff9640 1 -- 172.21.15.16:0/1934240989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f61e4004250 con 0x7f61f0102d50 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.265+0000 7f61f5eb5640 1 -- 172.21.15.16:0/1934240989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f019fe50 con 0x7f61f0102d50 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.265+0000 7f61ecff9640 1 -- 172.21.15.16:0/1934240989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4003a80 con 0x7f61f0102d50 2024-03-27T06:42:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.265+0000 7f61f5eb5640 1 -- 172.21.15.16:0/1934240989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f01a0310 con 0x7f61f0102d50 2024-03-27T06:42:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.266+0000 7f61f5eb5640 1 -- 172.21.15.16:0/1934240989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f0108ea0 con 0x7f61f0102d50 2024-03-27T06:42:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.267+0000 7f61ecff9640 1 -- 172.21.15.16:0/1934240989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f61e4046020 con 0x7f61f0102d50 2024-03-27T06:42:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.267+0000 7f61ecff9640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f61d80a3610 0x7f61d80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.267+0000 7f61ef7fe640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f61d80a3610 0x7f61d80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.267+0000 7f61ecff9640 1 -- 172.21.15.16:0/1934240989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f61e40f1dd0 con 0x7f61f0102d50 2024-03-27T06:42:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.268+0000 7f61ef7fe640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f61d80a3610 0x7f61d80a5ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f61f0102bb0 tx=0x7f61e0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.272+0000 7f61ecff9640 1 -- 172.21.15.16:0/1934240989 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f61e404d020 con 0x7f61f0102d50 2024-03-27T06:42:57.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.459+0000 7f61f5eb5640 1 -- 172.21.15.16:0/1934240989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f61f0062780 con 0x7f61f0102d50 2024-03-27T06:42:57.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.459+0000 7f61ecff9640 1 -- 172.21.15.16:0/1934240989 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f61e40f30a0 con 0x7f61f0102d50 2024-03-27T06:42:57.455 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-03-27T06:42:57.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 -- 172.21.15.16:0/1934240989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f61d80a3610 msgr2=0x7f61d80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f61d80a3610 0x7f61d80a5ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f61f0102bb0 tx=0x7f61e0005f90 comp rx=0 tx=0).stop 2024-03-27T06:42:57.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 -- 172.21.15.16:0/1934240989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f0102d50 msgr2=0x7f61f01984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f0102d50 0x7f61f01984e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009230 tx=0x7f61e4004ab0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 -- 172.21.15.16:0/1934240989 shutdown_connections 2024-03-27T06:42:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f61d80a3610 0x7f61d80a5ad0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f61f0103710 0x7f61f019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61f0102d50 0x7f61f01984e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 --2- 172.21.15.16:0/1934240989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61f0101b60 0x7f61f0197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 -- 172.21.15.16:0/1934240989 >> 172.21.15.16:0/1934240989 conn(0x7f61f00fd0f0 msgr2=0x7f61f0106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 -- 172.21.15.16:0/1934240989 shutdown_connections 2024-03-27T06:42:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.461+0000 7f61f5eb5640 1 -- 172.21.15.16:0/1934240989 wait complete. 2024-03-27T06:42:57.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:42:57.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.575+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3632670581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9bc1078b0 msgr2=0x7fd9bc109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.575+0000 7fd9c39bc640 1 --2- 172.21.15.16:0/3632670581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9bc1078b0 0x7fd9bc109ca0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009ca0 tx=0x7fd9ac02c0f0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.575+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3632670581 shutdown_connections 2024-03-27T06:42:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.575+0000 7fd9c39bc640 1 --2- 172.21.15.16:0/3632670581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9bc1078b0 0x7fd9bc109ca0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.575+0000 7fd9c39bc640 1 --2- 172.21.15.16:0/3632670581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9bc104f80 0x7fd9bc107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.575+0000 7fd9c39bc640 1 --2- 172.21.15.16:0/3632670581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc102650 0x7fd9bc104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.575+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3632670581 >> 172.21.15.16:0/3632670581 conn(0x7fd9bc0fbfe0 msgr2=0x7fd9bc0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.575+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3632670581 shutdown_connections 2024-03-27T06:42:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.575+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3632670581 wait complete. 2024-03-27T06:42:57.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c39bc640 1 Processor -- start 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c39bc640 1 -- start start 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c39bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9bc102650 0x7fd9bc197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c39bc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9bc104f80 0x7fd9bc198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c39bc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc1078b0 0x7fd9bc19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c39bc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc10d940 con 0x7fd9bc102650 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c39bc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc10d7c0 con 0x7fd9bc1078b0 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c39bc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc10dac0 con 0x7fd9bc104f80 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c1758640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9bc102650 0x7fd9bc197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c1758640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9bc102650 0x7fd9bc197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48532/0 (socket says 172.21.15.16:48532) 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c1758640 1 -- 172.21.15.16:0/3211961864 learned_addr learned my addr 172.21.15.16:0/3211961864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c1f59640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc1078b0 0x7fd9bc19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c0f57640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9bc104f80 0x7fd9bc198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c1758640 1 -- 172.21.15.16:0/3211961864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9bc104f80 msgr2=0x7fd9bc198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c1758640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9bc104f80 0x7fd9bc198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c1758640 1 -- 172.21.15.16:0/3211961864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc1078b0 msgr2=0x7fd9bc19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.576+0000 7fd9c1758640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc1078b0 0x7fd9bc19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.577+0000 7fd9c1758640 1 -- 172.21.15.16:0/3211961864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc19fb20 con 0x7fd9bc102650 2024-03-27T06:42:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.577+0000 7fd9c1758640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9bc102650 0x7fd9bc197f00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400e9f0 tx=0x7fd9b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.577+0000 7fd9ba7fc640 1 -- 172.21.15.16:0/3211961864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400cd80 con 0x7fd9bc102650 2024-03-27T06:42:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.577+0000 7fd9ba7fc640 1 -- 172.21.15.16:0/3211961864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd9b4010930 con 0x7fd9bc102650 2024-03-27T06:42:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.577+0000 7fd9ba7fc640 1 -- 172.21.15.16:0/3211961864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4018b00 con 0x7fd9bc102650 2024-03-27T06:42:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.577+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3211961864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9bc19fe10 con 0x7fd9bc102650 2024-03-27T06:42:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.577+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3211961864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9bc1a0270 con 0x7fd9bc102650 2024-03-27T06:42:57.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.578+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3211961864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9bc0fdad0 con 0x7fd9bc102650 2024-03-27T06:42:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.579+0000 7fd9ba7fc640 1 -- 172.21.15.16:0/3211961864 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fd9b4018ca0 con 0x7fd9bc102650 2024-03-27T06:42:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.579+0000 7fd9ba7fc640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd9a40a3610 0x7fd9a40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.579+0000 7fd9ba7fc640 1 -- 172.21.15.16:0/3211961864 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd9b40cffd0 con 0x7fd9bc102650 2024-03-27T06:42:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.579+0000 7fd9c0f57640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd9a40a3610 0x7fd9a40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.580+0000 7fd9c0f57640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd9a40a3610 0x7fd9a40a5ad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc199410 tx=0x7fd9b0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.584+0000 7fd9ba7fc640 1 -- 172.21.15.16:0/3211961864 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd9b409c600 con 0x7fd9bc102650 2024-03-27T06:42:57.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:57 smithi156 ceph-mon[36261]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T06:42:57.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:57 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1307884822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:42:57.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:57 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/4101908252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:42:57.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:57 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3770177309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:42:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.808+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3211961864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9bc02d5c0 con 0x7fd9bc102650 2024-03-27T06:42:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.808+0000 7fd9ba7fc640 1 -- 172.21.15.16:0/3211961864 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7fd9b40a0900 con 0x7fd9bc102650 2024-03-27T06:42:57.804 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:42:57.804 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:41:35.466274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24472,24493,24500,24527,24467]},"id":1}]} 2024-03-27T06:42:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-03-27T06:42:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3211961864 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd9a40a3610 msgr2=0x7fd9a40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd9a40a3610 0x7fd9a40a5ad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc199410 tx=0x7fd9b0005f90 comp rx=0 tx=0).stop 2024-03-27T06:42:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3211961864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9bc102650 msgr2=0x7fd9bc197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9bc102650 0x7fd9bc197f00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400e9f0 tx=0x7fd9b400eeb0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3211961864 shutdown_connections 2024-03-27T06:42:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd9a40a3610 0x7fd9a40a5ad0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9bc1078b0 0x7fd9bc19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9bc104f80 0x7fd9bc198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 --2- 172.21.15.16:0/3211961864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9bc102650 0x7fd9bc197f00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3211961864 >> 172.21.15.16:0/3211961864 conn(0x7fd9bc0fbfe0 msgr2=0x7fd9bc102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3211961864 shutdown_connections 2024-03-27T06:42:57.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.810+0000 7fd9c39bc640 1 -- 172.21.15.16:0/3211961864 wait complete. 2024-03-27T06:42:57.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:57 smithi016 ceph-mon[31908]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T06:42:57.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:57 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1307884822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:42:57.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:57 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/4101908252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:42:57.816 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:57 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3770177309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:42:57.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:42:57.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:57 smithi170 ceph-mon[36438]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T06:42:57.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:57 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1307884822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:42:57.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:57 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/4101908252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:42:57.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:57 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3770177309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:42:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.924+0000 7fc19d081640 1 -- 172.21.15.16:0/3300489146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc198100af0 msgr2=0x7fc198100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.924+0000 7fc19d081640 1 --2- 172.21.15.16:0/3300489146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc198100af0 0x7fc198100f10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc188009900 tx=0x7fc18802bc30 comp rx=0 tx=0).stop 2024-03-27T06:42:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.924+0000 7fc19d081640 1 -- 172.21.15.16:0/3300489146 shutdown_connections 2024-03-27T06:42:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.924+0000 7fc19d081640 1 --2- 172.21.15.16:0/3300489146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1981026a0 0x7fc198108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.924+0000 7fc19d081640 1 --2- 172.21.15.16:0/3300489146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc198101ce0 0x7fc198102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.924+0000 7fc19d081640 1 --2- 172.21.15.16:0/3300489146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc198100af0 0x7fc198100f10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.924+0000 7fc19d081640 1 -- 172.21.15.16:0/3300489146 >> 172.21.15.16:0/3300489146 conn(0x7fc1980fc080 msgr2=0x7fc1980fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.924+0000 7fc19d081640 1 -- 172.21.15.16:0/3300489146 shutdown_connections 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.924+0000 7fc19d081640 1 -- 172.21.15.16:0/3300489146 wait complete. 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.924+0000 7fc19d081640 1 Processor -- start 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19d081640 1 -- start start 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19d081640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc198100af0 0x7fc19806ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19d081640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc198101ce0 0x7fc19819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19d081640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1981026a0 0x7fc19819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19d081640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc19810c7e0 con 0x7fc1981026a0 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19d081640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fc19810c660 con 0x7fc198100af0 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19d081640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc19810c960 con 0x7fc198101ce0 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1981026a0 0x7fc19819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1981026a0 0x7fc19819f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48540/0 (socket says 172.21.15.16:48540) 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19759e640 1 -- 172.21.15.16:0/2738691936 learned_addr learned my addr 172.21.15.16:0/2738691936 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19659c640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc198101ce0 0x7fc19819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc196d9d640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc198100af0 0x7fc19806ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19759e640 1 -- 172.21.15.16:0/2738691936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc198101ce0 msgr2=0x7fc19819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19759e640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc198101ce0 0x7fc19819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19759e640 1 -- 172.21.15.16:0/2738691936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc198100af0 msgr2=0x7fc19806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19759e640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc198100af0 0x7fc19806ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19759e640 1 -- 172.21.15.16:0/2738691936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19819fbb0 con 0x7fc1981026a0 2024-03-27T06:42:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc19759e640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1981026a0 0x7fc19819f2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc19000eb10 tx=0x7fc19000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.925+0000 7fc17ffff640 1 -- 172.21.15.16:0/2738691936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190004b20 con 0x7fc1981026a0 2024-03-27T06:42:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.926+0000 7fc19d081640 1 -- 172.21.15.16:0/2738691936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19819fea0 con 0x7fc1981026a0 2024-03-27T06:42:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.926+0000 7fc17ffff640 1 -- 172.21.15.16:0/2738691936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc190004cc0 con 0x7fc1981026a0 2024-03-27T06:42:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.926+0000 7fc17ffff640 1 -- 172.21.15.16:0/2738691936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1900107f0 con 0x7fc1981026a0 2024-03-27T06:42:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.926+0000 7fc19d081640 1 -- 172.21.15.16:0/2738691936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1981a0370 con 0x7fc1981026a0 2024-03-27T06:42:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.927+0000 7fc19d081640 1 -- 172.21.15.16:0/2738691936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc198107e10 con 0x7fc1981026a0 2024-03-27T06:42:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.928+0000 7fc17ffff640 1 -- 172.21.15.16:0/2738691936 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc190005020 con 0x7fc1981026a0 2024-03-27T06:42:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.928+0000 7fc17ffff640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc1800a3610 0x7fc1800a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.928+0000 7fc196d9d640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc1800a3610 0x7fc1800a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.928+0000 7fc17ffff640 1 -- 172.21.15.16:0/2738691936 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc1900d0260 con 0x7fc1981026a0 2024-03-27T06:42:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.929+0000 7fc196d9d640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc1800a3610 0x7fc1800a5ad0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc188005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:57.933+0000 7fc17ffff640 1 -- 172.21.15.16:0/2738691936 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc19009ce40 con 0x7fc1981026a0 2024-03-27T06:42:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.120+0000 7fc19d081640 1 -- 172.21.15.16:0/2738691936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc198062750 con 0x7fc1981026a0 2024-03-27T06:42:58.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.122+0000 7fc17ffff640 1 -- 172.21.15.16:0/2738691936 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7fc19009c740 con 0x7fc1981026a0 2024-03-27T06:42:58.118 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:42:58.118 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:42:20.647546+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:42:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 -- 172.21.15.16:0/2738691936 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc1800a3610 msgr2=0x7fc1800a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc1800a3610 0x7fc1800a5ad0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc188005fd0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 -- 172.21.15.16:0/2738691936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1981026a0 msgr2=0x7fc19819f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1981026a0 0x7fc19819f2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc19000eb10 tx=0x7fc19000c2d0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 -- 172.21.15.16:0/2738691936 shutdown_connections 2024-03-27T06:42:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc1800a3610 0x7fc1800a5ad0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1981026a0 0x7fc19819f2f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc198101ce0 0x7fc19819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 --2- 172.21.15.16:0/2738691936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc198100af0 0x7fc19806ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 -- 172.21.15.16:0/2738691936 >> 172.21.15.16:0/2738691936 conn(0x7fc1980fc080 msgr2=0x7fc198104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 -- 172.21.15.16:0/2738691936 shutdown_connections 2024-03-27T06:42:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.124+0000 7fc19d081640 1 -- 172.21.15.16:0/2738691936 wait complete. 2024-03-27T06:42:58.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:42:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 -- 172.21.15.16:0/4232737420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774102650 msgr2=0x7f9774104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 --2- 172.21.15.16:0/4232737420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774102650 0x7f9774104a40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f976c009900 tx=0x7f976c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:42:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 -- 172.21.15.16:0/4232737420 shutdown_connections 2024-03-27T06:42:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 --2- 172.21.15.16:0/4232737420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97741078b0 0x7f9774109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 --2- 172.21.15.16:0/4232737420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9774104f80 0x7f9774107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 --2- 172.21.15.16:0/4232737420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774102650 0x7f9774104a40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 -- 172.21.15.16:0/4232737420 >> 172.21.15.16:0/4232737420 conn(0x7f97740fbfe0 msgr2=0x7f97740fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 -- 172.21.15.16:0/4232737420 shutdown_connections 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 -- 172.21.15.16:0/4232737420 wait complete. 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 Processor -- start 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 -- start start 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9774102650 0x7f9774101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774104f80 0x7f9774100600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.239+0000 7f977b90c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97741078b0 0x7f9774100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f977b90c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f977410d8d0 con 0x7f9774104f80 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f977b90c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f977410d750 con 0x7f97741078b0 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f977b90c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f977410da50 con 0x7f9774102650 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f9778ea7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774104f80 0x7f9774100600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f9778ea7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774104f80 0x7f9774100600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48572/0 (socket says 172.21.15.16:48572) 2024-03-27T06:42:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f9778ea7640 1 -- 172.21.15.16:0/4211711721 learned_addr learned my addr 172.21.15.16:0/4211711721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f9779ea9640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97741078b0 0x7f9774100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f97796a8640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9774102650 0x7f9774101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f9778ea7640 1 -- 172.21.15.16:0/4211711721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9774102650 msgr2=0x7f9774101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f9778ea7640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9774102650 0x7f9774101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f9778ea7640 1 -- 172.21.15.16:0/4211711721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97741078b0 msgr2=0x7f9774100b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f9778ea7640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97741078b0 0x7f9774100b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f9778ea7640 1 -- 172.21.15.16:0/4211711721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97741013d0 con 0x7f9774104f80 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f9779ea9640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97741078b0 0x7f9774100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f9778ea7640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774104f80 0x7f9774100600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f976800e9a0 tx=0x7f976800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f97727fc640 1 -- 172.21.15.16:0/4211711721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976800cd80 con 0x7f9774104f80 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f977b90c640 1 -- 172.21.15.16:0/4211711721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97741a2060 con 0x7f9774104f80 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f97727fc640 1 -- 172.21.15.16:0/4211711721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f97680035f0 con 0x7f9774104f80 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f97727fc640 1 -- 172.21.15.16:0/4211711721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768010700 con 0x7f9774104f80 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.240+0000 7f977b90c640 1 -- 172.21.15.16:0/4211711721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97741a2600 con 0x7f9774104f80 2024-03-27T06:42:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.241+0000 7f977b90c640 1 -- 172.21.15.16:0/4211711721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97740fdad0 con 0x7f9774104f80 2024-03-27T06:42:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.242+0000 7f97727fc640 1 -- 172.21.15.16:0/4211711721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f97680037a0 con 0x7f9774104f80 2024-03-27T06:42:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.243+0000 7f97727fc640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f975c0a3610 0x7f975c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.243+0000 7f97796a8640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f975c0a3610 0x7f975c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.243+0000 7f97727fc640 1 -- 172.21.15.16:0/4211711721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f97680d00c0 con 0x7f9774104f80 2024-03-27T06:42:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.244+0000 7f97796a8640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f975c0a3610 0x7f975c0a5ad0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f976c02c900 tx=0x7f976c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.248+0000 7f97727fc640 1 -- 172.21.15.16:0/4211711721 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f976809cc40 con 0x7f9774104f80 2024-03-27T06:42:58.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.475+0000 7f977b90c640 1 -- 172.21.15.16:0/4211711721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9774062750 con 0x7f9774104f80 2024-03-27T06:42:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.475+0000 7f97727fc640 1 -- 172.21.15.16:0/4211711721 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7f976809c540 con 0x7f9774104f80 2024-03-27T06:42:58.471 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:42:58.471 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:41:35.466274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24472,24493,24500,24527,24467]},"id":1}]} 2024-03-27T06:42:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-03-27T06:42:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 -- 172.21.15.16:0/4211711721 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f975c0a3610 msgr2=0x7f975c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f975c0a3610 0x7f975c0a5ad0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f976c02c900 tx=0x7f976c005fd0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 -- 172.21.15.16:0/4211711721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774104f80 msgr2=0x7f9774100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774104f80 0x7f9774100600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f976800e9a0 tx=0x7f976800ee60 comp rx=0 tx=0).stop 2024-03-27T06:42:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 -- 172.21.15.16:0/4211711721 shutdown_connections 2024-03-27T06:42:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f975c0a3610 0x7f975c0a5ad0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97741078b0 0x7f9774100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774104f80 0x7f9774100600 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 --2- 172.21.15.16:0/4211711721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9774102650 0x7f9774101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 -- 172.21.15.16:0/4211711721 >> 172.21.15.16:0/4211711721 conn(0x7f97740fbfe0 msgr2=0x7f9774102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 -- 172.21.15.16:0/4211711721 shutdown_connections 2024-03-27T06:42:58.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.478+0000 7f977b90c640 1 -- 172.21.15.16:0/4211711721 wait complete. 2024-03-27T06:42:58.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-27T06:41:28.746131+0000', 'modified': '2024-03-27T06:41:35.466274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24500, 'mds_2': 24493, 'mds_3': 24472, 'mds_4': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/420981926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 420981926}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 420981926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/2365316099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 2365316099}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 2365316099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/1368007304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 1368007304}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 1368007304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6833/521338694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6832', 'nonce': 521338694}, {'type': 'v1', 'addr': '172.21.15.156:6833', 'nonce': 521338694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/4279306199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 4279306199}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 4279306199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.156:6835/4256913032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 4256913032}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 4256913032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.170:6837/2408489138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 2408489138}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2408489138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.170:6839/246078270', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 246078270}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 246078270}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.156:6837/3016592835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 3016592835}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 3016592835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6839/479168397', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 479168397}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 479168397}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24472, 24493, 24500, 24527, 24467]} 2024-03-27T06:42:58.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-27T06:42:58.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-27T06:42:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.593+0000 7f3dc7e69640 1 -- 172.21.15.16:0/2091298326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0102600 msgr2=0x7f3dc0108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.593+0000 7f3dc7e69640 1 --2- 172.21.15.16:0/2091298326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0102600 0x7f3dc0108e90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009ca0 tx=0x7f3db002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 -- 172.21.15.16:0/2091298326 shutdown_connections 2024-03-27T06:42:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 --2- 172.21.15.16:0/2091298326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0102600 0x7f3dc0108e90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 --2- 172.21.15.16:0/2091298326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc0101c40 0x7f3dc01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 --2- 172.21.15.16:0/2091298326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dc0100a50 0x7f3dc0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 -- 172.21.15.16:0/2091298326 >> 172.21.15.16:0/2091298326 conn(0x7f3dc00fbfe0 msgr2=0x7f3dc00fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 -- 172.21.15.16:0/2091298326 shutdown_connections 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 -- 172.21.15.16:0/2091298326 wait complete. 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 Processor -- start 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 -- start start 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dc0100a50 0x7f3dc0196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc0101c40 0x7f3dc0197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0102600 0x7f3dc019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc010c800 con 0x7f3dc0102600 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc010c680 con 0x7f3dc0100a50 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.594+0000 7f3dc7e69640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc010c980 con 0x7f3dc0101c40 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc6406640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0102600 0x7f3dc019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc6406640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0102600 0x7f3dc019e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48590/0 (socket says 172.21.15.16:48590) 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc6406640 1 -- 172.21.15.16:0/287604709 learned_addr learned my addr 172.21.15.16:0/287604709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dc0100a50 0x7f3dc0196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc5404640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc0101c40 0x7f3dc0197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc6406640 1 -- 172.21.15.16:0/287604709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc0101c40 msgr2=0x7f3dc0197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc6406640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc0101c40 0x7f3dc0197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc6406640 1 -- 172.21.15.16:0/287604709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dc0100a50 msgr2=0x7f3dc0196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc6406640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dc0100a50 0x7f3dc0196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc6406640 1 -- 172.21.15.16:0/287604709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc019ea20 con 0x7f3dc0102600 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dc0100a50 0x7f3dc0196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc6406640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0102600 0x7f3dc019e3b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3db002c700 tx=0x7f3db00346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dbeffd640 1 -- 172.21.15.16:0/287604709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0039040 con 0x7f3dc0102600 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dbeffd640 1 -- 172.21.15.16:0/287604709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3db0044070 con 0x7f3dc0102600 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc7e69640 1 -- 172.21.15.16:0/287604709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc019ecb0 con 0x7f3dc0102600 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dbeffd640 1 -- 172.21.15.16:0/287604709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db00044d0 con 0x7f3dc0102600 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.595+0000 7f3dc7e69640 1 -- 172.21.15.16:0/287604709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc019f1a0 con 0x7f3dc0102600 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.597+0000 7f3dbeffd640 1 -- 172.21.15.16:0/287604709 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f3db0004670 con 0x7f3dc0102600 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dcaa4c740 1 Processor -- start 2024-03-27T06:42:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dcaa4c740 1 -- start start 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e1bd0 0x55f1335e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e6d40 0x55f1335e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f133544010 con 0x55f1335e11e0 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55f1334ec770 con 0x55f1335e6d40 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f13353ffc0 con 0x55f1335e1bd0 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dbeffd640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3da80a3610 0x7f3da80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dc5404640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dc5404640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335e1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48592/0 (socket says 172.21.15.16:48592) 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dc5c05640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e1bd0 0x55f1335e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dc5404640 1 -- 172.21.15.16:0/406348137 learned_addr learned my addr 172.21.15.16:0/406348137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dc6406640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3da80a3610 0x7f3da80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.598+0000 7f3dc5404640 1 --2- 172.21.15.16:0/406348137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e6d40 0x55f1335e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3dc5c05640 1 -- 172.21.15.16:0/406348137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e6d40 msgr2=0x55f1335e91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3dbeffd640 1 -- 172.21.15.16:0/287604709 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3db00f24a0 con 0x7f3dc0102600 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/406348137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e6d40 0x55f1335e91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3dc5c05640 1 -- 172.21.15.16:0/406348137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 msgr2=0x55f1335e1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/406348137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335e1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3dc5c05640 1 -- 172.21.15.16:0/406348137 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f13350b330 con 0x55f1335e1bd0 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3dc5404640 1 --2- 172.21.15.16:0/406348137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e6d40 0x55f1335e91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/406348137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e1bd0 0x55f1335e66d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009fa0 tx=0x7f3db80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfbc9b318ff35b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3dc6406640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3da80a3610 0x7f3da80a5ad0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3db00bad80 tx=0x7f3db00e9840 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3da7fff640 1 -- 172.21.15.16:0/406348137 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8004310 con 0x55f1335e1bd0 2024-03-27T06:42:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3da7fff640 1 -- 172.21.15.16:0/406348137 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3db80044b0 con 0x55f1335e1bd0 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.599+0000 7f3da7fff640 1 -- 172.21.15.16:0/406348137 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8003850 con 0x55f1335e1bd0 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.600+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/406348137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e1bd0 msgr2=0x55f1335e66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.600+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/406348137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e1bd0 0x55f1335e66d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009fa0 tx=0x7f3db80118d0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.600+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/406348137 shutdown_connections 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.600+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/406348137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e6d40 0x55f1335e91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.600+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/406348137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e1bd0 0x55f1335e66d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.600+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/406348137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335e1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.600+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/406348137 >> 172.21.15.16:0/406348137 conn(0x55f1335a6b20 msgr2=0x55f1335da940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.600+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/406348137 shutdown_connections 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.600+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/406348137 wait complete. 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.600+0000 7f3dcaa4c740 1 -- start start 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.600+0000 7f3dcaa4c740 1 Processor -- start 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335eb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e1bd0 0x55f1335ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e6d40 0x55f1335f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f1335e9cf0 con 0x55f1335e11e0 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55f1334ffb30 con 0x55f1335e1bd0 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f1335e9e70 con 0x55f1335e6d40 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5404640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dcaa4c740 10 client.0 _ll_register_callbacks cb 0x55f13353b1a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5404640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335eb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48594/0 (socket says 172.21.15.16:48594) 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5404640 1 -- 172.21.15.16:0/2583873580 learned_addr learned my addr 172.21.15.16:0/2583873580 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e1bd0 0x55f1335ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5404640 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e6d40 0x55f1335f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5404640 1 -- 172.21.15.16:0/2583873580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e6d40 msgr2=0x55f1335f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5404640 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e6d40 0x55f1335f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5404640 1 -- 172.21.15.16:0/2583873580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e1bd0 msgr2=0x55f1335ebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5404640 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e1bd0 0x55f1335ebc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5404640 1 -- 172.21.15.16:0/2583873580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1332582d0 con 0x55f1335e11e0 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e1bd0 0x55f1335ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3dc5404640 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335eb750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3db4012c50 tx=0x7f3db400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a0a44d60848348 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3da57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.601+0000 7f3da57fa640 1 -- 172.21.15.16:0/2583873580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db40120d0 con 0x55f1335e11e0 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3da57fa640 1 -- 172.21.15.16:0/2583873580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3db400ebe0 con 0x55f1335e11e0 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/2583873580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db80044b0 con 0x55f1335e11e0 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3da57fa640 1 -- 172.21.15.16:0/2583873580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3db400ede0 con 0x55f1335e11e0 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3dcaa4c740 10 client.15057 fetch_fsmap learned FSMap version 24 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/2583873580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f1333226b0 con 0x55f1335e11e0 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3da57fa640 1 -- 172.21.15.16:0/2583873580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3db80044b0 con 0x55f1335e11e0 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3da57fa640 1 -- 172.21.15.16:0/2583873580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f3db40234e0 con 0x55f1335e11e0 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3dcaa4c740 10 client.15057 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3dcaa4c740 10 client.15057 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-27T06:42:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3dcaa4c740 20 client.15057 populate_metadata read hostname 'smithi016' 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55f1335f8130 0x55f1335f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.602+0000 7f3dcaa4c740 4 client.15057 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.603+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/2583873580 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f133551550 con 0x55f1335f8130 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.603+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55f1335f8130 0x55f1335f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.603+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55f1335f8130 0x55f1335f95a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad85355452ce671b server_cookie=1a9b76fd3ed47ab9 in_seq=0 out_seq=0 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.603+0000 7f3da57fa640 10 client.15057 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3da57fa640 1 -- 172.21.15.16:0/2583873580 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f133551550 con 0x55f1335f8130 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3da57fa640 10 client.15057 handle_command_reply: tid=0 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 1 client.15057 shutdown 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/2583873580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 msgr2=0x55f1335eb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335eb750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3db4012c50 tx=0x7f3db400b410 comp rx=0 tx=0).stop 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/2583873580 shutdown_connections 2024-03-27T06:42:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55f1335f8130 0x55f1335f95a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e6d40 0x55f1335f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3da57fa640 0 client.15057 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:42:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e1bd0 0x55f1335ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/2583873580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335eb750 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/2583873580 >> 172.21.15.16:0/2583873580 conn(0x55f1335a6b20 msgr2=0x55f1335d9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/2583873580 shutdown_connections 2024-03-27T06:42:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/2583873580 wait complete. 2024-03-27T06:42:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.604+0000 7f3dcaa4c740 20 client.15057 trim_cache size 0 max 16384 2024-03-27T06:42:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.622+0000 7f3dcaa4c740 1 Processor -- start 2024-03-27T06:42:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.622+0000 7f3dcaa4c740 1 -- start start 2024-03-27T06:42:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.622+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335e3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.622+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e1bd0 0x55f1335e36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.622+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e6d40 0x55f1335f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.622+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f1335f78e0 con 0x55f1335e11e0 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.622+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55f1335f7fb0 con 0x55f1335e1bd0 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.622+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f1335f8680 con 0x55f1335e6d40 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3dc5404640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3dc5404640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335e3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48600/0 (socket says 172.21.15.16:48600) 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3dc5404640 1 -- 172.21.15.16:0/748740485 learned_addr learned my addr 172.21.15.16:0/748740485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/748740485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e1bd0 0x55f1335e36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3dc5404640 1 --2- 172.21.15.16:0/748740485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e6d40 0x55f1335f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3dc5404640 1 -- 172.21.15.16:0/748740485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e6d40 msgr2=0x55f1335f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3dc5404640 1 --2- 172.21.15.16:0/748740485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e6d40 0x55f1335f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3dc5404640 1 -- 172.21.15.16:0/748740485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e1bd0 msgr2=0x55f1335e36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3dc5404640 1 --2- 172.21.15.16:0/748740485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e1bd0 0x55f1335e36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3dc5404640 1 -- 172.21.15.16:0/748740485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1335db690 con 0x55f1335e11e0 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3dc5404640 1 --2- 172.21.15.16:0/748740485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335e3170 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3db4005d00 tx=0x7f3db4012930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ce4b36cc2404c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3da77fe640 1 -- 172.21.15.16:0/748740485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4005850 con 0x55f1335e11e0 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3da77fe640 1 -- 172.21.15.16:0/748740485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3db400ebb0 con 0x55f1335e11e0 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.623+0000 7f3da77fe640 1 -- 172.21.15.16:0/748740485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4005460 con 0x55f1335e11e0 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.624+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/748740485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 msgr2=0x55f1335e3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.624+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/748740485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335e3170 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3db4005d00 tx=0x7f3db4012930 comp rx=0 tx=0).stop 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.624+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/748740485 shutdown_connections 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.624+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/748740485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e6d40 0x55f1335f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.624+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/748740485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e1bd0 0x55f1335e36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.624+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/748740485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e11e0 0x55f1335e3170 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.624+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/748740485 >> 172.21.15.16:0/748740485 conn(0x55f1335a6b20 msgr2=0x55f1335d56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.624+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/748740485 shutdown_connections 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.624+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/748740485 wait complete. 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.624+0000 7f3dcaa4c740 1 -- start start 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.624+0000 7f3dcaa4c740 1 Processor -- start 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e11e0 0x55f1335f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e1bd0 0x55f133659e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dcaa4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e6d40 0x55f13365c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f1335f9420 con 0x55f1335e6d40 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55f1335f8d50 con 0x55f1335e11e0 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dcaa4c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f1335f9af0 con 0x55f1335e1bd0 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dcaa4c740 10 client.0 _ll_register_callbacks cb 0x7f3db400bed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dc5404640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e6d40 0x55f13365c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dc5404640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e6d40 0x55f13365c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48602/0 (socket says 172.21.15.16:48602) 2024-03-27T06:42:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dc5404640 1 -- 172.21.15.16:0/3405954720 learned_addr learned my addr 172.21.15.16:0/3405954720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dc5404640 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e11e0 0x55f1335f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e1bd0 0x55f133659e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dc5404640 1 -- 172.21.15.16:0/3405954720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e1bd0 msgr2=0x55f133659e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dc5404640 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e1bd0 0x55f133659e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dc5404640 1 -- 172.21.15.16:0/3405954720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e11e0 msgr2=0x55f1335f0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dc5404640 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e11e0 0x55f1335f0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dc5404640 1 -- 172.21.15.16:0/3405954720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f13365c910 con 0x55f1335e6d40 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dc5404640 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e6d40 0x55f13365c3d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3db4012740 tx=0x7f3db402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0dc3775d05e3d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3da57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3da57fa640 1 -- 172.21.15.16:0/3405954720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db40120d0 con 0x55f1335e6d40 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3da57fa640 1 -- 172.21.15.16:0/3405954720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3db4012270 con 0x55f1335e6d40 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.625+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/3405954720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db400ebb0 con 0x55f1335e6d40 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3da57fa640 1 -- 172.21.15.16:0/3405954720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3db4023040 con 0x55f1335e6d40 2024-03-27T06:42:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3dcaa4c740 10 client.15063 fetch_fsmap learned FSMap version 24 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/3405954720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f13365dab0 con 0x55f1335e6d40 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3da57fa640 1 -- 172.21.15.16:0/3405954720 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3db400ebb0 con 0x55f1335e6d40 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3da57fa640 1 -- 172.21.15.16:0/3405954720 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f3db4004c70 con 0x55f1335e6d40 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3dcaa4c740 10 client.15063 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3dcaa4c740 10 client.15063 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3dcaa4c740 20 client.15063 populate_metadata read hostname 'smithi016' 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55f133662430 0x55f133661140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3dcaa4c740 4 client.15063 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/3405954720 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "status"}) -- 0x55f13354b8a0 con 0x55f133662430 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55f133662430 0x55f133661140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3da57fa640 10 client.15063 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.626+0000 7f3dc5c05640 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55f133662430 0x55f133661140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=58f0617eb7618d8a server_cookie=a5674cb9ac21d0f9 in_seq=0 out_seq=0 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3da57fa640 1 -- 172.21.15.16:0/3405954720 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f13354b8a0 con 0x55f133662430 2024-03-27T06:42:58.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3da57fa640 10 client.15063 handle_command_reply: tid=0 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 1 client.15063 shutdown 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/3405954720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e6d40 msgr2=0x55f13365c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e6d40 0x55f13365c3d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3db4012740 tx=0x7f3db402e040 comp rx=0 tx=0).stop 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/3405954720 shutdown_connections 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55f133662430 0x55f133661140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1335e6d40 0x55f13365c3d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1335e1bd0 0x55f133659e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 1 --2- 172.21.15.16:0/3405954720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f1335e11e0 0x55f1335f0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3da57fa640 0 client.15063 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/3405954720 >> 172.21.15.16:0/3405954720 conn(0x55f1335a6b20 msgr2=0x55f1335e3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/3405954720 shutdown_connections 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 1 -- 172.21.15.16:0/3405954720 wait complete. 2024-03-27T06:42:58.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dcaa4c740 20 client.15063 trim_cache size 0 max 16384 2024-03-27T06:42:58.630 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:42:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:42:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-03-27T06:42:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24467, 2024-03-27T06:42:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-03-27T06:42:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-03-27T06:42:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-03-27T06:42:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 84.119606825000005, 2024-03-27T06:42:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-03-27T06:42:58.630 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-03-27T06:42:58.631 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-03-27T06:42:58.631 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 114.054982353 2024-03-27T06:42:58.631 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:42:58.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dc7e69640 1 -- 172.21.15.16:0/287604709 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3da80a3610 msgr2=0x7f3da80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dc7e69640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3da80a3610 0x7f3da80a5ad0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3db00bad80 tx=0x7f3db00e9840 comp rx=0 tx=0).stop 2024-03-27T06:42:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dc7e69640 1 -- 172.21.15.16:0/287604709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0102600 msgr2=0x7f3dc019e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dc7e69640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0102600 0x7f3dc019e3b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3db002c700 tx=0x7f3db00346d0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dc7e69640 1 -- 172.21.15.16:0/287604709 shutdown_connections 2024-03-27T06:42:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dc7e69640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3da80a3610 0x7f3da80a5ad0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dc7e69640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0102600 0x7f3dc019e3b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dc7e69640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc0101c40 0x7f3dc0197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dc7e69640 1 --2- 172.21.15.16:0/287604709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3dc0100a50 0x7f3dc0196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.627+0000 7f3dc7e69640 1 -- 172.21.15.16:0/287604709 >> 172.21.15.16:0/287604709 conn(0x7f3dc00fbfe0 msgr2=0x7f3dc0104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.628+0000 7f3dc7e69640 1 -- 172.21.15.16:0/287604709 shutdown_connections 2024-03-27T06:42:58.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.628+0000 7f3dc7e69640 1 -- 172.21.15.16:0/287604709 wait complete. 2024-03-27T06:42:58.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-27T06:42:58.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:58 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1934240989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:42:58.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:58 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3211961864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:42:58.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:58 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2738691936' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:42:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.743+0000 7f85e93ac640 1 -- 172.21.15.16:0/460631232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e40ffe40 msgr2=0x7f85e4100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.743+0000 7f85e93ac640 1 --2- 172.21.15.16:0/460631232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e40ffe40 0x7f85e4100260 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009930 tx=0x7f85d402bc30 comp rx=0 tx=0).stop 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.743+0000 7f85e93ac640 1 -- 172.21.15.16:0/460631232 shutdown_connections 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.743+0000 7f85e93ac640 1 --2- 172.21.15.16:0/460631232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e41037d0 0x7f85e410a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.743+0000 7f85e93ac640 1 --2- 172.21.15.16:0/460631232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85e4102e10 0x7f85e4103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.743+0000 7f85e93ac640 1 --2- 172.21.15.16:0/460631232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e40ffe40 0x7f85e4100260 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.743+0000 7f85e93ac640 1 -- 172.21.15.16:0/460631232 >> 172.21.15.16:0/460631232 conn(0x7f85e40fb750 msgr2=0x7f85e40fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.743+0000 7f85e93ac640 1 -- 172.21.15.16:0/460631232 shutdown_connections 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.743+0000 7f85e93ac640 1 -- 172.21.15.16:0/460631232 wait complete. 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.743+0000 7f85e93ac640 1 Processor -- start 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.743+0000 7f85e93ac640 1 -- start start 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e93ac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40ffe40 0x7f85e4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e93ac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85e4102e10 0x7f85e41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e93ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e41037d0 0x7f85e419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e93ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85e410d8e0 con 0x7f85e41037d0 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e93ac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f85e410d760 con 0x7f85e4102e10 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e93ac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f85e410da60 con 0x7f85e40ffe40 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e41037d0 0x7f85e419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e41037d0 0x7f85e419f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48614/0 (socket says 172.21.15.16:48614) 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e37fe640 1 -- 172.21.15.16:0/3630167551 learned_addr learned my addr 172.21.15.16:0/3630167551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85dbfff640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85e4102e10 0x7f85e41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40ffe40 0x7f85e4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e37fe640 1 -- 172.21.15.16:0/3630167551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40ffe40 msgr2=0x7f85e4197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e37fe640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40ffe40 0x7f85e4197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e37fe640 1 -- 172.21.15.16:0/3630167551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85e4102e10 msgr2=0x7f85e41984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e37fe640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85e4102e10 0x7f85e41984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e37fe640 1 -- 172.21.15.16:0/3630167551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e419fbd0 con 0x7f85e41037d0 2024-03-27T06:42:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e37fe640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e41037d0 0x7f85e419f560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00efd0 tx=0x7f85dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e0ff9640 1 -- 172.21.15.16:0/3630167551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc003b70 con 0x7f85e41037d0 2024-03-27T06:42:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e0ff9640 1 -- 172.21.15.16:0/3630167551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85dc003d10 con 0x7f85e41037d0 2024-03-27T06:42:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e93ac640 1 -- 172.21.15.16:0/3630167551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e419fec0 con 0x7f85e41037d0 2024-03-27T06:42:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e0ff9640 1 -- 172.21.15.16:0/3630167551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc010a90 con 0x7f85e41037d0 2024-03-27T06:42:58.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.744+0000 7f85e93ac640 1 -- 172.21.15.16:0/3630167551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e41a0330 con 0x7f85e41037d0 2024-03-27T06:42:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.746+0000 7f85e0ff9640 1 -- 172.21.15.16:0/3630167551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f85dc004020 con 0x7f85e41037d0 2024-03-27T06:42:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85ebf90740 1 Processor -- start 2024-03-27T06:42:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85ebf90740 1 -- start start 2024-03-27T06:42:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a28fd0 0x555644a293f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a299c0 0x555644a2e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a2eb30 0x555644a30fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85ebf90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55564498e010 con 0x555644a299c0 2024-03-27T06:42:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85ebf90740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555644936770 con 0x555644a2eb30 2024-03-27T06:42:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85ebf90740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555644989fc0 con 0x555644a28fd0 2024-03-27T06:42:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a299c0 0x555644a2e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a299c0 0x555644a2e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48624/0 (socket says 172.21.15.16:48624) 2024-03-27T06:42:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85e2ffd640 1 -- 172.21.15.16:0/2201177428 learned_addr learned my addr 172.21.15.16:0/2201177428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85dbfff640 1 --2- 172.21.15.16:0/2201177428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a28fd0 0x555644a293f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85e0ff9640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f85d00a3610 0x7f85d00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85dbfff640 1 --2- 172.21.15.16:0/2201177428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a2eb30 0x555644a30fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85e37fe640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f85d00a3610 0x7f85d00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85e2ffd640 1 -- 172.21.15.16:0/2201177428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a28fd0 msgr2=0x555644a293f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/2201177428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a28fd0 0x555644a293f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85e2ffd640 1 -- 172.21.15.16:0/2201177428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a2eb30 msgr2=0x555644a30fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/2201177428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a2eb30 0x555644a30fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85e2ffd640 1 -- 172.21.15.16:0/2201177428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555644955330 con 0x555644a299c0 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.747+0000 7f85e0ff9640 1 -- 172.21.15.16:0/3630167551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f85dc0d0160 con 0x7f85e41037d0 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.748+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/2201177428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a299c0 0x555644a2e4c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009900 tx=0x7f85d4009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137b07a7167b00e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.748+0000 7f85e37fe640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f85d00a3610 0x7f85d00a5ad0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f85dc09c5d0 tx=0x7f85dc091b20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.748+0000 7f85d97fa640 1 -- 172.21.15.16:0/2201177428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d402c5d0 con 0x555644a299c0 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.748+0000 7f85d97fa640 1 -- 172.21.15.16:0/2201177428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85d4004040 con 0x555644a299c0 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.748+0000 7f85d97fa640 1 -- 172.21.15.16:0/2201177428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d40046c0 con 0x555644a299c0 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 -- 172.21.15.16:0/2201177428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a299c0 msgr2=0x555644a2e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 --2- 172.21.15.16:0/2201177428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a299c0 0x555644a2e4c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009900 tx=0x7f85d4009670 comp rx=0 tx=0).stop 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 -- 172.21.15.16:0/2201177428 shutdown_connections 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 --2- 172.21.15.16:0/2201177428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a2eb30 0x555644a30fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 --2- 172.21.15.16:0/2201177428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a299c0 0x555644a2e4c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 --2- 172.21.15.16:0/2201177428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a28fd0 0x555644a293f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 -- 172.21.15.16:0/2201177428 >> 172.21.15.16:0/2201177428 conn(0x5556449f0b20 msgr2=0x555644a24940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 -- 172.21.15.16:0/2201177428 shutdown_connections 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 -- 172.21.15.16:0/2201177428 wait complete. 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 -- start start 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 Processor -- start 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a28fd0 0x555644a33480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a299c0 0x555644a339c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a2eb30 0x555644a3aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555644a31a80 con 0x555644a2eb30 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555644949b30 con 0x555644a28fd0 2024-03-27T06:42:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555644a31c00 con 0x555644a299c0 2024-03-27T06:42:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85ebf90740 10 client.0 _ll_register_callbacks cb 0x5556449851a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:42:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a2eb30 0x555644a3aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a2eb30 0x555644a3aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48636/0 (socket says 172.21.15.16:48636) 2024-03-27T06:42:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85dbfff640 1 -- 172.21.15.16:0/769432054 learned_addr learned my addr 172.21.15.16:0/769432054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85dbfff640 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a28fd0 0x555644a33480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.749+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a299c0 0x555644a339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85dbfff640 1 -- 172.21.15.16:0/769432054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a299c0 msgr2=0x555644a339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85dbfff640 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a299c0 0x555644a339c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85dbfff640 1 -- 172.21.15.16:0/769432054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a28fd0 msgr2=0x555644a33480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85dbfff640 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a28fd0 0x555644a33480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85dbfff640 1 -- 172.21.15.16:0/769432054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556446a22d0 con 0x555644a2eb30 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85dbfff640 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a2eb30 0x555644a3aa00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00f410 tx=0x7f85cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d536b690a1623dc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85ca7fc640 1 -- 172.21.15.16:0/769432054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc00fe50 con 0x555644a2eb30 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85ca7fc640 1 -- 172.21.15.16:0/769432054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f85cc004250 con 0x555644a2eb30 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85ebf90740 1 -- 172.21.15.16:0/769432054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d4004040 con 0x555644a2eb30 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85ca7fc640 1 -- 172.21.15.16:0/769432054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f85cc004450 con 0x555644a2eb30 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85ebf90740 10 client.15075 fetch_fsmap learned FSMap version 24 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85ebf90740 1 -- 172.21.15.16:0/769432054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55564476c6b0 con 0x555644a2eb30 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85ca7fc640 1 -- 172.21.15.16:0/769432054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85d4004040 con 0x555644a2eb30 2024-03-27T06:42:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85ca7fc640 1 -- 172.21.15.16:0/769432054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f85cc020040 con 0x555644a2eb30 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.750+0000 7f85ebf90740 10 client.15075 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.751+0000 7f85ebf90740 10 client.15075 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.751+0000 7f85ebf90740 20 client.15075 populate_metadata read hostname 'smithi016' 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.751+0000 7f85ebf90740 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555644a3ff90 0x555644a41400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.751+0000 7f85ebf90740 4 client.15075 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.751+0000 7f85ebf90740 1 -- 172.21.15.16:0/769432054 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55564499b550 con 0x555644a3ff90 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.751+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555644a3ff90 0x555644a41400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.751+0000 7f85ca7fc640 10 client.15075 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.751+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555644a3ff90 0x555644a41400 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=40d39296812cac19 server_cookie=a3ac3b677231d77f in_seq=0 out_seq=0 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ca7fc640 1 -- 172.21.15.16:0/769432054 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55564499b550 con 0x555644a3ff90 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ca7fc640 10 client.15075 handle_command_reply: tid=0 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 1 client.15075 shutdown 2024-03-27T06:42:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 1 -- 172.21.15.16:0/769432054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a2eb30 msgr2=0x555644a3aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a2eb30 0x555644a3aa00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00f410 tx=0x7f85cc0046c0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 1 -- 172.21.15.16:0/769432054 shutdown_connections 2024-03-27T06:42:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555644a3ff90 0x555644a41400 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a2eb30 0x555644a3aa00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a299c0 0x555644a339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ca7fc640 0 client.15075 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:42:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 1 --2- 172.21.15.16:0/769432054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a28fd0 0x555644a33480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 1 -- 172.21.15.16:0/769432054 >> 172.21.15.16:0/769432054 conn(0x5556449f0b20 msgr2=0x555644a23b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 1 -- 172.21.15.16:0/769432054 shutdown_connections 2024-03-27T06:42:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 1 -- 172.21.15.16:0/769432054 wait complete. 2024-03-27T06:42:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.752+0000 7f85ebf90740 20 client.15075 trim_cache size 0 max 16384 2024-03-27T06:42:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.770+0000 7f85ebf90740 1 Processor -- start 2024-03-27T06:42:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.770+0000 7f85ebf90740 1 -- start start 2024-03-27T06:42:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.770+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.770+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a299c0 0x555644a3a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.770+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a2eb30 0x555644a27b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.770+0000 7f85ebf90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555644a3f740 con 0x555644a28fd0 2024-03-27T06:42:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.770+0000 7f85ebf90740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555644a3fe10 con 0x555644a299c0 2024-03-27T06:42:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.770+0000 7f85ebf90740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555644a404e0 con 0x555644a2eb30 2024-03-27T06:42:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.770+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48642/0 (socket says 172.21.15.16:48642) 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85dbfff640 1 -- 172.21.15.16:0/674848645 learned_addr learned my addr 172.21.15.16:0/674848645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/674848645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a299c0 0x555644a3a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85dbfff640 1 --2- 172.21.15.16:0/674848645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a2eb30 0x555644a27b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85dbfff640 1 -- 172.21.15.16:0/674848645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a2eb30 msgr2=0x555644a27b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85dbfff640 1 --2- 172.21.15.16:0/674848645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a2eb30 0x555644a27b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85dbfff640 1 -- 172.21.15.16:0/674848645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a299c0 msgr2=0x555644a3a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85dbfff640 1 --2- 172.21.15.16:0/674848645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a299c0 0x555644a3a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85dbfff640 1 -- 172.21.15.16:0/674848645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555644a32890 con 0x555644a28fd0 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/674848645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a299c0 0x555644a3a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85dbfff640 1 --2- 172.21.15.16:0/674848645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a337a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0114c0 tx=0x7f85cc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742322b88cee3234 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85d8ff9640 1 -- 172.21.15.16:0/674848645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc032070 con 0x555644a28fd0 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85d8ff9640 1 -- 172.21.15.16:0/674848645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85cc033040 con 0x555644a28fd0 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.771+0000 7f85d8ff9640 1 -- 172.21.15.16:0/674848645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc003af0 con 0x555644a28fd0 2024-03-27T06:42:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 -- 172.21.15.16:0/674848645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 msgr2=0x555644a337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 --2- 172.21.15.16:0/674848645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a337a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0114c0 tx=0x7f85cc005f90 comp rx=0 tx=0).stop 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 -- 172.21.15.16:0/674848645 shutdown_connections 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 --2- 172.21.15.16:0/674848645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a2eb30 0x555644a27b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 --2- 172.21.15.16:0/674848645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a299c0 0x555644a3a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 --2- 172.21.15.16:0/674848645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a337a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 -- 172.21.15.16:0/674848645 >> 172.21.15.16:0/674848645 conn(0x5556449f0b20 msgr2=0x555644a33090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 -- 172.21.15.16:0/674848645 shutdown_connections 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 -- 172.21.15.16:0/674848645 wait complete. 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 -- start start 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 Processor -- start 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a36af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a299c0 0x555644a37030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a2eb30 0x555644a3a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555644a41280 con 0x555644a28fd0 2024-03-27T06:42:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555644a40bb0 con 0x555644a299c0 2024-03-27T06:42:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.772+0000 7f85ebf90740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555644a41950 con 0x555644a2eb30 2024-03-27T06:42:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85ebf90740 10 client.0 _ll_register_callbacks cb 0x7f85cc011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:42:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a36af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a36af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48652/0 (socket says 172.21.15.16:48652) 2024-03-27T06:42:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85dbfff640 1 -- 172.21.15.16:0/557555678 learned_addr learned my addr 172.21.15.16:0/557555678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a299c0 0x555644a37030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85dbfff640 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a2eb30 0x555644a3a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85dbfff640 1 -- 172.21.15.16:0/557555678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a2eb30 msgr2=0x555644a3a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85dbfff640 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a2eb30 0x555644a3a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85dbfff640 1 -- 172.21.15.16:0/557555678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a299c0 msgr2=0x555644a37030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85dbfff640 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a299c0 0x555644a37030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85dbfff640 1 -- 172.21.15.16:0/557555678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555644a3a640 con 0x555644a28fd0 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85dbfff640 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a36af0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0237c0 tx=0x7f85cc009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d647d18cb45174 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85ca7fc640 1 -- 172.21.15.16:0/557555678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc032040 con 0x555644a28fd0 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85ca7fc640 1 -- 172.21.15.16:0/557555678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f85cc005040 con 0x555644a28fd0 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85ebf90740 1 -- 172.21.15.16:0/557555678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85cc033040 con 0x555644a28fd0 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85ca7fc640 1 -- 172.21.15.16:0/557555678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f85cc00fe50 con 0x555644a28fd0 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85ebf90740 10 client.15081 fetch_fsmap learned FSMap version 24 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85ebf90740 1 -- 172.21.15.16:0/557555678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555644aa67e0 con 0x555644a28fd0 2024-03-27T06:42:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.773+0000 7f85ca7fc640 1 -- 172.21.15.16:0/557555678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85cc033040 con 0x555644a28fd0 2024-03-27T06:42:58.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ca7fc640 1 -- 172.21.15.16:0/557555678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f85cc0045f0 con 0x555644a28fd0 2024-03-27T06:42:58.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ebf90740 10 client.15081 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:42:58.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ebf90740 10 client.15081 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-27T06:42:58.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ebf90740 20 client.15081 populate_metadata read hostname 'smithi016' 2024-03-27T06:42:58.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ebf90740 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555644a3ff90 0x555644aa9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ebf90740 4 client.15081 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T06:42:58.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ebf90740 1 -- 172.21.15.16:0/557555678 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "status"}) -- 0x5556449958a0 con 0x555644a3ff90 2024-03-27T06:42:58.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555644a3ff90 0x555644aa9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ca7fc640 10 client.15081 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:42:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85e2ffd640 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555644a3ff90 0x555644aa9e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d142d7f3b1dc4d6 server_cookie=7a6f21ef427f2390 in_seq=0 out_seq=0 2024-03-27T06:42:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ca7fc640 1 -- 172.21.15.16:0/557555678 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5556449958a0 con 0x555644a3ff90 2024-03-27T06:42:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ca7fc640 10 client.15081 handle_command_reply: tid=0 2024-03-27T06:42:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ebf90740 1 client.15081 shutdown 2024-03-27T06:42:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ebf90740 1 -- 172.21.15.16:0/557555678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 msgr2=0x555644a36af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.774+0000 7f85ebf90740 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a36af0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0237c0 tx=0x7f85cc009fb0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85ebf90740 1 -- 172.21.15.16:0/557555678 shutdown_connections 2024-03-27T06:42:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85ebf90740 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555644a3ff90 0x555644aa9e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85ebf90740 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555644a2eb30 0x555644a3a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85ebf90740 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555644a299c0 0x555644a37030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85ebf90740 1 --2- 172.21.15.16:0/557555678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555644a28fd0 0x555644a36af0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85ca7fc640 0 client.15081 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:42:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85ebf90740 1 -- 172.21.15.16:0/557555678 >> 172.21.15.16:0/557555678 conn(0x5556449f0b20 msgr2=0x555644a23290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85ebf90740 1 -- 172.21.15.16:0/557555678 shutdown_connections 2024-03-27T06:42:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85ebf90740 1 -- 172.21.15.16:0/557555678 wait complete. 2024-03-27T06:42:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85ebf90740 20 client.15081 trim_cache size 0 max 16384 2024-03-27T06:42:58.779 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:42:58.779 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:42:58.779 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-03-27T06:42:58.779 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24472, 2024-03-27T06:42:58.779 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-03-27T06:42:58.780 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-03-27T06:42:58.780 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-03-27T06:42:58.780 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 85.283379339999996, 2024-03-27T06:42:58.780 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-03-27T06:42:58.780 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-03-27T06:42:58.780 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-03-27T06:42:58.780 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 108.703452555 2024-03-27T06:42:58.780 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:42:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 -- 172.21.15.16:0/3630167551 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f85d00a3610 msgr2=0x7f85d00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f85d00a3610 0x7f85d00a5ad0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f85dc09c5d0 tx=0x7f85dc091b20 comp rx=0 tx=0).stop 2024-03-27T06:42:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 -- 172.21.15.16:0/3630167551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e41037d0 msgr2=0x7f85e419f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e41037d0 0x7f85e419f560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00efd0 tx=0x7f85dc00c6a0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 -- 172.21.15.16:0/3630167551 shutdown_connections 2024-03-27T06:42:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f85d00a3610 0x7f85d00a5ad0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e41037d0 0x7f85e419f560 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85e4102e10 0x7f85e41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 --2- 172.21.15.16:0/3630167551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40ffe40 0x7f85e4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 -- 172.21.15.16:0/3630167551 >> 172.21.15.16:0/3630167551 conn(0x7f85e40fb750 msgr2=0x7f85e4106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 -- 172.21.15.16:0/3630167551 shutdown_connections 2024-03-27T06:42:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.775+0000 7f85e93ac640 1 -- 172.21.15.16:0/3630167551 wait complete. 2024-03-27T06:42:58.782 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:58 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1934240989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:42:58.782 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:58 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3211961864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:42:58.782 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:58 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2738691936' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:42:58.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-27T06:42:58.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:58 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1934240989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:42:58.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:58 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3211961864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:42:58.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:58 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2738691936' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:42:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.892+0000 7f50fe581640 1 -- 172.21.15.16:0/219535600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50f8102610 msgr2=0x7f50f8108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.892+0000 7f50fe581640 1 --2- 172.21.15.16:0/219535600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50f8102610 0x7f50f8108ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009be0 tx=0x7f50f002bef0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.892+0000 7f50fe581640 1 -- 172.21.15.16:0/219535600 shutdown_connections 2024-03-27T06:42:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.892+0000 7f50fe581640 1 --2- 172.21.15.16:0/219535600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50f8102610 0x7f50f8108ea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.892+0000 7f50fe581640 1 --2- 172.21.15.16:0/219535600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50f8101c50 0x7f50f81020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.892+0000 7f50fe581640 1 --2- 172.21.15.16:0/219535600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f8100a60 0x7f50f8100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.892+0000 7f50fe581640 1 -- 172.21.15.16:0/219535600 >> 172.21.15.16:0/219535600 conn(0x7f50f80fbff0 msgr2=0x7f50f80fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.892+0000 7f50fe581640 1 -- 172.21.15.16:0/219535600 shutdown_connections 2024-03-27T06:42:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.892+0000 7f50fe581640 1 -- 172.21.15.16:0/219535600 wait complete. 2024-03-27T06:42:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.892+0000 7f50fe581640 1 Processor -- start 2024-03-27T06:42:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.892+0000 7f50fe581640 1 -- start start 2024-03-27T06:42:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50fe581640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50f8100a60 0x7f50f8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50fe581640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f8101c50 0x7f50f81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50fe581640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50f8102610 0x7f50f819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50fe581640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50f810d920 con 0x7f50f8101c50 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50fe581640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f50f810d7a0 con 0x7f50f8102610 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50fe581640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f50f810daa0 con 0x7f50f8100a60 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f8101c50 0x7f50f81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f8101c50 0x7f50f81984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48668/0 (socket says 172.21.15.16:48668) 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f77fe640 1 -- 172.21.15.16:0/558494553 learned_addr learned my addr 172.21.15.16:0/558494553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50f8102610 0x7f50f819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f7fff640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50f8100a60 0x7f50f8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f77fe640 1 -- 172.21.15.16:0/558494553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50f8100a60 msgr2=0x7f50f8197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f77fe640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50f8100a60 0x7f50f8197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f77fe640 1 -- 172.21.15.16:0/558494553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50f8102610 msgr2=0x7f50f819f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f77fe640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50f8102610 0x7f50f819f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f77fe640 1 -- 172.21.15.16:0/558494553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f819fb80 con 0x7f50f8101c50 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f77fe640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f8101c50 0x7f50f81984a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00e9f0 tx=0x7f50ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f57fa640 1 -- 172.21.15.16:0/558494553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00cd80 con 0x7f50f8101c50 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f57fa640 1 -- 172.21.15.16:0/558494553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f50ec005520 con 0x7f50f8101c50 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50fe581640 1 -- 172.21.15.16:0/558494553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f819fe70 con 0x7f50f8101c50 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50f57fa640 1 -- 172.21.15.16:0/558494553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec010690 con 0x7f50f8101c50 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.893+0000 7f50fe581640 1 -- 172.21.15.16:0/558494553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f81a02e0 con 0x7f50f8101c50 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.895+0000 7f50f57fa640 1 -- 172.21.15.16:0/558494553 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f50ec0046c0 con 0x7f50f8101c50 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f510116b740 1 Processor -- start 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f510116b740 1 -- start start 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f510116b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf303a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f510116b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf303f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f510116b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303fb30 0x55dbf3041fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f510116b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dbf2f9f010 con 0x55dbf303a9c0 2024-03-27T06:42:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f510116b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55dbf2f47770 con 0x55dbf3039fd0 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f510116b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dbf2f9afc0 con 0x55dbf303fb30 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50f57fa640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f50e00a3610 0x7f50e00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf303f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50f77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303fb30 0x55dbf3041fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf303f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48682/0 (socket says 172.21.15.16:48682) 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50f7fff640 1 -- 172.21.15.16:0/822832361 learned_addr learned my addr 172.21.15.16:0/822832361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f50e00a3610 0x7f50e00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/822832361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf303a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50f57fa640 1 -- 172.21.15.16:0/558494553 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f50ec0d00d0 con 0x7f50f8101c50 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50f7fff640 1 -- 172.21.15.16:0/822832361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303fb30 msgr2=0x55dbf3041fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50f7fff640 1 --2- 172.21.15.16:0/822832361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303fb30 0x55dbf3041fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50f7fff640 1 -- 172.21.15.16:0/822832361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 msgr2=0x55dbf303a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50f7fff640 1 --2- 172.21.15.16:0/822832361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf303a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50f7fff640 1 -- 172.21.15.16:0/822832361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf2f66330 con 0x55dbf303a9c0 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.896+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/822832361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf303a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.897+0000 7f50f7fff640 1 --2- 172.21.15.16:0/822832361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf303f4c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009c80 tx=0x7f50e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d58a742b938b6a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.897+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f50e00a3610 0x7f50e00a5ad0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f50f0004af0 tx=0x7f50f00092e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.897+0000 7f50de7fc640 1 -- 172.21.15.16:0/822832361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8005220 con 0x55dbf303a9c0 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.897+0000 7f50de7fc640 1 -- 172.21.15.16:0/822832361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f50e8009e30 con 0x55dbf303a9c0 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.897+0000 7f50de7fc640 1 -- 172.21.15.16:0/822832361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e80198d0 con 0x55dbf303a9c0 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.897+0000 7f510116b740 1 -- 172.21.15.16:0/822832361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 msgr2=0x55dbf303f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.897+0000 7f510116b740 1 --2- 172.21.15.16:0/822832361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf303f4c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009c80 tx=0x7f50e8011810 comp rx=0 tx=0).stop 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 -- 172.21.15.16:0/822832361 shutdown_connections 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 --2- 172.21.15.16:0/822832361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303fb30 0x55dbf3041fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 --2- 172.21.15.16:0/822832361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf303f4c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 --2- 172.21.15.16:0/822832361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf303a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 -- 172.21.15.16:0/822832361 >> 172.21.15.16:0/822832361 conn(0x55dbf3001b20 msgr2=0x55dbf3035940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 -- 172.21.15.16:0/822832361 shutdown_connections 2024-03-27T06:42:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 -- 172.21.15.16:0/822832361 wait complete. 2024-03-27T06:42:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 -- start start 2024-03-27T06:42:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 Processor -- start 2024-03-27T06:42:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf3044510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303a9c0 0x55dbf3044a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 0x55dbf304ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dbf3042ae0 con 0x55dbf303fb30 2024-03-27T06:42:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55dbf2f5ab30 con 0x55dbf3039fd0 2024-03-27T06:42:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dbf3042c60 con 0x55dbf303a9c0 2024-03-27T06:42:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f510116b740 10 client.0 _ll_register_callbacks cb 0x55dbf2f961a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:42:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f50fcb1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 0x55dbf304ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.898+0000 7f50fcb1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 0x55dbf304ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48694/0 (socket says 172.21.15.16:48694) 2024-03-27T06:42:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50fcb1e640 1 -- 172.21.15.16:0/1052910325 learned_addr learned my addr 172.21.15.16:0/1052910325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50f7fff640 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf3044510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50f77fe640 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303a9c0 0x55dbf3044a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50fcb1e640 1 -- 172.21.15.16:0/1052910325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303a9c0 msgr2=0x55dbf3044a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303a9c0 0x55dbf3044a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50fcb1e640 1 -- 172.21.15.16:0/1052910325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 msgr2=0x55dbf3044510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf3044510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50fcb1e640 1 -- 172.21.15.16:0/1052910325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf2cb32d0 con 0x55dbf303fb30 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 0x55dbf304ba20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f50f0035000 tx=0x7f50f0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6aa67f3e0c0be server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50d77fe640 1 -- 172.21.15.16:0/1052910325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0051070 con 0x55dbf303fb30 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50d77fe640 1 -- 172.21.15.16:0/1052910325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f50f0003e00 con 0x55dbf303fb30 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f510116b740 1 -- 172.21.15.16:0/1052910325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e8009e30 con 0x55dbf303fb30 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50d77fe640 1 -- 172.21.15.16:0/1052910325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f50f0004b40 con 0x55dbf303fb30 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f510116b740 10 client.15093 fetch_fsmap learned FSMap version 24 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f510116b740 1 -- 172.21.15.16:0/1052910325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dbf2d7d6b0 con 0x55dbf303fb30 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.899+0000 7f50d77fe640 1 -- 172.21.15.16:0/1052910325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f50e8009e30 con 0x55dbf303fb30 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.900+0000 7f50d77fe640 1 -- 172.21.15.16:0/1052910325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f50f004c070 con 0x55dbf303fb30 2024-03-27T06:42:58.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.900+0000 7f510116b740 10 client.15093 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.900+0000 7f510116b740 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.900+0000 7f510116b740 20 client.15093 populate_metadata read hostname 'smithi016' 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.900+0000 7f510116b740 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55dbf3050fa0 0x55dbf3052410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.900+0000 7f510116b740 4 client.15093 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.900+0000 7f510116b740 1 -- 172.21.15.16:0/1052910325 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf2fac550 con 0x55dbf3050fa0 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.900+0000 7f50f7fff640 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55dbf3050fa0 0x55dbf3052410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.900+0000 7f50f7fff640 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55dbf3050fa0 0x55dbf3052410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=263d8b758c281afd server_cookie=f699b097edcfeb5 in_seq=0 out_seq=0 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.900+0000 7f50d77fe640 10 client.15093 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f50d77fe640 1 -- 172.21.15.16:0/1052910325 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dbf2fac550 con 0x55dbf3050fa0 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f50d77fe640 10 client.15093 handle_command_reply: tid=0 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 1 client.15093 shutdown 2024-03-27T06:42:58.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 1 -- 172.21.15.16:0/1052910325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 msgr2=0x55dbf304ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 0x55dbf304ba20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f50f0035000 tx=0x7f50f0009500 comp rx=0 tx=0).stop 2024-03-27T06:42:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 1 -- 172.21.15.16:0/1052910325 shutdown_connections 2024-03-27T06:42:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55dbf3050fa0 0x55dbf3052410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 0x55dbf304ba20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303a9c0 0x55dbf3044a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 1 --2- 172.21.15.16:0/1052910325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf3044510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f50d77fe640 0 client.15093 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:42:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 1 -- 172.21.15.16:0/1052910325 >> 172.21.15.16:0/1052910325 conn(0x55dbf3001b20 msgr2=0x55dbf3034b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 1 -- 172.21.15.16:0/1052910325 shutdown_connections 2024-03-27T06:42:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 1 -- 172.21.15.16:0/1052910325 wait complete. 2024-03-27T06:42:58.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.901+0000 7f510116b740 20 client.15093 trim_cache size 0 max 16384 2024-03-27T06:42:58.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.919+0000 7f510116b740 1 Processor -- start 2024-03-27T06:42:58.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.919+0000 7f510116b740 1 -- start start 2024-03-27T06:42:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.919+0000 7f510116b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf304c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.919+0000 7f510116b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf304b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f510116b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303fb30 0x55dbf3044510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f510116b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dbf3050750 con 0x55dbf303a9c0 2024-03-27T06:42:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f510116b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55dbf3050e20 con 0x55dbf3039fd0 2024-03-27T06:42:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f510116b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dbf30514f0 con 0x55dbf303fb30 2024-03-27T06:42:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf304b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf304b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48708/0 (socket says 172.21.15.16:48708) 2024-03-27T06:42:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50f77fe640 1 -- 172.21.15.16:0/2365433259 learned_addr learned my addr 172.21.15.16:0/2365433259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50f7fff640 1 --2- 172.21.15.16:0/2365433259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf304c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/2365433259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303fb30 0x55dbf3044510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50f77fe640 1 -- 172.21.15.16:0/2365433259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303fb30 msgr2=0x55dbf3044510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50f77fe640 1 --2- 172.21.15.16:0/2365433259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303fb30 0x55dbf3044510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50f77fe640 1 -- 172.21.15.16:0/2365433259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 msgr2=0x55dbf304c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50f77fe640 1 --2- 172.21.15.16:0/2365433259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf304c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50f77fe640 1 -- 172.21.15.16:0/2365433259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf30352a0 con 0x55dbf303a9c0 2024-03-27T06:42:58.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50f77fe640 1 --2- 172.21.15.16:0/2365433259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf304b940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0cdbe0 tx=0x7f50ec087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5248e84b186b69 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50ddffb640 1 -- 172.21.15.16:0/2365433259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0b0e60 con 0x55dbf303a9c0 2024-03-27T06:42:58.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50ddffb640 1 -- 172.21.15.16:0/2365433259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f50ec09ccb0 con 0x55dbf303a9c0 2024-03-27T06:42:58.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.920+0000 7f50ddffb640 1 -- 172.21.15.16:0/2365433259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec09a980 con 0x55dbf303a9c0 2024-03-27T06:42:58.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 -- 172.21.15.16:0/2365433259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 msgr2=0x55dbf304b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 --2- 172.21.15.16:0/2365433259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf304b940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0cdbe0 tx=0x7f50ec087040 comp rx=0 tx=0).stop 2024-03-27T06:42:58.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 -- 172.21.15.16:0/2365433259 shutdown_connections 2024-03-27T06:42:58.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 --2- 172.21.15.16:0/2365433259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf303fb30 0x55dbf3044510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 --2- 172.21.15.16:0/2365433259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303a9c0 0x55dbf304b940 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 --2- 172.21.15.16:0/2365433259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf3039fd0 0x55dbf304c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 -- 172.21.15.16:0/2365433259 >> 172.21.15.16:0/2365433259 conn(0x55dbf3001b20 msgr2=0x55dbf30306a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 -- 172.21.15.16:0/2365433259 shutdown_connections 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 -- 172.21.15.16:0/2365433259 wait complete. 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 -- start start 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 Processor -- start 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf3039fd0 0x55dbf3047760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf303a9c0 0x55dbf3047ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 0x55dbf304ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dbf3052290 con 0x55dbf303fb30 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55dbf3051bc0 con 0x55dbf303a9c0 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.921+0000 7f510116b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dbf3052960 con 0x55dbf3039fd0 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f510116b740 10 client.0 _ll_register_callbacks cb 0x7f50ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:42:58.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50fcb1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 0x55dbf304ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50f7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf3039fd0 0x55dbf3047760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50f7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf3039fd0 0x55dbf3047760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:57986/0 (socket says 172.21.15.16:57986) 2024-03-27T06:42:58.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50f7fff640 1 -- 172.21.15.16:0/1760624262 learned_addr learned my addr 172.21.15.16:0/1760624262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:58.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50f77fe640 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf303a9c0 0x55dbf3047ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50fcb1e640 1 -- 172.21.15.16:0/1760624262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf3039fd0 msgr2=0x55dbf3047760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf3039fd0 0x55dbf3047760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50fcb1e640 1 -- 172.21.15.16:0/1760624262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf303a9c0 msgr2=0x55dbf3047ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:58.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf303a9c0 0x55dbf3047ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50fcb1e640 1 -- 172.21.15.16:0/1760624262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf304b210 con 0x55dbf303fb30 2024-03-27T06:42:58.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50fcb1e640 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 0x55dbf304ac20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f50f0036040 tx=0x7f50f0005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d930e296ceee8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:58.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:42:58.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50d77fe640 1 -- 172.21.15.16:0/1760624262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0065070 con 0x55dbf303fb30 2024-03-27T06:42:58.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50d77fe640 1 -- 172.21.15.16:0/1760624262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f50f003a040 con 0x55dbf303fb30 2024-03-27T06:42:58.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f510116b740 1 -- 172.21.15.16:0/1760624262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50ec09ccb0 con 0x55dbf303fb30 2024-03-27T06:42:58.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f50d77fe640 1 -- 172.21.15.16:0/1760624262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f50f0051040 con 0x55dbf303fb30 2024-03-27T06:42:58.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f510116b740 10 client.15099 fetch_fsmap learned FSMap version 24 2024-03-27T06:42:58.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.922+0000 7f510116b740 1 -- 172.21.15.16:0/1760624262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dbf30b7220 con 0x55dbf303fb30 2024-03-27T06:42:58.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f50d77fe640 1 -- 172.21.15.16:0/1760624262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f50ec09ccb0 con 0x55dbf303fb30 2024-03-27T06:42:58.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f50d77fe640 1 -- 172.21.15.16:0/1760624262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f50f004c040 con 0x55dbf303fb30 2024-03-27T06:42:58.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f510116b740 10 client.15099 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:42:58.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f510116b740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-27T06:42:58.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f510116b740 20 client.15099 populate_metadata read hostname 'smithi016' 2024-03-27T06:42:58.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f510116b740 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55dbf30bbe50 0x55dbf30bab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:58.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f510116b740 4 client.15099 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T06:42:58.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f510116b740 1 -- 172.21.15.16:0/1760624262 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "status"}) -- 0x55dbf2fa68a0 con 0x55dbf30bbe50 2024-03-27T06:42:58.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f50f7fff640 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55dbf30bbe50 0x55dbf30bab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f50d77fe640 10 client.15099 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f50f7fff640 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55dbf30bbe50 0x55dbf30bab60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38b279e837f9f42a server_cookie=e47a7586358d473b in_seq=0 out_seq=0 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f50d77fe640 1 -- 172.21.15.16:0/1760624262 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dbf2fa68a0 con 0x55dbf30bbe50 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f50d77fe640 10 client.15099 handle_command_reply: tid=0 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.923+0000 7f510116b740 1 client.15099 shutdown 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f510116b740 1 -- 172.21.15.16:0/1760624262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 msgr2=0x55dbf304ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f510116b740 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 0x55dbf304ac20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f50f0036040 tx=0x7f50f0005e30 comp rx=0 tx=0).stop 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f510116b740 1 -- 172.21.15.16:0/1760624262 shutdown_connections 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f510116b740 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55dbf30bbe50 0x55dbf30bab60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f510116b740 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbf303fb30 0x55dbf304ac20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f510116b740 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dbf303a9c0 0x55dbf3047ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50d77fe640 0 client.15099 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f510116b740 1 --2- 172.21.15.16:0/1760624262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbf3039fd0 0x55dbf3047760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f510116b740 1 -- 172.21.15.16:0/1760624262 >> 172.21.15.16:0/1760624262 conn(0x55dbf3001b20 msgr2=0x55dbf303be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f510116b740 1 -- 172.21.15.16:0/1760624262 shutdown_connections 2024-03-27T06:42:58.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f510116b740 1 -- 172.21.15.16:0/1760624262 wait complete. 2024-03-27T06:42:58.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f510116b740 20 client.15099 trim_cache size 0 max 16384 2024-03-27T06:42:58.929 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:42:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:42:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-03-27T06:42:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24493, 2024-03-27T06:42:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-03-27T06:42:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-03-27T06:42:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-03-27T06:42:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 86.437704405000005, 2024-03-27T06:42:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-03-27T06:42:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-03-27T06:42:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-03-27T06:42:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 103.197375024 2024-03-27T06:42:58.930 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:42:58.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 -- 172.21.15.16:0/558494553 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f50e00a3610 msgr2=0x7f50e00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f50e00a3610 0x7f50e00a5ad0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f50f0004af0 tx=0x7f50f00092e0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 -- 172.21.15.16:0/558494553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f8101c50 msgr2=0x7f50f81984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:58.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f8101c50 0x7f50f81984a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00e9f0 tx=0x7f50ec00edf0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 -- 172.21.15.16:0/558494553 shutdown_connections 2024-03-27T06:42:58.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f50e00a3610 0x7f50e00a5ad0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50f8102610 0x7f50f819f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f8101c50 0x7f50f81984a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 --2- 172.21.15.16:0/558494553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50f8100a60 0x7f50f8197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:58.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 -- 172.21.15.16:0/558494553 >> 172.21.15.16:0/558494553 conn(0x7f50f80fbff0 msgr2=0x7f50f8104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:58.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 -- 172.21.15.16:0/558494553 shutdown_connections 2024-03-27T06:42:58.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:58.924+0000 7f50fe581640 1 -- 172.21.15.16:0/558494553 wait complete. 2024-03-27T06:42:58.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-27T06:42:59.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.040+0000 7f74ff5a1640 1 -- 172.21.15.16:0/1043590682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f8104f00 msgr2=0x7f74f81072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.040+0000 7f74ff5a1640 1 --2- 172.21.15.16:0/1043590682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f8104f00 0x7f74f81072f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009900 tx=0x7f74ec02bc30 comp rx=0 tx=0).stop 2024-03-27T06:42:59.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.040+0000 7f74ff5a1640 1 -- 172.21.15.16:0/1043590682 shutdown_connections 2024-03-27T06:42:59.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.040+0000 7f74ff5a1640 1 --2- 172.21.15.16:0/1043590682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74f8107830 0x7f74f8109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.040+0000 7f74ff5a1640 1 --2- 172.21.15.16:0/1043590682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f8104f00 0x7f74f81072f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.040+0000 7f74ff5a1640 1 --2- 172.21.15.16:0/1043590682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f81025d0 0x7f74f81049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.040+0000 7f74ff5a1640 1 -- 172.21.15.16:0/1043590682 >> 172.21.15.16:0/1043590682 conn(0x7f74f80fbfe0 msgr2=0x7f74f80fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.040+0000 7f74ff5a1640 1 -- 172.21.15.16:0/1043590682 shutdown_connections 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.040+0000 7f74ff5a1640 1 -- 172.21.15.16:0/1043590682 wait complete. 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74ff5a1640 1 Processor -- start 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74ff5a1640 1 -- start start 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74ff5a1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74f81025d0 0x7f74f8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74ff5a1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f8104f00 0x7f74f81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74ff5a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f8107830 0x7f74f819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74ff5a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74f810d8b0 con 0x7f74f8107830 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74ff5a1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f74f810d730 con 0x7f74f81025d0 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74ff5a1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74f810da30 con 0x7f74f8104f00 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74fdb3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f8107830 0x7f74f819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74fdb3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f8107830 0x7f74f819f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48738/0 (socket says 172.21.15.16:48738) 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74fdb3e640 1 -- 172.21.15.16:0/2053619812 learned_addr learned my addr 172.21.15.16:0/2053619812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74fd33d640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74f81025d0 0x7f74f8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f8104f00 0x7f74f81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74fdb3e640 1 -- 172.21.15.16:0/2053619812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f8104f00 msgr2=0x7f74f81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74fdb3e640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f8104f00 0x7f74f81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.041+0000 7f74fdb3e640 1 -- 172.21.15.16:0/2053619812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74f81025d0 msgr2=0x7f74f8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.042+0000 7f74fdb3e640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74f81025d0 0x7f74f8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.042+0000 7f74fdb3e640 1 -- 172.21.15.16:0/2053619812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f819fbd0 con 0x7f74f8107830 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.042+0000 7f74fdb3e640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f8107830 0x7f74f819f560 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f74e800eb10 tx=0x7f74e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.042+0000 7f74f67fc640 1 -- 172.21.15.16:0/2053619812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8004b20 con 0x7f74f8107830 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.042+0000 7f74f67fc640 1 -- 172.21.15.16:0/2053619812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f74e8004cc0 con 0x7f74f8107830 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.042+0000 7f74ff5a1640 1 -- 172.21.15.16:0/2053619812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f819fec0 con 0x7f74f8107830 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.042+0000 7f74f67fc640 1 -- 172.21.15.16:0/2053619812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e80107f0 con 0x7f74f8107830 2024-03-27T06:42:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.042+0000 7f74ff5a1640 1 -- 172.21.15.16:0/2053619812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f81a0330 con 0x7f74f8107830 2024-03-27T06:42:59.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.044+0000 7f74f67fc640 1 -- 172.21.15.16:0/2053619812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f74e8005020 con 0x7f74f8107830 2024-03-27T06:42:59.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.044+0000 7f750218e740 1 Processor -- start 2024-03-27T06:42:59.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.044+0000 7f750218e740 1 -- start start 2024-03-27T06:42:59.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f750218e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3ee4fd0 0x5592a3ee53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74f67fc640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f74e00a3610 0x7f74e00a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f750218e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee59c0 0x5592a3eea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f750218e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3eeab30 0x5592a3eecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f750218e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592a3e4a010 con 0x5592a3eeab30 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f750218e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5592a3df2770 con 0x5592a3ee59c0 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f750218e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5592a3e45fc0 con 0x5592a3ee4fd0 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fd33d640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f74e00a3610 0x7f74e00a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fcb3c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee59c0 0x5592a3eea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74f67fc640 1 -- 172.21.15.16:0/2053619812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f74e80d0290 con 0x7f74f8107830 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fcb3c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee59c0 0x5592a3eea4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:49796/0 (socket says 172.21.15.16:49796) 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fdb3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3eeab30 0x5592a3eecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fcb3c640 1 -- 172.21.15.16:0/1293846735 learned_addr learned my addr 172.21.15.16:0/1293846735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/1293846735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3ee4fd0 0x5592a3ee53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fcb3c640 1 -- 172.21.15.16:0/1293846735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3ee4fd0 msgr2=0x5592a3ee53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/1293846735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3ee4fd0 0x5592a3ee53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fcb3c640 1 -- 172.21.15.16:0/1293846735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3eeab30 msgr2=0x5592a3eecfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/1293846735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3eeab30 0x5592a3eecfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fcb3c640 1 -- 172.21.15.16:0/1293846735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a3e11330 con 0x5592a3ee59c0 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fdb3e640 1 --2- 172.21.15.16:0/1293846735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3eeab30 0x5592a3eecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fd33d640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f74e00a3610 0x7f74e00a5ad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009bd0 tx=0x7f74f0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/1293846735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee59c0 0x5592a3eea4c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5592a3ee6220 tx=0x7f74ec0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32f01dfd18c1d9ae server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74df7fe640 1 -- 172.21.15.16:0/1293846735 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec04a070 con 0x5592a3ee59c0 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74df7fe640 1 -- 172.21.15.16:0/1293846735 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f74ec0035f0 con 0x5592a3ee59c0 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.045+0000 7f74df7fe640 1 -- 172.21.15.16:0/1293846735 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec034720 con 0x5592a3ee59c0 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.046+0000 7f750218e740 1 -- 172.21.15.16:0/1293846735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee59c0 msgr2=0x5592a3eea4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.046+0000 7f750218e740 1 --2- 172.21.15.16:0/1293846735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee59c0 0x5592a3eea4c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5592a3ee6220 tx=0x7f74ec0056d0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.046+0000 7f750218e740 1 -- 172.21.15.16:0/1293846735 shutdown_connections 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.046+0000 7f750218e740 1 --2- 172.21.15.16:0/1293846735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3eeab30 0x5592a3eecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.046+0000 7f750218e740 1 --2- 172.21.15.16:0/1293846735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee59c0 0x5592a3eea4c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.046+0000 7f750218e740 1 --2- 172.21.15.16:0/1293846735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3ee4fd0 0x5592a3ee53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.046+0000 7f750218e740 1 -- 172.21.15.16:0/1293846735 >> 172.21.15.16:0/1293846735 conn(0x5592a3eacb20 msgr2=0x5592a3ee0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.046+0000 7f750218e740 1 -- 172.21.15.16:0/1293846735 shutdown_connections 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.046+0000 7f750218e740 1 -- 172.21.15.16:0/1293846735 wait complete. 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f750218e740 1 -- start start 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f750218e740 1 Processor -- start 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f750218e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee4fd0 0x5592a3eef580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f750218e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3eefac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f750218e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3eeab30 0x5592a3ef6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f750218e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592a3eeda80 con 0x5592a3ee59c0 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f750218e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5592a3e05b30 con 0x5592a3ee4fd0 2024-03-27T06:42:59.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f750218e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5592a3eedc00 con 0x5592a3eeab30 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f750218e740 10 client.0 _ll_register_callbacks cb 0x5592a3e411a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f74fcb3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3eefac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f74fcb3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3eefac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48764/0 (socket says 172.21.15.16:48764) 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f74fcb3c640 1 -- 172.21.15.16:0/532141273 learned_addr learned my addr 172.21.15.16:0/532141273 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f74fd33d640 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee4fd0 0x5592a3eef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f74fdb3e640 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3eeab30 0x5592a3ef6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f74fcb3c640 1 -- 172.21.15.16:0/532141273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3eeab30 msgr2=0x5592a3ef6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3eeab30 0x5592a3ef6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f74fcb3c640 1 -- 172.21.15.16:0/532141273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee4fd0 msgr2=0x5592a3eef580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee4fd0 0x5592a3eef580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f74fcb3c640 1 -- 172.21.15.16:0/532141273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a3b5e2d0 con 0x5592a3ee59c0 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.047+0000 7f74fdb3e640 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3eeab30 0x5592a3ef6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3eefac0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f74ec02c170 tx=0x7f74ec003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3cfb4f3e3b66a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f74dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:42:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f74dcff9640 1 -- 172.21.15.16:0/532141273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec04a070 con 0x5592a3ee59c0 2024-03-27T06:42:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f74dcff9640 1 -- 172.21.15.16:0/532141273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f74ec0344e0 con 0x5592a3ee59c0 2024-03-27T06:42:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f750218e740 1 -- 172.21.15.16:0/532141273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74ec0035f0 con 0x5592a3ee59c0 2024-03-27T06:42:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f74dcff9640 1 -- 172.21.15.16:0/532141273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f74ec0346e0 con 0x5592a3ee59c0 2024-03-27T06:42:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f750218e740 10 client.15111 fetch_fsmap learned FSMap version 24 2024-03-27T06:42:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f750218e740 1 -- 172.21.15.16:0/532141273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5592a3c286b0 con 0x5592a3ee59c0 2024-03-27T06:42:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f74dcff9640 1 -- 172.21.15.16:0/532141273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f74ec0035f0 con 0x5592a3ee59c0 2024-03-27T06:42:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f74dcff9640 1 -- 172.21.15.16:0/532141273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f74ec043a20 con 0x5592a3ee59c0 2024-03-27T06:42:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f750218e740 10 client.15111 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:42:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f750218e740 10 client.15111 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-27T06:42:59.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f750218e740 20 client.15111 populate_metadata read hostname 'smithi016' 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f750218e740 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x5592a3efac60 0x5592a3efce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f750218e740 4 client.15111 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.048+0000 7f750218e740 1 -- 172.21.15.16:0/532141273 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592a3e57550 con 0x5592a3efac60 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.049+0000 7f74fd33d640 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x5592a3efac60 0x5592a3efce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.049+0000 7f74fd33d640 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x5592a3efac60 0x5592a3efce00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbf4edc62a7c1cde server_cookie=da423722f95e497c in_seq=0 out_seq=0 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.049+0000 7f74dcff9640 10 client.15111 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f74dcff9640 1 -- 172.21.15.16:0/532141273 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5592a3e57550 con 0x5592a3efac60 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f74dcff9640 10 client.15111 handle_command_reply: tid=0 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 1 client.15111 shutdown 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 1 -- 172.21.15.16:0/532141273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 msgr2=0x5592a3eefac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3eefac0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f74ec02c170 tx=0x7f74ec003d00 comp rx=0 tx=0).stop 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 1 -- 172.21.15.16:0/532141273 shutdown_connections 2024-03-27T06:42:59.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x5592a3efac60 0x5592a3efce00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3eeab30 0x5592a3ef6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3eefac0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f74dcff9640 0 client.15111 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:42:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 1 --2- 172.21.15.16:0/532141273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee4fd0 0x5592a3eef580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 1 -- 172.21.15.16:0/532141273 >> 172.21.15.16:0/532141273 conn(0x5592a3eacb20 msgr2=0x5592a3edfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 1 -- 172.21.15.16:0/532141273 shutdown_connections 2024-03-27T06:42:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 1 -- 172.21.15.16:0/532141273 wait complete. 2024-03-27T06:42:59.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.050+0000 7f750218e740 20 client.15111 trim_cache size 0 max 16384 2024-03-27T06:42:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f750218e740 1 Processor -- start 2024-03-27T06:42:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f750218e740 1 -- start start 2024-03-27T06:42:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f750218e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3ee4fd0 0x5592a3ee3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f750218e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f750218e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3eeab30 0x5592a3ee40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f750218e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592a3efb140 con 0x5592a3ee59c0 2024-03-27T06:42:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f750218e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5592a3efb810 con 0x5592a3eeab30 2024-03-27T06:42:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f750218e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5592a3efbee0 con 0x5592a3ee4fd0 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f74fcb3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f74fcb3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48766/0 (socket says 172.21.15.16:48766) 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f74fcb3c640 1 -- 172.21.15.16:0/3183401668 learned_addr learned my addr 172.21.15.16:0/3183401668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.068+0000 7f74fd33d640 1 --2- 172.21.15.16:0/3183401668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3ee4fd0 0x5592a3ee3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.069+0000 7f74fdb3e640 1 --2- 172.21.15.16:0/3183401668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3eeab30 0x5592a3ee40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.069+0000 7f74fcb3c640 1 -- 172.21.15.16:0/3183401668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3ee4fd0 msgr2=0x5592a3ee3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.069+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/3183401668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3ee4fd0 0x5592a3ee3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.069+0000 7f74fcb3c640 1 -- 172.21.15.16:0/3183401668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3eeab30 msgr2=0x5592a3ee40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.069+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/3183401668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3eeab30 0x5592a3ee40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.069+0000 7f74fcb3c640 1 -- 172.21.15.16:0/3183401668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a3ee0d70 con 0x5592a3ee59c0 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.069+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/3183401668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef6ac0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f74ec000c00 tx=0x7f74ec02cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3ebd6a3911557f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.069+0000 7f74deffd640 1 -- 172.21.15.16:0/3183401668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec04a070 con 0x5592a3ee59c0 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.069+0000 7f74deffd640 1 -- 172.21.15.16:0/3183401668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f74ec054030 con 0x5592a3ee59c0 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.069+0000 7f74deffd640 1 -- 172.21.15.16:0/3183401668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec004310 con 0x5592a3ee59c0 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 -- 172.21.15.16:0/3183401668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 msgr2=0x5592a3ef6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 --2- 172.21.15.16:0/3183401668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef6ac0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f74ec000c00 tx=0x7f74ec02cc00 comp rx=0 tx=0).stop 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 -- 172.21.15.16:0/3183401668 shutdown_connections 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 --2- 172.21.15.16:0/3183401668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3eeab30 0x5592a3ee40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 --2- 172.21.15.16:0/3183401668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef6ac0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 --2- 172.21.15.16:0/3183401668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3ee4fd0 0x5592a3ee3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 -- 172.21.15.16:0/3183401668 >> 172.21.15.16:0/3183401668 conn(0x5592a3eacb20 msgr2=0x5592a3eedfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 -- 172.21.15.16:0/3183401668 shutdown_connections 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 -- 172.21.15.16:0/3183401668 wait complete. 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 -- start start 2024-03-27T06:42:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 Processor -- start 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee4fd0 0x5592a3ef29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3eeab30 0x5592a3ef5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592a3efcc80 con 0x5592a3ee59c0 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5592a3efc5b0 con 0x5592a3ee4fd0 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5592a3efd350 con 0x5592a3eeab30 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f750218e740 10 client.0 _ll_register_callbacks cb 0x7f74ec048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f74fcb3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f74fcb3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48772/0 (socket says 172.21.15.16:48772) 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f74fcb3c640 1 -- 172.21.15.16:0/2057466148 learned_addr learned my addr 172.21.15.16:0/2057466148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.070+0000 7f74fd33d640 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee4fd0 0x5592a3ef29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74fdb3e640 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3eeab30 0x5592a3ef5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74fcb3c640 1 -- 172.21.15.16:0/2057466148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3eeab30 msgr2=0x5592a3ef5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3eeab30 0x5592a3ef5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74fcb3c640 1 -- 172.21.15.16:0/2057466148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee4fd0 msgr2=0x5592a3ef29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee4fd0 0x5592a3ef29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74fcb3c640 1 -- 172.21.15.16:0/2057466148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a3eed590 con 0x5592a3ee59c0 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74fcb3c640 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef2ef0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f74ec02c170 tx=0x7f74ec005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffbde76a1a8a9a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74dcff9640 1 -- 172.21.15.16:0/2057466148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec04a040 con 0x5592a3ee59c0 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74dcff9640 1 -- 172.21.15.16:0/2057466148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f74ec0344e0 con 0x5592a3ee59c0 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f750218e740 1 -- 172.21.15.16:0/2057466148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74ec054030 con 0x5592a3ee59c0 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74dcff9640 1 -- 172.21.15.16:0/2057466148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f74ec004310 con 0x5592a3ee59c0 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f750218e740 10 client.15117 fetch_fsmap learned FSMap version 24 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f750218e740 1 -- 172.21.15.16:0/2057466148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5592a3f61c40 con 0x5592a3ee59c0 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74dcff9640 1 -- 172.21.15.16:0/2057466148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f74ec054030 con 0x5592a3ee59c0 2024-03-27T06:42:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.071+0000 7f74dcff9640 1 -- 172.21.15.16:0/2057466148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f74ec069030 con 0x5592a3ee59c0 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 10 client.15117 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 20 client.15117 populate_metadata read hostname 'smithi016' 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x5592a3f666d0 0x5592a3f653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 4 client.15117 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 1 -- 172.21.15.16:0/2057466148 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "status"}) -- 0x5592a3e518a0 con 0x5592a3f666d0 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f74fd33d640 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x5592a3f666d0 0x5592a3f653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f74fd33d640 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x5592a3f666d0 0x5592a3f653e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=995d6ea724cfb5 server_cookie=d1b27ba672e15937 in_seq=0 out_seq=0 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f74dcff9640 10 client.15117 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f74dcff9640 1 -- 172.21.15.16:0/2057466148 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5592a3e518a0 con 0x5592a3f666d0 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f74dcff9640 10 client.15117 handle_command_reply: tid=0 2024-03-27T06:42:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 1 client.15117 shutdown 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 1 -- 172.21.15.16:0/2057466148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 msgr2=0x5592a3ef2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef2ef0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f74ec02c170 tx=0x7f74ec005dd0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 1 -- 172.21.15.16:0/2057466148 shutdown_connections 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x5592a3f666d0 0x5592a3f653e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5592a3eeab30 0x5592a3ef5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592a3ee59c0 0x5592a3ef2ef0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 1 --2- 172.21.15.16:0/2057466148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5592a3ee4fd0 0x5592a3ef29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f74dcff9640 0 client.15117 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.072+0000 7f750218e740 1 -- 172.21.15.16:0/2057466148 >> 172.21.15.16:0/2057466148 conn(0x5592a3eacb20 msgr2=0x5592a3ee3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f750218e740 1 -- 172.21.15.16:0/2057466148 shutdown_connections 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f750218e740 1 -- 172.21.15.16:0/2057466148 wait complete. 2024-03-27T06:42:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f750218e740 20 client.15117 trim_cache size 0 max 16384 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24500, 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 87.596399073000001, 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-03-27T06:42:59.076 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 106.08589577399999 2024-03-27T06:42:59.077 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:42:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 -- 172.21.15.16:0/2053619812 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f74e00a3610 msgr2=0x7f74e00a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f74e00a3610 0x7f74e00a5ad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009bd0 tx=0x7f74f0005f90 comp rx=0 tx=0).stop 2024-03-27T06:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 -- 172.21.15.16:0/2053619812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f8107830 msgr2=0x7f74f819f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f8107830 0x7f74f819f560 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f74e800eb10 tx=0x7f74e800c2d0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 -- 172.21.15.16:0/2053619812 shutdown_connections 2024-03-27T06:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f74e00a3610 0x7f74e00a5ad0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f8107830 0x7f74f819f560 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f8104f00 0x7f74f81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 --2- 172.21.15.16:0/2053619812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74f81025d0 0x7f74f8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 -- 172.21.15.16:0/2053619812 >> 172.21.15.16:0/2053619812 conn(0x7f74f80fbfe0 msgr2=0x7f74f81030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 -- 172.21.15.16:0/2053619812 shutdown_connections 2024-03-27T06:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.073+0000 7f74ff5a1640 1 -- 172.21.15.16:0/2053619812 wait complete. 2024-03-27T06:42:59.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-27T06:42:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.188+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1534758971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e8103710 msgr2=0x7fc0e8109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.188+0000 7fc0efa8d640 1 --2- 172.21.15.16:0/1534758971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e8103710 0x7fc0e8109fa0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009ca0 tx=0x7fc0e002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.188+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1534758971 shutdown_connections 2024-03-27T06:42:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.188+0000 7fc0efa8d640 1 --2- 172.21.15.16:0/1534758971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e8103710 0x7fc0e8109fa0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.188+0000 7fc0efa8d640 1 --2- 172.21.15.16:0/1534758971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0e8102d50 0x7fc0e81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.188+0000 7fc0efa8d640 1 --2- 172.21.15.16:0/1534758971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0e8101b60 0x7fc0e8101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.188+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1534758971 >> 172.21.15.16:0/1534758971 conn(0x7fc0e80fd0f0 msgr2=0x7fc0e80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.188+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1534758971 shutdown_connections 2024-03-27T06:42:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.188+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1534758971 wait complete. 2024-03-27T06:42:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.188+0000 7fc0efa8d640 1 Processor -- start 2024-03-27T06:42:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.188+0000 7fc0efa8d640 1 -- start start 2024-03-27T06:42:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0efa8d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0e8101b60 0x7fc0e8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0efa8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e8102d50 0x7fc0e8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0efa8d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0e8103710 0x7fc0e819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0efa8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e810d910 con 0x7fc0e8102d50 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0efa8d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e810d790 con 0x7fc0e8101b60 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0efa8d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e810da90 con 0x7fc0e8103710 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0ed028640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e8102d50 0x7fc0e8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0ed028640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e8102d50 0x7fc0e8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48792/0 (socket says 172.21.15.16:48792) 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0ed028640 1 -- 172.21.15.16:0/1089540209 learned_addr learned my addr 172.21.15.16:0/1089540209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0ed829640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0e8101b60 0x7fc0e8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0ee02a640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0e8103710 0x7fc0e819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0ed028640 1 -- 172.21.15.16:0/1089540209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0e8103710 msgr2=0x7fc0e819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0ed028640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0e8103710 0x7fc0e819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0ed028640 1 -- 172.21.15.16:0/1089540209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0e8101b60 msgr2=0x7fc0e8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0ed028640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0e8101b60 0x7fc0e8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0ed028640 1 -- 172.21.15.16:0/1089540209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e819fb10 con 0x7fc0e8102d50 2024-03-27T06:42:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0ed028640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e8102d50 0x7fc0e8198430 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800e9f0 tx=0x7fc0d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0deffd640 1 -- 172.21.15.16:0/1089540209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d800cd80 con 0x7fc0e8102d50 2024-03-27T06:42:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0deffd640 1 -- 172.21.15.16:0/1089540209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc0d8005520 con 0x7fc0e8102d50 2024-03-27T06:42:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1089540209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e819fe00 con 0x7fc0e8102d50 2024-03-27T06:42:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0deffd640 1 -- 172.21.15.16:0/1089540209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8010690 con 0x7fc0e8102d50 2024-03-27T06:42:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.189+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1089540209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e81a0270 con 0x7fc0e8102d50 2024-03-27T06:42:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.191+0000 7fc0deffd640 1 -- 172.21.15.16:0/1089540209 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc0d80046c0 con 0x7fc0e8102d50 2024-03-27T06:42:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.192+0000 7fc0f2671740 1 Processor -- start 2024-03-27T06:42:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.192+0000 7fc0f2671740 1 -- start start 2024-03-27T06:42:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.192+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b389bfd0 0x5568b389c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.192+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.192+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38a1b30 0x5568b38a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.192+0000 7fc0f2671740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5568b3801010 con 0x5568b389c9c0 2024-03-27T06:42:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.192+0000 7fc0f2671740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5568b37a9770 con 0x5568b38a1b30 2024-03-27T06:42:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.192+0000 7fc0f2671740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5568b37fcfc0 con 0x5568b389bfd0 2024-03-27T06:42:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.192+0000 7fc0deffd640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0d40a3610 0x7fc0d40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ed829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ed028640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38a1b30 0x5568b38a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ee02a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b389bfd0 0x5568b389c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ed829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48804/0 (socket says 172.21.15.16:48804) 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ed829640 1 -- 172.21.15.16:0/13220114 learned_addr learned my addr 172.21.15.16:0/13220114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ee02a640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0d40a3610 0x7fc0d40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0deffd640 1 -- 172.21.15.16:0/1089540209 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc0d80d00d0 con 0x7fc0e8102d50 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ed829640 1 -- 172.21.15.16:0/13220114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b389bfd0 msgr2=0x5568b389c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ed829640 1 --2- 172.21.15.16:0/13220114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b389bfd0 0x5568b389c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ed829640 1 -- 172.21.15.16:0/13220114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38a1b30 msgr2=0x5568b38a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ed829640 1 --2- 172.21.15.16:0/13220114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38a1b30 0x5568b38a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ed829640 1 -- 172.21.15.16:0/13220114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b37c8330 con 0x5568b389c9c0 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ed829640 1 --2- 172.21.15.16:0/13220114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a14c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009c80 tx=0x7fc0e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a70b1406f48f392 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0ee02a640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0d40a3610 0x7fc0d40a5ad0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005e30 tx=0x7fc0e0032670 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0d3fff640 1 -- 172.21.15.16:0/13220114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e401e070 con 0x5568b389c9c0 2024-03-27T06:42:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0d3fff640 1 -- 172.21.15.16:0/13220114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc0e4009e30 con 0x5568b389c9c0 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.193+0000 7fc0d3fff640 1 -- 172.21.15.16:0/13220114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400eac0 con 0x5568b389c9c0 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.194+0000 7fc0f2671740 1 -- 172.21.15.16:0/13220114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 msgr2=0x5568b38a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.194+0000 7fc0f2671740 1 --2- 172.21.15.16:0/13220114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a14c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009c80 tx=0x7fc0e4011810 comp rx=0 tx=0).stop 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.194+0000 7fc0f2671740 1 -- 172.21.15.16:0/13220114 shutdown_connections 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.194+0000 7fc0f2671740 1 --2- 172.21.15.16:0/13220114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38a1b30 0x5568b38a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.194+0000 7fc0f2671740 1 --2- 172.21.15.16:0/13220114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a14c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.194+0000 7fc0f2671740 1 --2- 172.21.15.16:0/13220114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b389bfd0 0x5568b389c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.194+0000 7fc0f2671740 1 -- 172.21.15.16:0/13220114 >> 172.21.15.16:0/13220114 conn(0x5568b3863b20 msgr2=0x5568b3897940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.194+0000 7fc0f2671740 1 -- 172.21.15.16:0/13220114 shutdown_connections 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.194+0000 7fc0f2671740 1 -- 172.21.15.16:0/13220114 wait complete. 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.194+0000 7fc0f2671740 1 -- start start 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.194+0000 7fc0f2671740 1 Processor -- start 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389bfd0 0x5568b38a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b38a1b30 0x5568b38ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0f2671740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5568b38a4ae0 con 0x5568b389c9c0 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0f2671740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5568b37bcb30 con 0x5568b389bfd0 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0f2671740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5568b38a4c60 con 0x5568b38a1b30 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0f2671740 10 client.0 _ll_register_callbacks cb 0x5568b37f81a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0ed028640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0ee02a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b38a1b30 0x5568b38ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0ed028640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48812/0 (socket says 172.21.15.16:48812) 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0ed028640 1 -- 172.21.15.16:0/3694527563 learned_addr learned my addr 172.21.15.16:0/3694527563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0ed829640 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389bfd0 0x5568b38a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0ed028640 1 -- 172.21.15.16:0/3694527563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b38a1b30 msgr2=0x5568b38ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0ed028640 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b38a1b30 0x5568b38ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0ed028640 1 -- 172.21.15.16:0/3694527563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389bfd0 msgr2=0x5568b38a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0ed028640 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389bfd0 0x5568b38a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0ed028640 1 -- 172.21.15.16:0/3694527563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b35152d0 con 0x5568b389c9c0 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0ed028640 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a6a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8098f80 tx=0x7fc0d809c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d34b7918b304a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.195+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3694527563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80b0e60 con 0x5568b389c9c0 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3694527563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc0d8085b40 con 0x5568b389c9c0 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0f2671740 1 -- 172.21.15.16:0/3694527563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e4009e30 con 0x5568b389c9c0 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3694527563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc0d809ccb0 con 0x5568b389c9c0 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0f2671740 10 client.15129 fetch_fsmap learned FSMap version 24 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0f2671740 1 -- 172.21.15.16:0/3694527563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5568b35df6b0 con 0x5568b389c9c0 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3694527563 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc0e4009e30 con 0x5568b389c9c0 2024-03-27T06:42:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3694527563 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fc0d80d2170 con 0x5568b389c9c0 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0f2671740 10 client.15129 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0f2671740 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0f2671740 20 client.15129 populate_metadata read hostname 'smithi016' 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5568b38b2f90 0x5568b38b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0f2671740 4 client.15129 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0f2671740 1 -- 172.21.15.16:0/3694527563 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568b380e550 con 0x5568b38b2f90 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.196+0000 7fc0ed829640 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5568b38b2f90 0x5568b38b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.197+0000 7fc0d17fa640 10 client.15129 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.197+0000 7fc0ed829640 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5568b38b2f90 0x5568b38b4400 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71dde451f5fbb2bf server_cookie=756c4525d8b02fad in_seq=0 out_seq=0 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.197+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3694527563 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5568b380e550 con 0x5568b38b2f90 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.197+0000 7fc0d17fa640 10 client.15129 handle_command_reply: tid=0 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.197+0000 7fc0f2671740 1 client.15129 shutdown 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0f2671740 1 -- 172.21.15.16:0/3694527563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 msgr2=0x5568b38a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a6a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8098f80 tx=0x7fc0d809c440 comp rx=0 tx=0).stop 2024-03-27T06:42:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0f2671740 1 -- 172.21.15.16:0/3694527563 shutdown_connections 2024-03-27T06:42:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5568b38b2f90 0x5568b38b4400 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b38a1b30 0x5568b38ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389c9c0 0x5568b38a6a20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3694527563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389bfd0 0x5568b38a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0d17fa640 0 client.15129 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:42:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0f2671740 1 -- 172.21.15.16:0/3694527563 >> 172.21.15.16:0/3694527563 conn(0x5568b3863b20 msgr2=0x5568b3896b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0f2671740 1 -- 172.21.15.16:0/3694527563 shutdown_connections 2024-03-27T06:42:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0f2671740 1 -- 172.21.15.16:0/3694527563 wait complete. 2024-03-27T06:42:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.198+0000 7fc0f2671740 20 client.15129 trim_cache size 0 max 16384 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0f2671740 1 Processor -- start 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0f2671740 1 -- start start 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b389c9c0 0x5568b38ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38a1b30 0x5568b389de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0f2671740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5568b38b2740 con 0x5568b389bfd0 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0f2671740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5568b38b2e10 con 0x5568b38a1b30 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0f2671740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5568b38b34e0 con 0x5568b389c9c0 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ed829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ed829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38a64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48826/0 (socket says 172.21.15.16:48826) 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ed829640 1 -- 172.21.15.16:0/2812792694 learned_addr learned my addr 172.21.15.16:0/2812792694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ed028640 1 --2- 172.21.15.16:0/2812792694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b389c9c0 0x5568b38ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ee02a640 1 --2- 172.21.15.16:0/2812792694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38a1b30 0x5568b389de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ed829640 1 -- 172.21.15.16:0/2812792694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b389c9c0 msgr2=0x5568b38ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ed829640 1 --2- 172.21.15.16:0/2812792694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b389c9c0 0x5568b38ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ed829640 1 -- 172.21.15.16:0/2812792694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38a1b30 msgr2=0x5568b389de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ed829640 1 --2- 172.21.15.16:0/2812792694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38a1b30 0x5568b389de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ed829640 1 -- 172.21.15.16:0/2812792694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b38a1120 con 0x5568b389bfd0 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ee02a640 1 --2- 172.21.15.16:0/2812792694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38a1b30 0x5568b389de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.216+0000 7fc0ed829640 1 --2- 172.21.15.16:0/2812792694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38a64e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4000c00 tx=0x7fc0e4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab3f9fa50f2d58c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0d37fe640 1 -- 172.21.15.16:0/2812792694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e401e040 con 0x5568b389bfd0 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0d37fe640 1 -- 172.21.15.16:0/2812792694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc0e401f040 con 0x5568b389bfd0 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0d37fe640 1 -- 172.21.15.16:0/2812792694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4003be0 con 0x5568b389bfd0 2024-03-27T06:42:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0f2671740 1 -- 172.21.15.16:0/2812792694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 msgr2=0x5568b38a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0f2671740 1 --2- 172.21.15.16:0/2812792694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38a64e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4000c00 tx=0x7fc0e4005f10 comp rx=0 tx=0).stop 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0f2671740 1 -- 172.21.15.16:0/2812792694 shutdown_connections 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0f2671740 1 --2- 172.21.15.16:0/2812792694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b38a1b30 0x5568b389de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0f2671740 1 --2- 172.21.15.16:0/2812792694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b389c9c0 0x5568b38ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0f2671740 1 --2- 172.21.15.16:0/2812792694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38a64e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0f2671740 1 -- 172.21.15.16:0/2812792694 >> 172.21.15.16:0/2812792694 conn(0x5568b3863b20 msgr2=0x5568b389ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0f2671740 1 -- 172.21.15.16:0/2812792694 shutdown_connections 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0f2671740 1 -- 172.21.15.16:0/2812792694 wait complete. 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.217+0000 7fc0f2671740 1 -- start start 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0f2671740 1 Processor -- start 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389c9c0 0x5568b38aa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0f2671740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b38a1b30 0x5568b38aa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0f2671740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5568b38b4280 con 0x5568b389bfd0 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0f2671740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5568b38b3bb0 con 0x5568b389c9c0 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0f2671740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5568b38b4950 con 0x5568b38a1b30 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0f2671740 10 client.0 _ll_register_callbacks cb 0x7fc0d8009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0ed829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0ed829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38ad7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48834/0 (socket says 172.21.15.16:48834) 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0ed829640 1 -- 172.21.15.16:0/3282360586 learned_addr learned my addr 172.21.15.16:0/3282360586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0ee02a640 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b38a1b30 0x5568b38aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0ed028640 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389c9c0 0x5568b38aa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0ed829640 1 -- 172.21.15.16:0/3282360586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b38a1b30 msgr2=0x5568b38aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0ed829640 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b38a1b30 0x5568b38aa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0ed829640 1 -- 172.21.15.16:0/3282360586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389c9c0 msgr2=0x5568b38aa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0ed829640 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389c9c0 0x5568b38aa2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0ed829640 1 -- 172.21.15.16:0/3282360586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b3918d00 con 0x5568b389bfd0 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0ed829640 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38ad7e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40095f0 tx=0x7fc0e4005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cda208bf5e873d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.218+0000 7fc0d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3282360586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4034070 con 0x5568b389bfd0 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3282360586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc0e40052d0 con 0x5568b389bfd0 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0f2671740 1 -- 172.21.15.16:0/3282360586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e401f040 con 0x5568b389bfd0 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3282360586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc0e401e040 con 0x5568b389bfd0 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0f2671740 10 client.15135 fetch_fsmap learned FSMap version 24 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0f2671740 1 -- 172.21.15.16:0/3282360586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5568b3919e70 con 0x5568b389bfd0 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3282360586 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc0e401f040 con 0x5568b389bfd0 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3282360586 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fc0e4030690 con 0x5568b389bfd0 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0f2671740 10 client.15135 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0f2671740 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0f2671740 20 client.15135 populate_metadata read hostname 'smithi016' 2024-03-27T06:42:59.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5568b391e3d0 0x5568b391aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:42:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0f2671740 4 client.15135 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T06:42:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0f2671740 1 -- 172.21.15.16:0/3282360586 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "status"}) -- 0x5568b38088a0 con 0x5568b391e3d0 2024-03-27T06:42:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.219+0000 7fc0ed028640 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5568b391e3d0 0x5568b391aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:42:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0d17fa640 10 client.15135 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:42:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0ed028640 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5568b391e3d0 0x5568b391aee0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ab426db082ea7c server_cookie=83bab2951d5c30bc in_seq=0 out_seq=0 2024-03-27T06:42:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0d17fa640 1 -- 172.21.15.16:0/3282360586 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5568b38088a0 con 0x5568b391e3d0 2024-03-27T06:42:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0d17fa640 10 client.15135 handle_command_reply: tid=0 2024-03-27T06:42:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 1 client.15135 shutdown 2024-03-27T06:42:59.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 1 -- 172.21.15.16:0/3282360586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 msgr2=0x5568b38ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38ad7e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40095f0 tx=0x7fc0e4005970 comp rx=0 tx=0).stop 2024-03-27T06:42:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 1 -- 172.21.15.16:0/3282360586 shutdown_connections 2024-03-27T06:42:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5568b391e3d0 0x5568b391aee0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568b38a1b30 0x5568b38aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5568b389c9c0 0x5568b38aa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0d17fa640 0 client.15135 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:42:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 1 --2- 172.21.15.16:0/3282360586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568b389bfd0 0x5568b38ad7e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 1 -- 172.21.15.16:0/3282360586 >> 172.21.15.16:0/3282360586 conn(0x5568b3863b20 msgr2=0x5568b389b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 1 -- 172.21.15.16:0/3282360586 shutdown_connections 2024-03-27T06:42:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 1 -- 172.21.15.16:0/3282360586 wait complete. 2024-03-27T06:42:59.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.220+0000 7fc0f2671740 20 client.15135 trim_cache size 0 max 16384 2024-03-27T06:42:59.225 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:42:59.225 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:42:59.225 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-03-27T06:42:59.225 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24527, 2024-03-27T06:42:59.225 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-03-27T06:42:59.226 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-03-27T06:42:59.226 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-03-27T06:42:59.226 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 89.837780047999999, 2024-03-27T06:42:59.226 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-03-27T06:42:59.226 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-03-27T06:42:59.226 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-03-27T06:42:59.227 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 94.969433065000004 2024-03-27T06:42:59.227 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:42:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1089540209 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0d40a3610 msgr2=0x7fc0d40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0d40a3610 0x7fc0d40a5ad0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005e30 tx=0x7fc0e0032670 comp rx=0 tx=0).stop 2024-03-27T06:42:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1089540209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e8102d50 msgr2=0x7fc0e8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:42:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e8102d50 0x7fc0e8198430 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800e9f0 tx=0x7fc0d800edf0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1089540209 shutdown_connections 2024-03-27T06:42:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0d40a3610 0x7fc0d40a5ad0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0e8103710 0x7fc0e819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e8102d50 0x7fc0e8198430 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 --2- 172.21.15.16:0/1089540209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0e8101b60 0x7fc0e8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:42:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1089540209 >> 172.21.15.16:0/1089540209 conn(0x7fc0e80fd0f0 msgr2=0x7fc0e8106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:42:59.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1089540209 shutdown_connections 2024-03-27T06:42:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:42:59.221+0000 7fc0efa8d640 1 -- 172.21.15.16:0/1089540209 wait complete. 2024-03-27T06:42:59.231 INFO:tasks.cephadm:Setup complete, yielding 2024-03-27T06:42:59.232 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-27T06:42:59.248 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-03-27T06:42:59.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-27T06:42:59.491 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:42:59.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:59 smithi156 ceph-mon[36261]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T06:42:59.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:42:59 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/4211711721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:42:59.788 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:59 smithi016 ceph-mon[31908]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T06:42:59.788 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:42:59 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/4211711721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:42:59.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:59 smithi170 ceph-mon[36438]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-27T06:42:59.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:42:59 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/4211711721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:01.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.354+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/1529366733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4101560 msgr2=0x7fa9e4101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:01.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.354+0000 7fa9ec1d5700 1 --2- 172.21.15.16:0/1529366733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4101560 0x7fa9e4101980 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00099c0 tx=0x7fa9e0009cd0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.354+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/1529366733 shutdown_connections 2024-03-27T06:43:01.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.354+0000 7fa9ec1d5700 1 --2- 172.21.15.16:0/1529366733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e4103100 0x7fa9e41099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.354+0000 7fa9ec1d5700 1 --2- 172.21.15.16:0/1529366733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4101560 0x7fa9e4101980 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.354+0000 7fa9ec1d5700 1 --2- 172.21.15.16:0/1529366733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9e4102760 0x7fa9e4102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.354+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/1529366733 >> 172.21.15.16:0/1529366733 conn(0x7fa9e40fcb00 msgr2=0x7fa9e40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:01.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.354+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/1529366733 shutdown_connections 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.354+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/1529366733 wait complete. 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.354+0000 7fa9ec1d5700 1 Processor -- start 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.354+0000 7fa9ec1d5700 1 -- start start 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ec1d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4101560 0x7fa9e4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ec1d5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e4102760 0x7fa9e4198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ec1d5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9e4103100 0x7fa9e419fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ec1d5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e410d230 con 0x7fa9e4101560 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ec1d5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e410d0b0 con 0x7fa9e4103100 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ec1d5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e410d3b0 con 0x7fa9e4102760 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9e9f71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4101560 0x7fa9e4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9e9f71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4101560 0x7fa9e4198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48854/0 (socket says 172.21.15.16:48854) 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9e9f71700 1 -- 172.21.15.16:0/2667996015 learned_addr learned my addr 172.21.15.16:0/2667996015 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9e9770700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e4102760 0x7fa9e4198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ea772700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9e4103100 0x7fa9e419fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ea772700 1 -- 172.21.15.16:0/2667996015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e4102760 msgr2=0x7fa9e4198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ea772700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e4102760 0x7fa9e4198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ea772700 1 -- 172.21.15.16:0/2667996015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4101560 msgr2=0x7fa9e4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ea772700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4101560 0x7fa9e4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ea772700 1 -- 172.21.15.16:0/2667996015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e41a0590 con 0x7fa9e4103100 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9e9f71700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4101560 0x7fa9e4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9e9770700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e4102760 0x7fa9e4198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.355+0000 7fa9ea772700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9e4103100 0x7fa9e419fef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00efc0 tx=0x7fa9dc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.356+0000 7fa9d6ffd700 1 -- 172.21.15.16:0/2667996015 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0092e0 con 0x7fa9e4103100 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.356+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/2667996015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e41a0820 con 0x7fa9e4103100 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.356+0000 7fa9d6ffd700 1 -- 172.21.15.16:0/2667996015 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa9dc004bb0 con 0x7fa9e4103100 2024-03-27T06:43:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.356+0000 7fa9d6ffd700 1 -- 172.21.15.16:0/2667996015 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc004280 con 0x7fa9e4103100 2024-03-27T06:43:01.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.356+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/2667996015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e41a0db0 con 0x7fa9e4103100 2024-03-27T06:43:01.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.357+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/2667996015 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e4069300 con 0x7fa9e4103100 2024-03-27T06:43:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.358+0000 7fa9d6ffd700 1 -- 172.21.15.16:0/2667996015 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fa9dc01b030 con 0x7fa9e4103100 2024-03-27T06:43:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.358+0000 7fa9d6ffd700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa9d00a7b30 0x7fa9d00a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.359+0000 7fa9e9f71700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa9d00a7b30 0x7fa9d00a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.359+0000 7fa9d6ffd700 1 -- 172.21.15.16:0/2667996015 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fa9dc0cf960 con 0x7fa9e4103100 2024-03-27T06:43:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.359+0000 7fa9e9f71700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa9d00a7b30 0x7fa9d00a9ff0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0000c00 tx=0x7fa9e0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:01.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.364+0000 7fa9d6ffd700 1 -- 172.21.15.16:0/2667996015 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fa9dc022020 con 0x7fa9e4103100 2024-03-27T06:43:01.528 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:01 smithi016 ceph-mon[31908]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-27T06:43:01.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.592+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/2667996015 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fa9e406b290 con 0x7fa9d00a7b30 2024-03-27T06:43:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.593+0000 7fa9d6ffd700 1 -- 172.21.15.16:0/2667996015 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fa9e406b290 con 0x7fa9d00a7b30 2024-03-27T06:43:01.591 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-03-27T06:43:01.591 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-03-27T06:43:01.591 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-03-27T06:43:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.595+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/2667996015 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa9d00a7b30 msgr2=0x7fa9d00a9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.595+0000 7fa9ec1d5700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa9d00a7b30 0x7fa9d00a9ff0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0000c00 tx=0x7fa9e0005fb0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.595+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/2667996015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9e4103100 msgr2=0x7fa9e419fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.595+0000 7fa9ec1d5700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9e4103100 0x7fa9e419fef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00efc0 tx=0x7fa9dc00c3a0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.595+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/2667996015 shutdown_connections 2024-03-27T06:43:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.595+0000 7fa9ec1d5700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9e4103100 0x7fa9e419fef0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.595+0000 7fa9ec1d5700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9e4101560 0x7fa9e4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.595+0000 7fa9ec1d5700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fa9d00a7b30 0x7fa9d00a9ff0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.595+0000 7fa9ec1d5700 1 --2- 172.21.15.16:0/2667996015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e4102760 0x7fa9e4198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.595+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/2667996015 >> 172.21.15.16:0/2667996015 conn(0x7fa9e40fcb00 msgr2=0x7fa9e406ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.596+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/2667996015 shutdown_connections 2024-03-27T06:43:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:01.596+0000 7fa9ec1d5700 1 -- 172.21.15.16:0/2667996015 wait complete. 2024-03-27T06:43:01.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:01 smithi156 ceph-mon[36261]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-27T06:43:01.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:01 smithi170 ceph-mon[36438]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-27T06:43:02.532 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-27T06:43:02.778 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:43:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:03 smithi156 ceph-mon[36261]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-27T06:43:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:03 smithi156 ceph-mon[36261]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:03.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:03 smithi170 ceph-mon[36438]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-27T06:43:03.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:03 smithi170 ceph-mon[36438]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:03.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:03 smithi016 ceph-mon[31908]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-27T06:43:03.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:03 smithi016 ceph-mon[31908]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.570+0000 7f6589210700 1 -- 172.21.15.16:0/674929544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584102670 msgr2=0x7f6584102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.570+0000 7f6589210700 1 --2- 172.21.15.16:0/674929544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584102670 0x7f6584102a90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f65740099c0 tx=0x7f6574009cd0 comp rx=0 tx=0).stop 2024-03-27T06:43:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.570+0000 7f6589210700 1 -- 172.21.15.16:0/674929544 shutdown_connections 2024-03-27T06:43:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.570+0000 7f6589210700 1 --2- 172.21.15.16:0/674929544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6584104210 0x7f658410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.570+0000 7f6589210700 1 --2- 172.21.15.16:0/674929544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584102670 0x7f6584102a90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.570+0000 7f6589210700 1 --2- 172.21.15.16:0/674929544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6584103870 0x7f6584103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.570+0000 7f6589210700 1 -- 172.21.15.16:0/674929544 >> 172.21.15.16:0/674929544 conn(0x7f65840fdc50 msgr2=0x7f6584100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.571+0000 7f6589210700 1 -- 172.21.15.16:0/674929544 shutdown_connections 2024-03-27T06:43:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.571+0000 7f6589210700 1 -- 172.21.15.16:0/674929544 wait complete. 2024-03-27T06:43:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.571+0000 7f6589210700 1 Processor -- start 2024-03-27T06:43:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f6589210700 1 -- start start 2024-03-27T06:43:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f6589210700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584102670 0x7f65840763e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f6589210700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6584103870 0x7f6584074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f6589210700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6584104210 0x7f6584074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f6589210700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f658410e320 con 0x7f6584102670 2024-03-27T06:43:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f6589210700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f658410e1a0 con 0x7f6584104210 2024-03-27T06:43:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f6589210700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f658410e4a0 con 0x7f6584103870 2024-03-27T06:43:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f6582d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584102670 0x7f65840763e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f6582d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584102670 0x7f65840763e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48880/0 (socket says 172.21.15.16:48880) 2024-03-27T06:43:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f6582d9d700 1 -- 172.21.15.16:0/1879830852 learned_addr learned my addr 172.21.15.16:0/1879830852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f658259c700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6584103870 0x7f6584074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f658359e700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6584104210 0x7f6584074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f658259c700 1 -- 172.21.15.16:0/1879830852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6584104210 msgr2=0x7f6584074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f658259c700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6584104210 0x7f6584074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f658259c700 1 -- 172.21.15.16:0/1879830852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584102670 msgr2=0x7f65840763e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f658259c700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584102670 0x7f65840763e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f658259c700 1 -- 172.21.15.16:0/1879830852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6584075820 con 0x7f6584103870 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f6582d9d700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584102670 0x7f65840763e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.572+0000 7f658359e700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6584104210 0x7f6584074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.573+0000 7f658259c700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6584103870 0x7f6584074a30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f657800e9f0 tx=0x7f657800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.573+0000 7f656bfff700 1 -- 172.21.15.16:0/1879830852 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657800c9c0 con 0x7f6584103870 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.573+0000 7f6589210700 1 -- 172.21.15.16:0/1879830852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65841985f0 con 0x7f6584103870 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.573+0000 7f656bfff700 1 -- 172.21.15.16:0/1879830852 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f657800cb60 con 0x7f6584103870 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.573+0000 7f656bfff700 1 -- 172.21.15.16:0/1879830852 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578018660 con 0x7f6584103870 2024-03-27T06:43:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.573+0000 7f6589210700 1 -- 172.21.15.16:0/1879830852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6584198b30 con 0x7f6584103870 2024-03-27T06:43:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.574+0000 7f6589210700 1 -- 172.21.15.16:0/1879830852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6584069300 con 0x7f6584103870 2024-03-27T06:43:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.575+0000 7f656bfff700 1 -- 172.21.15.16:0/1879830852 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f6578010430 con 0x7f6584103870 2024-03-27T06:43:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.575+0000 7f656bfff700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f656c0a36a0 0x7f656c0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.576+0000 7f6582d9d700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f656c0a36a0 0x7f656c0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.576+0000 7f656bfff700 1 -- 172.21.15.16:0/1879830852 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f65780d00b0 con 0x7f6584103870 2024-03-27T06:43:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.576+0000 7f6582d9d700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f656c0a36a0 0x7f656c0a5b60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6574000c00 tx=0x7f6574024040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.579+0000 7f656bfff700 1 -- 172.21.15.16:0/1879830852 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f657809cc30 con 0x7f6584103870 2024-03-27T06:43:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.797+0000 7f6589210700 1 -- 172.21.15.16:0/1879830852 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6584062a80 con 0x7f656c0a36a0 2024-03-27T06:43:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.811+0000 7f656bfff700 1 -- 172.21.15.16:0/1879830852 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f6584062a80 con 0x7f656c0a36a0 2024-03-27T06:43:04.806 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-27T06:43:04.806 INFO:teuthology.orchestra.run.smithi016.stdout:mds.a smithi016 running (103s) 19s ago 103s 16.9M - 19.0.0-2478-g155268c4 b379bbd8da6b e4e0d4d26dd0 2024-03-27T06:43:04.806 INFO:teuthology.orchestra.run.smithi016.stdout:mds.b smithi156 running (100s) 34s ago 100s 25.5M - 19.0.0-2478-g155268c4 b379bbd8da6b c90d68b0db85 2024-03-27T06:43:04.806 INFO:teuthology.orchestra.run.smithi016.stdout:mds.c smithi170 running (115s) 34s ago 114s 19.8M - 19.0.0-2478-g155268c4 b379bbd8da6b 45944325da25 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mds.d smithi016 running (2m) 19s ago 2m 16.2M - 19.0.0-2478-g155268c4 b379bbd8da6b 0b99dee22991 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mds.e smithi156 running (2m) 34s ago 2m 16.7M - 19.0.0-2478-g155268c4 b379bbd8da6b 0979e0d68b54 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mds.f smithi170 running (117s) 34s ago 117s 16.9M - 19.0.0-2478-g155268c4 b379bbd8da6b dac4d49d7a38 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mds.g smithi016 running (106s) 19s ago 106s 17.1M - 19.0.0-2478-g155268c4 b379bbd8da6b c254894f86e3 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mds.h smithi156 running (112s) 34s ago 111s 17.4M - 19.0.0-2478-g155268c4 b379bbd8da6b a728dab8dfcd 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mds.i smithi170 running (109s) 34s ago 108s 17.9M - 19.0.0-2478-g155268c4 b379bbd8da6b 2ae822f5ef10 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mds.j smithi016 running (2m) 19s ago 2m 15.4M - 19.0.0-2478-g155268c4 b379bbd8da6b 79533e008018 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mds.k smithi156 running (2m) 34s ago 2m 19.3M - 19.0.0-2478-g155268c4 b379bbd8da6b c3b70b9c62bf 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mds.l smithi170 running (2m) 34s ago 2m 16.8M - 19.0.0-2478-g155268c4 b379bbd8da6b 03bcfff9364f 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.x smithi016 *:9283,8765 running (9m) 19s ago 9m 514M - 19.0.0-2478-g155268c4 b379bbd8da6b 9699eb46f80f 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.y smithi156 *:8443,8765 running (7m) 34s ago 7m 451M - 19.0.0-2478-g155268c4 b379bbd8da6b 1df63da56546 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.z smithi170 *:8443,8765 running (7m) 34s ago 7m 454M - 19.0.0-2478-g155268c4 b379bbd8da6b 712531754fdd 2024-03-27T06:43:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:mon.a smithi016 running (9m) 19s ago 9m 49.3M 2048M 19.0.0-2478-g155268c4 b379bbd8da6b 06aedba28008 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:mon.b smithi156 running (7m) 34s ago 7m 42.6M 2048M 19.0.0-2478-g155268c4 b379bbd8da6b 1e165e1b9c54 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:mon.c smithi170 running (7m) 34s ago 7m 44.0M 2048M 19.0.0-2478-g155268c4 b379bbd8da6b 610ce06acfe8 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (7m) 19s ago 7m 47.5M 4273M 19.0.0-2478-g155268c4 b379bbd8da6b 3b50ffebef81 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi156 running (6m) 34s ago 6m 58.4M 4270M 19.0.0-2478-g155268c4 b379bbd8da6b 5c7ca1add42e 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi170 running (6m) 34s ago 6m 46.2M 4270M 19.0.0-2478-g155268c4 b379bbd8da6b 9cb38c41d966 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi016 running (5m) 19s ago 5m 55.8M 4273M 19.0.0-2478-g155268c4 b379bbd8da6b 908f355be05b 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi156 running (5m) 34s ago 5m 55.4M 4270M 19.0.0-2478-g155268c4 b379bbd8da6b 70d98398ff6f 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi170 running (5m) 34s ago 5m 43.2M 4270M 19.0.0-2478-g155268c4 b379bbd8da6b 66769cd36ca0 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 smithi016 running (4m) 19s ago 4m 42.4M 4273M 19.0.0-2478-g155268c4 b379bbd8da6b ca65ed3daf3f 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 smithi156 running (4m) 34s ago 4m 42.9M 4270M 19.0.0-2478-g155268c4 b379bbd8da6b fc7658ca7ad6 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 smithi170 running (3m) 34s ago 3m 42.8M 4270M 19.0.0-2478-g155268c4 b379bbd8da6b 61c64560a94a 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 smithi016 running (3m) 19s ago 3m 41.8M 4273M 19.0.0-2478-g155268c4 b379bbd8da6b 0084ba052911 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 smithi156 running (2m) 34s ago 2m 53.2M 4270M 19.0.0-2478-g155268c4 b379bbd8da6b c3a8c1c4632d 2024-03-27T06:43:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 smithi170 running (2m) 34s ago 2m 41.4M 4270M 19.0.0-2478-g155268c4 b379bbd8da6b e90426f74279 2024-03-27T06:43:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 -- 172.21.15.16:0/1879830852 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f656c0a36a0 msgr2=0x7f656c0a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f656c0a36a0 0x7f656c0a5b60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6574000c00 tx=0x7f6574024040 comp rx=0 tx=0).stop 2024-03-27T06:43:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 -- 172.21.15.16:0/1879830852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6584103870 msgr2=0x7f6584074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6584103870 0x7f6584074a30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f657800e9f0 tx=0x7f657800ed00 comp rx=0 tx=0).stop 2024-03-27T06:43:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 -- 172.21.15.16:0/1879830852 shutdown_connections 2024-03-27T06:43:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6584104210 0x7f6584074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584102670 0x7f65840763e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f656c0a36a0 0x7f656c0a5b60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 --2- 172.21.15.16:0/1879830852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6584103870 0x7f6584074a30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 -- 172.21.15.16:0/1879830852 >> 172.21.15.16:0/1879830852 conn(0x7f65840fdc50 msgr2=0x7f6584106c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 -- 172.21.15.16:0/1879830852 shutdown_connections 2024-03-27T06:43:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:04.813+0000 7f6589210700 1 -- 172.21.15.16:0/1879830852 wait complete. 2024-03-27T06:43:05.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:05 smithi016 ceph-mon[31908]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T06:43:05.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-27T06:43:05.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:05 smithi156 ceph-mon[36261]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T06:43:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:43:05.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:05 smithi170 ceph-mon[36438]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-27T06:43:06.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:06 smithi156 ceph-mon[36261]: from='client.24994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:06.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:06 smithi170 ceph-mon[36438]: from='client.24994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:06.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:06 smithi016 ceph-mon[31908]: from='client.24994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.459+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3907056912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b9c1015f0 msgr2=0x7f2b9c101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.459+0000 7f2ba2aac700 1 --2- 172.21.15.16:0/3907056912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b9c1015f0 0x7f2b9c101a10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0099c0 tx=0x7f2b8c009cd0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.459+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3907056912 shutdown_connections 2024-03-27T06:43:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.459+0000 7f2ba2aac700 1 --2- 172.21.15.16:0/3907056912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c103190 0x7f2b9c109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.459+0000 7f2ba2aac700 1 --2- 172.21.15.16:0/3907056912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b9c1015f0 0x7f2b9c101a10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.459+0000 7f2ba2aac700 1 --2- 172.21.15.16:0/3907056912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c1027f0 0x7f2b9c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.459+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3907056912 >> 172.21.15.16:0/3907056912 conn(0x7f2b9c0fcb90 msgr2=0x7f2b9c0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.459+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3907056912 shutdown_connections 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.459+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3907056912 wait complete. 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba2aac700 1 Processor -- start 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba2aac700 1 -- start start 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba2aac700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c1015f0 0x7f2b9c105580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba2aac700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c1027f0 0x7f2b9c103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba2aac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b9c103190 0x7f2b9c104110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba2aac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c10d2d0 con 0x7f2b9c103190 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba2aac700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c10d150 con 0x7f2b9c1015f0 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba2aac700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c10d450 con 0x7f2b9c1027f0 2024-03-27T06:43:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba1049700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b9c103190 0x7f2b9c104110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba1049700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b9c103190 0x7f2b9c104110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57132/0 (socket says 172.21.15.16:57132) 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba1049700 1 -- 172.21.15.16:0/3873352472 learned_addr learned my addr 172.21.15.16:0/3873352472 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba0848700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c1015f0 0x7f2b9c105580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2b9bfff700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c1027f0 0x7f2b9c103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba1049700 1 -- 172.21.15.16:0/3873352472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c1027f0 msgr2=0x7f2b9c103bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba1049700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c1027f0 0x7f2b9c103bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba1049700 1 -- 172.21.15.16:0/3873352472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c1015f0 msgr2=0x7f2b9c105580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba1049700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c1015f0 0x7f2b9c105580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba1049700 1 -- 172.21.15.16:0/3873352472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9c1049c0 con 0x7f2b9c103190 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2b9bfff700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c1027f0 0x7f2b9c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.460+0000 7f2ba1049700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b9c103190 0x7f2b9c104110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2b94007ee0 tx=0x7f2b9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.461+0000 7f2b99ffb700 1 -- 172.21.15.16:0/3873352472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400ddd0 con 0x7f2b9c103190 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.461+0000 7f2b99ffb700 1 -- 172.21.15.16:0/3873352472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2b94004020 con 0x7f2b9c103190 2024-03-27T06:43:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.461+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3873352472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9c1a2b90 con 0x7f2b9c103190 2024-03-27T06:43:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.461+0000 7f2b99ffb700 1 -- 172.21.15.16:0/3873352472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94015660 con 0x7f2b9c103190 2024-03-27T06:43:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.461+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3873352472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9c1a30a0 con 0x7f2b9c103190 2024-03-27T06:43:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.462+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3873352472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b9c069300 con 0x7f2b9c103190 2024-03-27T06:43:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.463+0000 7f2b99ffb700 1 -- 172.21.15.16:0/3873352472 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f2b940041d0 con 0x7f2b9c103190 2024-03-27T06:43:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.463+0000 7f2b99ffb700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2b880a36a0 0x7f2b880a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.464+0000 7f2ba0848700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2b880a36a0 0x7f2b880a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.464+0000 7f2b99ffb700 1 -- 172.21.15.16:0/3873352472 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2b940cf700 con 0x7f2b9c103190 2024-03-27T06:43:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.464+0000 7f2ba0848700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2b880a36a0 0x7f2b880a5b60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c006010 tx=0x7f2b8c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.468+0000 7f2b99ffb700 1 -- 172.21.15.16:0/3873352472 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f2b94010070 con 0x7f2b9c103190 2024-03-27T06:43:07.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:07 smithi016 ceph-mon[31908]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-27T06:43:07.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.685+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3873352472 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2b9c0fed20 con 0x7f2b880a36a0 2024-03-27T06:43:07.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.691+0000 7f2b99ffb700 1 -- 172.21.15.16:0/3873352472 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2b9c0fed20 con 0x7f2b880a36a0 2024-03-27T06:43:07.686 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-27T06:43:07.686 INFO:teuthology.orchestra.run.smithi016.stdout:mds.all 12/12 37s ago 2m smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi156=b;smithi156=e;smithi156=h;smithi156=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-03-27T06:43:07.686 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 3/3 37s ago 7m smithi016=x;smithi156=y;smithi170=z;count:3 2024-03-27T06:43:07.686 INFO:teuthology.orchestra.run.smithi016.stdout:mon 3/3 37s ago 8m smithi016:172.21.15.16=a;smithi156:172.21.15.156=b;smithi170:172.21.15.170=c;count:3 2024-03-27T06:43:07.686 INFO:teuthology.orchestra.run.smithi016.stdout:osd 12 37s ago - 2024-03-27T06:43:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3873352472 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2b880a36a0 msgr2=0x7f2b880a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2b880a36a0 0x7f2b880a5b60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c006010 tx=0x7f2b8c000bc0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3873352472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b9c103190 msgr2=0x7f2b9c104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b9c103190 0x7f2b9c104110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2b94007ee0 tx=0x7f2b9400d3b0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3873352472 shutdown_connections 2024-03-27T06:43:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b9c103190 0x7f2b9c104110 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b9c1015f0 0x7f2b9c105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2b880a36a0 0x7f2b880a5b60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 --2- 172.21.15.16:0/3873352472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c1027f0 0x7f2b9c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3873352472 >> 172.21.15.16:0/3873352472 conn(0x7f2b9c0fcb90 msgr2=0x7f2b9c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3873352472 shutdown_connections 2024-03-27T06:43:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:07.694+0000 7f2ba2aac700 1 -- 172.21.15.16:0/3873352472 wait complete. 2024-03-27T06:43:07.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:07 smithi156 ceph-mon[36261]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-27T06:43:07.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:07 smithi170 ceph-mon[36438]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-27T06:43:08.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-27T06:43:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:43:09.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:09 smithi156 ceph-mon[36261]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:43:09.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:09 smithi156 ceph-mon[36261]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:09.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:09 smithi016 ceph-mon[31908]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:43:09.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:09 smithi016 ceph-mon[31908]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:09.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:09 smithi170 ceph-mon[36438]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:43:09.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:09 smithi170 ceph-mon[36438]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:10.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.334+0000 7f3b46744700 1 -- 172.21.15.16:0/3774923592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b40101840 msgr2=0x7f3b40101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:10.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.334+0000 7f3b46744700 1 --2- 172.21.15.16:0/3774923592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b40101840 0x7f3b40101cc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009990 tx=0x7f3b28009ca0 comp rx=0 tx=0).stop 2024-03-27T06:43:10.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.334+0000 7f3b46744700 1 -- 172.21.15.16:0/3774923592 shutdown_connections 2024-03-27T06:43:10.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.334+0000 7f3b46744700 1 --2- 172.21.15.16:0/3774923592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b40102200 0x7f3b40102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:10.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.334+0000 7f3b46744700 1 --2- 172.21.15.16:0/3774923592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b40100ee0 0x7f3b40101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:10.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.334+0000 7f3b46744700 1 --2- 172.21.15.16:0/3774923592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b40101840 0x7f3b40101cc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.334+0000 7f3b46744700 1 -- 172.21.15.16:0/3774923592 >> 172.21.15.16:0/3774923592 conn(0x7f3b400fcb20 msgr2=0x7f3b400fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.334+0000 7f3b46744700 1 -- 172.21.15.16:0/3774923592 shutdown_connections 2024-03-27T06:43:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.334+0000 7f3b46744700 1 -- 172.21.15.16:0/3774923592 wait complete. 2024-03-27T06:43:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.334+0000 7f3b46744700 1 Processor -- start 2024-03-27T06:43:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b46744700 1 -- start start 2024-03-27T06:43:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b46744700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b40100ee0 0x7f3b4019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b46744700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b40101840 0x7f3b4019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b46744700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b40102200 0x7f3b401a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b46744700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40110570 con 0x7f3b40102200 2024-03-27T06:43:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b46744700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401103f0 con 0x7f3b40101840 2024-03-27T06:43:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b46744700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401106f0 con 0x7f3b40100ee0 2024-03-27T06:43:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b44ce1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b40102200 0x7f3b401a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:10.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b44ce1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b40102200 0x7f3b401a20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57154/0 (socket says 172.21.15.16:57154) 2024-03-27T06:43:10.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b44ce1700 1 -- 172.21.15.16:0/2769662750 learned_addr learned my addr 172.21.15.16:0/2769662750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:10.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b3ffff700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b40100ee0 0x7f3b4019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:10.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b3f7fe700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b40101840 0x7f3b4019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:10.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b44ce1700 1 -- 172.21.15.16:0/2769662750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b40100ee0 msgr2=0x7f3b4019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:10.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b44ce1700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b40100ee0 0x7f3b4019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:10.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b44ce1700 1 -- 172.21.15.16:0/2769662750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b40101840 msgr2=0x7f3b4019b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b44ce1700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b40101840 0x7f3b4019b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b44ce1700 1 -- 172.21.15.16:0/2769662750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b401a2790 con 0x7f3b40102200 2024-03-27T06:43:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.335+0000 7f3b3ffff700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b40100ee0 0x7f3b4019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.336+0000 7f3b44ce1700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b40102200 0x7f3b401a20f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800ef60 tx=0x7f3b3800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.336+0000 7f3b3d7fa700 1 -- 172.21.15.16:0/2769662750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380092e0 con 0x7f3b40102200 2024-03-27T06:43:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.336+0000 7f3b3d7fa700 1 -- 172.21.15.16:0/2769662750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3b38004bb0 con 0x7f3b40102200 2024-03-27T06:43:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.336+0000 7f3b46744700 1 -- 172.21.15.16:0/2769662750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b401a2a20 con 0x7f3b40102200 2024-03-27T06:43:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.336+0000 7f3b3d7fa700 1 -- 172.21.15.16:0/2769662750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38004280 con 0x7f3b40102200 2024-03-27T06:43:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.336+0000 7f3b46744700 1 -- 172.21.15.16:0/2769662750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b401a2f10 con 0x7f3b40102200 2024-03-27T06:43:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.337+0000 7f3b46744700 1 -- 172.21.15.16:0/2769662750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b40069300 con 0x7f3b40102200 2024-03-27T06:43:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.338+0000 7f3b3d7fa700 1 -- 172.21.15.16:0/2769662750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f3b3801b030 con 0x7f3b40102200 2024-03-27T06:43:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.339+0000 7f3b3d7fa700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3b2c0a36a0 0x7f3b2c0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.339+0000 7f3b3ffff700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3b2c0a36a0 0x7f3b2c0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.339+0000 7f3b3d7fa700 1 -- 172.21.15.16:0/2769662750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3b380cf740 con 0x7f3b40102200 2024-03-27T06:43:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.339+0000 7f3b3ffff700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3b2c0a36a0 0x7f3b2c0a5b60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3b30011490 tx=0x7f3b30009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.343+0000 7f3b3d7fa700 1 -- 172.21.15.16:0/2769662750 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f3b38022020 con 0x7f3b40102200 2024-03-27T06:43:10.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.556+0000 7f3b46744700 1 -- 172.21.15.16:0/2769662750 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3b40062a80 con 0x7f3b2c0a36a0 2024-03-27T06:43:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.557+0000 7f3b3d7fa700 1 -- 172.21.15.16:0/2769662750 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3b40062a80 con 0x7f3b2c0a36a0 2024-03-27T06:43:10.552 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-03-27T06:43:10.552 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-03-27T06:43:10.552 INFO:teuthology.orchestra.run.smithi016.stdout:smithi156 172.21.15.156 2024-03-27T06:43:10.552 INFO:teuthology.orchestra.run.smithi016.stdout:smithi170 172.21.15.170 2024-03-27T06:43:10.552 INFO:teuthology.orchestra.run.smithi016.stdout:3 hosts in cluster 2024-03-27T06:43:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 -- 172.21.15.16:0/2769662750 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3b2c0a36a0 msgr2=0x7f3b2c0a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3b2c0a36a0 0x7f3b2c0a5b60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3b30011490 tx=0x7f3b30009260 comp rx=0 tx=0).stop 2024-03-27T06:43:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 -- 172.21.15.16:0/2769662750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b40102200 msgr2=0x7f3b401a20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b40102200 0x7f3b401a20f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800ef60 tx=0x7f3b3800ef90 comp rx=0 tx=0).stop 2024-03-27T06:43:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 -- 172.21.15.16:0/2769662750 shutdown_connections 2024-03-27T06:43:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b40102200 0x7f3b401a20f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f3b2c0a36a0 0x7f3b2c0a5b60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b40100ee0 0x7f3b4019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 --2- 172.21.15.16:0/2769662750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b40101840 0x7f3b4019b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 -- 172.21.15.16:0/2769662750 >> 172.21.15.16:0/2769662750 conn(0x7f3b400fcb20 msgr2=0x7f3b400fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 -- 172.21.15.16:0/2769662750 shutdown_connections 2024-03-27T06:43:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:10.559+0000 7f3b46744700 1 -- 172.21.15.16:0/2769662750 wait complete. 2024-03-27T06:43:11.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-27T06:43:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:43:11.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:11 smithi016 ceph-mon[31908]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:43:11.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:11 smithi156 ceph-mon[36261]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:43:11.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:11 smithi170 ceph-mon[36438]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:43:12.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:12 smithi016 ceph-mon[31908]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:12.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:12 smithi156 ceph-mon[36261]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:12.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:12 smithi170 ceph-mon[36438]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.359+0000 7f2cb3465700 1 -- 172.21.15.16:0/4124242882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cac102770 msgr2=0x7f2cac102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.359+0000 7f2cb3465700 1 --2- 172.21.15.16:0/4124242882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cac102770 0x7f2cac102bd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0099c0 tx=0x7f2c9c009cd0 comp rx=0 tx=0).stop 2024-03-27T06:43:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.359+0000 7f2cb3465700 1 -- 172.21.15.16:0/4124242882 shutdown_connections 2024-03-27T06:43:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.359+0000 7f2cb3465700 1 --2- 172.21.15.16:0/4124242882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cac103110 0x7f2cac1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.359+0000 7f2cb3465700 1 --2- 172.21.15.16:0/4124242882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cac101570 0x7f2cac101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.359+0000 7f2cb3465700 1 --2- 172.21.15.16:0/4124242882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cac102770 0x7f2cac102bd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.359+0000 7f2cb3465700 1 -- 172.21.15.16:0/4124242882 >> 172.21.15.16:0/4124242882 conn(0x7f2cac0fcb30 msgr2=0x7f2cac0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.359+0000 7f2cb3465700 1 -- 172.21.15.16:0/4124242882 shutdown_connections 2024-03-27T06:43:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.359+0000 7f2cb3465700 1 -- 172.21.15.16:0/4124242882 wait complete. 2024-03-27T06:43:13.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb3465700 1 Processor -- start 2024-03-27T06:43:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb3465700 1 -- start start 2024-03-27T06:43:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb3465700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cac101570 0x7f2cac198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb3465700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cac102770 0x7f2cac198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb3465700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cac103110 0x7f2cac19fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb3465700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10d240 con 0x7f2cac102770 2024-03-27T06:43:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb3465700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10d0c0 con 0x7f2cac101570 2024-03-27T06:43:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb3465700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10d3c0 con 0x7f2cac103110 2024-03-27T06:43:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb0a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cac102770 0x7f2cac198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb0a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cac102770 0x7f2cac198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57168/0 (socket says 172.21.15.16:57168) 2024-03-27T06:43:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb0a00700 1 -- 172.21.15.16:0/3000821581 learned_addr learned my addr 172.21.15.16:0/3000821581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.360+0000 7f2cb1201700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cac101570 0x7f2cac198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2cb1a02700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cac103110 0x7f2cac19fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2cb0a00700 1 -- 172.21.15.16:0/3000821581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cac103110 msgr2=0x7f2cac19fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2cb0a00700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cac103110 0x7f2cac19fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2cb0a00700 1 -- 172.21.15.16:0/3000821581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cac101570 msgr2=0x7f2cac198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2cb0a00700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cac101570 0x7f2cac198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2cb0a00700 1 -- 172.21.15.16:0/3000821581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac1a0590 con 0x7f2cac102770 2024-03-27T06:43:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2cb1201700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cac101570 0x7f2cac198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2cb0a00700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cac102770 0x7f2cac198ec0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c005910 tx=0x7f2c9c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2ca27fc700 1 -- 172.21.15.16:0/3000821581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c009e60 con 0x7f2cac102770 2024-03-27T06:43:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2cb3465700 1 -- 172.21.15.16:0/3000821581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac1a0820 con 0x7f2cac102770 2024-03-27T06:43:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2ca27fc700 1 -- 172.21.15.16:0/3000821581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2c9c00be40 con 0x7f2cac102770 2024-03-27T06:43:13.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2ca27fc700 1 -- 172.21.15.16:0/3000821581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0173f0 con 0x7f2cac102770 2024-03-27T06:43:13.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.361+0000 7f2cb3465700 1 -- 172.21.15.16:0/3000821581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac1a0cd0 con 0x7f2cac102770 2024-03-27T06:43:13.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.362+0000 7f2cb3465700 1 -- 172.21.15.16:0/3000821581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac069300 con 0x7f2cac102770 2024-03-27T06:43:13.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.363+0000 7f2ca27fc700 1 -- 172.21.15.16:0/3000821581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f2c9c003b00 con 0x7f2cac102770 2024-03-27T06:43:13.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.364+0000 7f2ca27fc700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2c980a3720 0x7f2c980a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:13.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.364+0000 7f2cb1201700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2c980a3720 0x7f2c980a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:13.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.364+0000 7f2ca27fc700 1 -- 172.21.15.16:0/3000821581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2c9c0d0ab0 con 0x7f2cac102770 2024-03-27T06:43:13.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.365+0000 7f2cb1201700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2c980a3720 0x7f2c980a5be0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2cac1025d0 tx=0x7f2ca8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:13.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.368+0000 7f2ca27fc700 1 -- 172.21.15.16:0/3000821581 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f2c9c013070 con 0x7f2cac102770 2024-03-27T06:43:13.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.588+0000 7f2cb3465700 1 -- 172.21.15.16:0/3000821581 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f2cac0fecd0 con 0x7f2c980a3720 2024-03-27T06:43:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.589+0000 7f2ca27fc700 1 -- 172.21.15.16:0/3000821581 <== mgr.14148 v2:172.21.15.16:6800/4198806087 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f2cac0fecd0 con 0x7f2c980a3720 2024-03-27T06:43:13.609 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-27T06:43:13.609 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-27T06:43:13.609 INFO:teuthology.orchestra.run.smithi016.stdout:smithi156 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760040480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-03-27T06:43:13.609 INFO:teuthology.orchestra.run.smithi016.stdout:smithi170 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002R480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-03-27T06:43:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.591+0000 7f2cb3465700 1 -- 172.21.15.16:0/3000821581 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2c980a3720 msgr2=0x7f2c980a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.591+0000 7f2cb3465700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2c980a3720 0x7f2c980a5be0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2cac1025d0 tx=0x7f2ca8009260 comp rx=0 tx=0).stop 2024-03-27T06:43:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.591+0000 7f2cb3465700 1 -- 172.21.15.16:0/3000821581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cac102770 msgr2=0x7f2cac198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.591+0000 7f2cb3465700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cac102770 0x7f2cac198ec0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c005910 tx=0x7f2c9c00f690 comp rx=0 tx=0).stop 2024-03-27T06:43:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.591+0000 7f2cb3465700 1 -- 172.21.15.16:0/3000821581 shutdown_connections 2024-03-27T06:43:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.592+0000 7f2cb3465700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f2c980a3720 0x7f2c980a5be0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.592+0000 7f2cb3465700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cac103110 0x7f2cac19fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.592+0000 7f2cb3465700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cac101570 0x7f2cac198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.592+0000 7f2cb3465700 1 --2- 172.21.15.16:0/3000821581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cac102770 0x7f2cac198ec0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.592+0000 7f2cb3465700 1 -- 172.21.15.16:0/3000821581 >> 172.21.15.16:0/3000821581 conn(0x7f2cac0fcb30 msgr2=0x7f2cac0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.592+0000 7f2cb3465700 1 -- 172.21.15.16:0/3000821581 shutdown_connections 2024-03-27T06:43:13.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:13.592+0000 7f2cb3465700 1 -- 172.21.15.16:0/3000821581 wait complete. 2024-03-27T06:43:13.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:13 smithi016 ceph-mon[31908]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:43:13.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:13 smithi156 ceph-mon[36261]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:43:13.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:13 smithi170 ceph-mon[36438]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:43:14.397 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-27T06:43:14.406 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-03-27T06:43:14.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-27T06:43:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:43:15.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:15 smithi156 ceph-mon[36261]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:43:15.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:15 smithi156 ceph-mon[36261]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:15.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:15 smithi170 ceph-mon[36438]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:43:15.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:15 smithi170 ceph-mon[36438]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:15.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:15 smithi016 ceph-mon[31908]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:43:15.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:15 smithi016 ceph-mon[31908]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.716+0000 7f8f54f15700 1 -- 172.21.15.16:0/1546441971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f50102670 msgr2=0x7f8f50102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.716+0000 7f8f54f15700 1 --2- 172.21.15.16:0/1546441971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f50102670 0x7f8f50102a90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8f400099c0 tx=0x7f8f40009cd0 comp rx=0 tx=0).stop 2024-03-27T06:43:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.716+0000 7f8f54f15700 1 -- 172.21.15.16:0/1546441971 shutdown_connections 2024-03-27T06:43:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.716+0000 7f8f54f15700 1 --2- 172.21.15.16:0/1546441971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f50104210 0x7f8f5010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.716+0000 7f8f54f15700 1 --2- 172.21.15.16:0/1546441971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f50102670 0x7f8f50102a90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.716+0000 7f8f54f15700 1 --2- 172.21.15.16:0/1546441971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f50103870 0x7f8f50103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.716+0000 7f8f54f15700 1 -- 172.21.15.16:0/1546441971 >> 172.21.15.16:0/1546441971 conn(0x7f8f500fdc50 msgr2=0x7f8f50100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.716+0000 7f8f54f15700 1 -- 172.21.15.16:0/1546441971 shutdown_connections 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.716+0000 7f8f54f15700 1 -- 172.21.15.16:0/1546441971 wait complete. 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f54f15700 1 Processor -- start 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f54f15700 1 -- start start 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f54f15700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f50102670 0x7f8f501989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f54f15700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f50103870 0x7f8f50198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f54f15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f50104210 0x7f8f5019ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f54f15700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5010e340 con 0x7f8f50104210 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f54f15700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5010e1c0 con 0x7f8f50103870 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f54f15700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5010e4c0 con 0x7f8f50102670 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f4ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f50104210 0x7f8f5019ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f4e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f50102670 0x7f8f501989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f4dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f50103870 0x7f8f50198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f4e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f50102670 0x7f8f501989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:48468/0 (socket says 172.21.15.16:48468) 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f4e59c700 1 -- 172.21.15.16:0/948583669 learned_addr learned my addr 172.21.15.16:0/948583669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.717+0000 7f8f4ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f50104210 0x7f8f5019ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47752/0 (socket says 172.21.15.16:47752) 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f4e59c700 1 -- 172.21.15.16:0/948583669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f50103870 msgr2=0x7f8f50198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f4e59c700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f50103870 0x7f8f50198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f4e59c700 1 -- 172.21.15.16:0/948583669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f50104210 msgr2=0x7f8f5019ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f4e59c700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f50104210 0x7f8f5019ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f4e59c700 1 -- 172.21.15.16:0/948583669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f501a0610 con 0x7f8f50102670 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f4ed9d700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f50104210 0x7f8f5019ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f4e59c700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f50102670 0x7f8f501989b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000f710 tx=0x7f8f4000f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f477fe700 1 -- 172.21.15.16:0/948583669 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40027070 con 0x7f8f50102670 2024-03-27T06:43:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f54f15700 1 -- 172.21.15.16:0/948583669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f501a08a0 con 0x7f8f50102670 2024-03-27T06:43:16.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f477fe700 1 -- 172.21.15.16:0/948583669 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8f4000be40 con 0x7f8f50102670 2024-03-27T06:43:16.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f477fe700 1 -- 172.21.15.16:0/948583669 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f400173f0 con 0x7f8f50102670 2024-03-27T06:43:16.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.718+0000 7f8f54f15700 1 -- 172.21.15.16:0/948583669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f501a0dd0 con 0x7f8f50102670 2024-03-27T06:43:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.719+0000 7f8f54f15700 1 -- 172.21.15.16:0/948583669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f50069300 con 0x7f8f50102670 2024-03-27T06:43:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.720+0000 7f8f477fe700 1 -- 172.21.15.16:0/948583669 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f8f4002a070 con 0x7f8f50102670 2024-03-27T06:43:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.721+0000 7f8f477fe700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8f3c0a36a0 0x7f8f3c0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.721+0000 7f8f4dd9b700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8f3c0a36a0 0x7f8f3c0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.721+0000 7f8f477fe700 1 -- 172.21.15.16:0/948583669 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8f400d0b80 con 0x7f8f50102670 2024-03-27T06:43:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.721+0000 7f8f4dd9b700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8f3c0a36a0 0x7f8f3c0a5b60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009c60 tx=0x7f8f38009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:16.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.724+0000 7f8f477fe700 1 -- 172.21.15.16:0/948583669 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8f40013070 con 0x7f8f50102670 2024-03-27T06:43:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.977+0000 7f8f54f15700 1 -- 172.21.15.16:0/948583669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8f5002dd30 con 0x7f8f50102670 2024-03-27T06:43:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.978+0000 7f8f477fe700 1 -- 172.21.15.16:0/948583669 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3090 (secure 0 0 0) 0x7f8f4009d820 con 0x7f8f50102670 2024-03-27T06:43:16.973 INFO:teuthology.orchestra.run.smithi016.stdout:e24 2024-03-27T06:43:16.973 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-27T06:43:16.973 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-27T06:43:16.973 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-03-27T06:43:16.973 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:16.973 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-03-27T06:43:16.973 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-03-27T06:43:16.973 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 24 2024-03-27T06:43:16.973 INFO:teuthology.orchestra.run.smithi016.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-27T06:43:16.973 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-03-27T06:41:28.746131+0000 2024-03-27T06:43:16.973 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-03-27T06:41:35.466274+0000 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 300 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 5 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:in 0,1,2,3,4 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24527,1=24500,2=24493,3=24472,4=24467} 2024-03-27T06:43:16.974 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 24527 members: 24472,24493,24500,24527,24467 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b{0:24527} state up:active seq 3 addr [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.f{0:24460} state up:standby-replay seq 1 addr [v2:172.21.15.170:6834/4279306199,v1:172.21.15.170:6835/4279306199] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.h{1:24500} state up:active seq 6 addr [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.k{1:24428} state up:standby-replay seq 1 addr [v2:172.21.15.156:6832/521338694,v1:172.21.15.156:6833/521338694] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.i{2:24493} state up:active seq 6 addr [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.975 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.l{2:24409} state up:standby-replay seq 1 addr [v2:172.21.15.170:6832/1368007304,v1:172.21.15.170:6833/1368007304] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.976 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c{3:24472} state up:active seq 7 addr [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.976 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a{3:14640} state up:standby-replay seq 1 addr [v2:172.21.15.16:6840/2365316099,v1:172.21.15.16:6841/2365316099] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.976 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.e{4:24467} state up:active seq 9 addr [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.976 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.g{4:14628} state up:standby-replay seq 1 addr [v2:172.21.15.16:6838/420981926,v1:172.21.15.16:6839/420981926] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:16.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:16.976 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-03-27T06:43:16.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:16.976 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.16:6834/1554082728,v1:172.21.15.16:6835/1554082728] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.976 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.16:6836/1241440055,v1:172.21.15.16:6837/1241440055] compat {c=[1],r=[1],i=[1fff]}] 2024-03-27T06:43:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 -- 172.21.15.16:0/948583669 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8f3c0a36a0 msgr2=0x7f8f3c0a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8f3c0a36a0 0x7f8f3c0a5b60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009c60 tx=0x7f8f38009260 comp rx=0 tx=0).stop 2024-03-27T06:43:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 -- 172.21.15.16:0/948583669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f50102670 msgr2=0x7f8f501989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f50102670 0x7f8f501989b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000f710 tx=0x7f8f4000f740 comp rx=0 tx=0).stop 2024-03-27T06:43:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 -- 172.21.15.16:0/948583669 shutdown_connections 2024-03-27T06:43:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f50104210 0x7f8f5019ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8f3c0a36a0 0x7f8f3c0a5b60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f50102670 0x7f8f501989b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 --2- 172.21.15.16:0/948583669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f50103870 0x7f8f50198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 -- 172.21.15.16:0/948583669 >> 172.21.15.16:0/948583669 conn(0x7f8f500fdc50 msgr2=0x7f8f50106c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 -- 172.21.15.16:0/948583669 shutdown_connections 2024-03-27T06:43:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:16.980+0000 7f8f54f15700 1 -- 172.21.15.16:0/948583669 wait complete. 2024-03-27T06:43:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-03-27T06:43:17.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:17 smithi016 ceph-mon[31908]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:43:17.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:17 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/948583669' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-27T06:43:17.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:17 smithi156 ceph-mon[36261]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:43:17.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:17 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/948583669' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-27T06:43:17.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-27T06:43:17.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:17 smithi170 ceph-mon[36438]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:43:17.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:17 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/948583669' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-27T06:43:18.069 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:43:19.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:19 smithi156 ceph-mon[36261]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:43:19.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:19 smithi016 ceph-mon[31908]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:43:19.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:19 smithi170 ceph-mon[36438]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:43:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.916+0000 7ff89cd9a700 1 -- 172.21.15.16:0/1161565747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff898102770 msgr2=0x7ff898102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.916+0000 7ff89cd9a700 1 --2- 172.21.15.16:0/1161565747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff898102770 0x7ff898102bd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff8880099c0 tx=0x7ff888009cd0 comp rx=0 tx=0).stop 2024-03-27T06:43:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.916+0000 7ff89cd9a700 1 -- 172.21.15.16:0/1161565747 shutdown_connections 2024-03-27T06:43:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.916+0000 7ff89cd9a700 1 --2- 172.21.15.16:0/1161565747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff898103110 0x7ff8981099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.916+0000 7ff89cd9a700 1 --2- 172.21.15.16:0/1161565747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff898101570 0x7ff898101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.916+0000 7ff89cd9a700 1 --2- 172.21.15.16:0/1161565747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff898102770 0x7ff898102bd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.916+0000 7ff89cd9a700 1 -- 172.21.15.16:0/1161565747 >> 172.21.15.16:0/1161565747 conn(0x7ff8980fcb10 msgr2=0x7ff8980fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.916+0000 7ff89cd9a700 1 -- 172.21.15.16:0/1161565747 shutdown_connections 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.916+0000 7ff89cd9a700 1 -- 172.21.15.16:0/1161565747 wait complete. 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff89cd9a700 1 Processor -- start 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff89cd9a700 1 -- start start 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff89cd9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff898101570 0x7ff89806ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff89cd9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff898102770 0x7ff89819fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff89cd9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff898103110 0x7ff8981a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff89cd9a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff89810d230 con 0x7ff898102770 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff89cd9a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ff89810d0b0 con 0x7ff898103110 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff89cd9a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff89810d3b0 con 0x7ff898101570 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff895d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff898102770 0x7ff89819fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff895d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff898102770 0x7ff89819fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47774/0 (socket says 172.21.15.16:47774) 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff895d9b700 1 -- 172.21.15.16:0/3628775482 learned_addr learned my addr 172.21.15.16:0/3628775482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff896d9d700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff898103110 0x7ff8981a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff89659c700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff898101570 0x7ff89806ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff895d9b700 1 -- 172.21.15.16:0/3628775482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff898101570 msgr2=0x7ff89806ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff895d9b700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff898101570 0x7ff89806ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff895d9b700 1 -- 172.21.15.16:0/3628775482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff898103110 msgr2=0x7ff8981a2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff895d9b700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff898103110 0x7ff8981a2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.917+0000 7ff895d9b700 1 -- 172.21.15.16:0/3628775482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8981a2800 con 0x7ff898102770 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.918+0000 7ff895d9b700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff898102770 0x7ff89819fad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff888006010 tx=0x7ff88800b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.918+0000 7ff8877fe700 1 -- 172.21.15.16:0/3628775482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88800be40 con 0x7ff898102770 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.918+0000 7ff89cd9a700 1 -- 172.21.15.16:0/3628775482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8981a2a90 con 0x7ff898102770 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.918+0000 7ff8877fe700 1 -- 172.21.15.16:0/3628775482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff88800f460 con 0x7ff898102770 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.918+0000 7ff8877fe700 1 -- 172.21.15.16:0/3628775482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888017650 con 0x7ff898102770 2024-03-27T06:43:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.918+0000 7ff89cd9a700 1 -- 172.21.15.16:0/3628775482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8981a2f40 con 0x7ff898102770 2024-03-27T06:43:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.919+0000 7ff89cd9a700 1 -- 172.21.15.16:0/3628775482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff898069300 con 0x7ff898102770 2024-03-27T06:43:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.920+0000 7ff8877fe700 1 -- 172.21.15.16:0/3628775482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7ff88800f610 con 0x7ff898102770 2024-03-27T06:43:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.920+0000 7ff8877fe700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8800a36a0 0x7ff8800a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.921+0000 7ff89659c700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8800a36a0 0x7ff8800a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.921+0000 7ff8877fe700 1 -- 172.21.15.16:0/3628775482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ff8880d0be0 con 0x7ff898102770 2024-03-27T06:43:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.921+0000 7ff89659c700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8800a36a0 0x7ff8800a5b60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8981025d0 tx=0x7ff88c007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:19.924+0000 7ff8877fe700 1 -- 172.21.15.16:0/3628775482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff888013070 con 0x7ff898102770 2024-03-27T06:43:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.131+0000 7ff89cd9a700 1 -- 172.21.15.16:0/3628775482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7ff89802dd60 con 0x7ff898102770 2024-03-27T06:43:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.132+0000 7ff8877fe700 1 -- 172.21.15.16:0/3628775482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5269 (secure 0 0 0) 0x7ff88809d850 con 0x7ff898102770 2024-03-27T06:43:20.127 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 88 2024-03-27T06:43:20.127 INFO:teuthology.orchestra.run.smithi016.stdout:fsid c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:43:20.127 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-03-27T06:33:25.534035+0000 2024-03-27T06:43:20.127 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-03-27T06:42:20.647546+0000 2024-03-27T06:43:20.127 INFO:teuthology.orchestra.run.smithi016.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-27T06:43:20.127 INFO:teuthology.orchestra.run.smithi016.stdout:crush_version 26 2024-03-27T06:43:20.127 INFO:teuthology.orchestra.run.smithi016.stdout:full_ratio 0.95 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:backfillfull_ratio 0.9 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:nearfull_ratio 0.85 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:require_min_compat_client luminous 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:min_compat_client luminous 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:require_osd_release squid 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:stretch_mode_enabled false 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:max_osd 12 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6802/1317357078,v1:172.21.15.16:6803/1317357078] [v2:172.21.15.16:6804/1317357078,v1:172.21.15.16:6805/1317357078] exists,up e99199df-d2f6-4677-8f3c-b96df2232304 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6800/1327802995,v1:172.21.15.156:6801/1327802995] [v2:172.21.15.156:6802/1327802995,v1:172.21.15.156:6803/1327802995] exists,up 948ae15e-e954-4653-9d1d-da3bac45bd28 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/3044432379,v1:172.21.15.170:6801/3044432379] [v2:172.21.15.170:6802/3044432379,v1:172.21.15.170:6803/3044432379] exists,up a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6810/875136606,v1:172.21.15.16:6811/875136606] [v2:172.21.15.16:6812/875136606,v1:172.21.15.16:6813/875136606] exists,up f77f4328-207f-48e0-bd4e-8c2cb212ae4f 2024-03-27T06:43:20.128 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6808/3654211974,v1:172.21.15.156:6809/3654211974] [v2:172.21.15.156:6810/3654211974,v1:172.21.15.156:6811/3654211974] exists,up 78647188-4493-4037-97d1-92e1f6f82fc2 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/493340882,v1:172.21.15.170:6809/493340882] [v2:172.21.15.170:6810/493340882,v1:172.21.15.170:6811/493340882] exists,up 9f4efed7-91d9-4adc-881a-ce3f44f80813 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6818/237875547,v1:172.21.15.16:6819/237875547] [v2:172.21.15.16:6820/237875547,v1:172.21.15.16:6821/237875547] exists,up ffbf31b8-a44e-4f81-819b-48213bf4cc84 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6816/229134476,v1:172.21.15.156:6817/229134476] [v2:172.21.15.156:6818/229134476,v1:172.21.15.156:6819/229134476] exists,up fb83f843-12b4-4b70-9a85-dd2ca56ed9d8 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6816/504489551,v1:172.21.15.170:6817/504489551] [v2:172.21.15.170:6818/504489551,v1:172.21.15.170:6819/504489551] exists,up 4849c1e0-60fa-4333-b423-20b759447717 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 up in weight 1 up_from 68 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6826/2308288184,v1:172.21.15.16:6827/2308288184] [v2:172.21.15.16:6828/2308288184,v1:172.21.15.16:6829/2308288184] exists,up d6bb6987-e0df-4d98-9044-63946f8567cd 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6824/399510561,v1:172.21.15.156:6825/399510561] [v2:172.21.15.156:6826/399510561,v1:172.21.15.156:6827/399510561] exists,up eb7e2309-4c4e-496e-9043-ff264a805f66 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6824/703899164,v1:172.21.15.170:6825/703899164] [v2:172.21.15.170:6826/703899164,v1:172.21.15.170:6827/703899164] exists,up a73e6132-619a-4b5c-85a7-e5c3ee40188e 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.5 [8,11] 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.e [2,4] 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.10 [8,11] 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.16 [2,4] 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.1 [4,9] 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.13 [4,9] 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.20 [5,9] 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.23 [4,8] 2024-03-27T06:43:20.129 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3456912888 expires 2024-03-28T06:34:19.478814+0000 2024-03-27T06:43:20.130 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/1685933023 expires 2024-03-28T06:34:19.478814+0000 2024-03-27T06:43:20.130 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/1685933023 expires 2024-03-28T06:34:19.478814+0000 2024-03-27T06:43:20.130 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2765383798 expires 2024-03-28T06:34:19.478814+0000 2024-03-27T06:43:20.130 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2967489845 expires 2024-03-28T06:33:51.395962+0000 2024-03-27T06:43:20.130 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3026086781 expires 2024-03-28T06:33:51.395962+0000 2024-03-27T06:43:20.130 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2945509975 expires 2024-03-28T06:33:51.395962+0000 2024-03-27T06:43:20.130 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/1221908125 expires 2024-03-28T06:33:51.395962+0000 2024-03-27T06:43:20.130 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/1221908125 expires 2024-03-28T06:33:51.395962+0000 2024-03-27T06:43:20.130 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/1636919118 expires 2024-03-28T06:34:19.478814+0000 2024-03-27T06:43:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.135+0000 7ff89cd9a700 1 -- 172.21.15.16:0/3628775482 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8800a36a0 msgr2=0x7ff8800a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.135+0000 7ff89cd9a700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8800a36a0 0x7ff8800a5b60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8981025d0 tx=0x7ff88c007400 comp rx=0 tx=0).stop 2024-03-27T06:43:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.135+0000 7ff89cd9a700 1 -- 172.21.15.16:0/3628775482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff898102770 msgr2=0x7ff89819fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.135+0000 7ff89cd9a700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff898102770 0x7ff89819fad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff888006010 tx=0x7ff88800b9c0 comp rx=0 tx=0).stop 2024-03-27T06:43:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.135+0000 7ff89cd9a700 1 -- 172.21.15.16:0/3628775482 shutdown_connections 2024-03-27T06:43:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.135+0000 7ff89cd9a700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff898103110 0x7ff8981a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.135+0000 7ff89cd9a700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7ff8800a36a0 0x7ff8800a5b60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.135+0000 7ff89cd9a700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff898101570 0x7ff89806ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.135+0000 7ff89cd9a700 1 --2- 172.21.15.16:0/3628775482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff898102770 0x7ff89819fad0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.135+0000 7ff89cd9a700 1 -- 172.21.15.16:0/3628775482 >> 172.21.15.16:0/3628775482 conn(0x7ff8980fcb10 msgr2=0x7ff898105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.136+0000 7ff89cd9a700 1 -- 172.21.15.16:0/3628775482 shutdown_connections 2024-03-27T06:43:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:20.136+0000 7ff89cd9a700 1 -- 172.21.15.16:0/3628775482 wait complete. 2024-03-27T06:43:20.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:20 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3628775482' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-27T06:43:20.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:20 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3628775482' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-27T06:43:20.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:20 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3628775482' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-27T06:43:21.013 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-27T06:43:21.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:43:21.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.138+0000 7f5218bc7640 1 -- 172.21.15.16:0/3538726313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52141078f0 msgr2=0x7f5214109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.138+0000 7f5218bc7640 1 --2- 172.21.15.16:0/3538726313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52141078f0 0x7f5214109ce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f520c009ca0 tx=0x7f520c02c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.138+0000 7f5218bc7640 1 -- 172.21.15.16:0/3538726313 shutdown_connections 2024-03-27T06:43:21.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.138+0000 7f5218bc7640 1 --2- 172.21.15.16:0/3538726313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52141078f0 0x7f5214109ce0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.138+0000 7f5218bc7640 1 --2- 172.21.15.16:0/3538726313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5214104fc0 0x7f52141073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.138+0000 7f5218bc7640 1 --2- 172.21.15.16:0/3538726313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5214102690 0x7f5214104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.138+0000 7f5218bc7640 1 -- 172.21.15.16:0/3538726313 >> 172.21.15.16:0/3538726313 conn(0x7f52140fbfe0 msgr2=0x7f52140fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:21.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.138+0000 7f5218bc7640 1 -- 172.21.15.16:0/3538726313 shutdown_connections 2024-03-27T06:43:21.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.138+0000 7f5218bc7640 1 -- 172.21.15.16:0/3538726313 wait complete. 2024-03-27T06:43:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f5218bc7640 1 Processor -- start 2024-03-27T06:43:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f5218bc7640 1 -- start start 2024-03-27T06:43:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f5218bc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214102690 0x7f5214197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f5218bc7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5214104fc0 0x7f5214198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f5218bc7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52141078f0 0x7f521419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f5218bc7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f521410d940 con 0x7f5214102690 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f5218bc7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f521410d7c0 con 0x7f5214104fc0 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f5218bc7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f521410dac0 con 0x7f52141078f0 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f521259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214102690 0x7f5214197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f521259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214102690 0x7f5214197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47792/0 (socket says 172.21.15.16:47792) 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f521259c640 1 -- 172.21.15.16:0/1872575068 learned_addr learned my addr 172.21.15.16:0/1872575068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f5212d9d640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52141078f0 0x7f521419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f5211d9b640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5214104fc0 0x7f5214198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f521259c640 1 -- 172.21.15.16:0/1872575068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52141078f0 msgr2=0x7f521419f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f521259c640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52141078f0 0x7f521419f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f521259c640 1 -- 172.21.15.16:0/1872575068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5214104fc0 msgr2=0x7f5214198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f521259c640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5214104fc0 0x7f5214198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.139+0000 7f521259c640 1 -- 172.21.15.16:0/1872575068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521419faf0 con 0x7f5214102690 2024-03-27T06:43:21.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.140+0000 7f521259c640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214102690 0x7f5214197ed0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f520400e9f0 tx=0x7f520400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.140+0000 7f52037fe640 1 -- 172.21.15.16:0/1872575068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520400cd80 con 0x7f5214102690 2024-03-27T06:43:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.140+0000 7f52037fe640 1 -- 172.21.15.16:0/1872575068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5204010930 con 0x7f5214102690 2024-03-27T06:43:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.140+0000 7f5218bc7640 1 -- 172.21.15.16:0/1872575068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521419fde0 con 0x7f5214102690 2024-03-27T06:43:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.140+0000 7f52037fe640 1 -- 172.21.15.16:0/1872575068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204018b00 con 0x7f5214102690 2024-03-27T06:43:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.140+0000 7f5218bc7640 1 -- 172.21.15.16:0/1872575068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52141a0240 con 0x7f5214102690 2024-03-27T06:43:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.141+0000 7f5218bc7640 1 -- 172.21.15.16:0/1872575068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52140fdad0 con 0x7f5214102690 2024-03-27T06:43:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.143+0000 7f52037fe640 1 -- 172.21.15.16:0/1872575068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5204018cc0 con 0x7f5214102690 2024-03-27T06:43:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.143+0000 7f52037fe640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51fc0a3610 0x7f51fc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.144+0000 7f5211d9b640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51fc0a3610 0x7f51fc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.144+0000 7f52037fe640 1 -- 172.21.15.16:0/1872575068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f52040d0390 con 0x7f5214102690 2024-03-27T06:43:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.144+0000 7f5211d9b640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51fc0a3610 0x7f51fc0a5ad0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f52141993e0 tx=0x7f5208005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.147+0000 7f52037fe640 1 -- 172.21.15.16:0/1872575068 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f520409c920 con 0x7f5214102690 2024-03-27T06:43:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.335+0000 7f5218bc7640 1 -- 172.21.15.16:0/1872575068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f521402d5e0 con 0x7f5214102690 2024-03-27T06:43:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.336+0000 7f52037fe640 1 -- 172.21.15.16:0/1872575068 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7f52040a0c20 con 0x7f5214102690 2024-03-27T06:43:21.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:21.331 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:42:20.647546+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:43:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 -- 172.21.15.16:0/1872575068 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51fc0a3610 msgr2=0x7f51fc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51fc0a3610 0x7f51fc0a5ad0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f52141993e0 tx=0x7f5208005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 -- 172.21.15.16:0/1872575068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214102690 msgr2=0x7f5214197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214102690 0x7f5214197ed0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f520400e9f0 tx=0x7f520400eeb0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 -- 172.21.15.16:0/1872575068 shutdown_connections 2024-03-27T06:43:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51fc0a3610 0x7f51fc0a5ad0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52141078f0 0x7f521419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5214104fc0 0x7f5214198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 --2- 172.21.15.16:0/1872575068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5214102690 0x7f5214197ed0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 -- 172.21.15.16:0/1872575068 >> 172.21.15.16:0/1872575068 conn(0x7f52140fbfe0 msgr2=0x7f5214102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 -- 172.21.15.16:0/1872575068 shutdown_connections 2024-03-27T06:43:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.338+0000 7f5218bc7640 1 -- 172.21.15.16:0/1872575068 wait complete. 2024-03-27T06:43:21.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T06:41:26.393978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T06:41:27.707469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:43:21.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T06:43:21.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.460+0000 7f5fd8125640 1 -- 172.21.15.16:0/60351299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd00ffe70 msgr2=0x7f5fd0100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.460+0000 7f5fd8125640 1 --2- 172.21.15.16:0/60351299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd00ffe70 0x7f5fd0100290 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009930 tx=0x7f5fcc02bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.460+0000 7f5fd8125640 1 -- 172.21.15.16:0/60351299 shutdown_connections 2024-03-27T06:43:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.460+0000 7f5fd8125640 1 --2- 172.21.15.16:0/60351299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd01038a0 0x7f5fd010a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.460+0000 7f5fd8125640 1 --2- 172.21.15.16:0/60351299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fd0102ee0 0x7f5fd0103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.460+0000 7f5fd8125640 1 --2- 172.21.15.16:0/60351299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd00ffe70 0x7f5fd0100290 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.460+0000 7f5fd8125640 1 -- 172.21.15.16:0/60351299 >> 172.21.15.16:0/60351299 conn(0x7f5fd00fb780 msgr2=0x7f5fd00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd8125640 1 -- 172.21.15.16:0/60351299 shutdown_connections 2024-03-27T06:43:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd8125640 1 -- 172.21.15.16:0/60351299 wait complete. 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd8125640 1 Processor -- start 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd8125640 1 -- start start 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd8125640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fd00ffe70 0x7f5fd0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd8125640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0102ee0 0x7f5fd0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd8125640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd01038a0 0x7f5fd019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd8125640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd010d900 con 0x7f5fd0102ee0 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd8125640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd010d780 con 0x7f5fd01038a0 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd8125640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd010da80 con 0x7f5fd00ffe70 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd56c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0102ee0 0x7f5fd0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd56c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0102ee0 0x7f5fd0198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47820/0 (socket says 172.21.15.16:47820) 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd56c0640 1 -- 172.21.15.16:0/2035293042 learned_addr learned my addr 172.21.15.16:0/2035293042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd5ec1640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fd00ffe70 0x7f5fd0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.461+0000 7f5fd66c2640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd01038a0 0x7f5fd019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.462+0000 7f5fd56c0640 1 -- 172.21.15.16:0/2035293042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fd00ffe70 msgr2=0x7f5fd0197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.462+0000 7f5fd56c0640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fd00ffe70 0x7f5fd0197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.462+0000 7f5fd56c0640 1 -- 172.21.15.16:0/2035293042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd01038a0 msgr2=0x7f5fd019f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.462+0000 7f5fd56c0640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd01038a0 0x7f5fd019f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.462+0000 7f5fd56c0640 1 -- 172.21.15.16:0/2035293042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd019fbf0 con 0x7f5fd0102ee0 2024-03-27T06:43:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.462+0000 7f5fd56c0640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0102ee0 0x7f5fd0198510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000e9f0 tx=0x7f5fc000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.462+0000 7f5fc6ffd640 1 -- 172.21.15.16:0/2035293042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000cd80 con 0x7f5fd0102ee0 2024-03-27T06:43:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.462+0000 7f5fc6ffd640 1 -- 172.21.15.16:0/2035293042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5fc0005520 con 0x7f5fd0102ee0 2024-03-27T06:43:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.462+0000 7f5fd8125640 1 -- 172.21.15.16:0/2035293042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd019fee0 con 0x7f5fd0102ee0 2024-03-27T06:43:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.462+0000 7f5fc6ffd640 1 -- 172.21.15.16:0/2035293042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0010690 con 0x7f5fd0102ee0 2024-03-27T06:43:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.462+0000 7f5fd8125640 1 -- 172.21.15.16:0/2035293042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd01a0350 con 0x7f5fd0102ee0 2024-03-27T06:43:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.463+0000 7f5fd8125640 1 -- 172.21.15.16:0/2035293042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd0109060 con 0x7f5fd0102ee0 2024-03-27T06:43:21.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.464+0000 7f5fc6ffd640 1 -- 172.21.15.16:0/2035293042 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f5fc00046c0 con 0x7f5fd0102ee0 2024-03-27T06:43:21.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.465+0000 7f5fc6ffd640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5fbc0a3610 0x7f5fbc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.465+0000 7f5fd5ec1640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5fbc0a3610 0x7f5fbc0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.465+0000 7f5fc6ffd640 1 -- 172.21.15.16:0/2035293042 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5fc00d00d0 con 0x7f5fd0102ee0 2024-03-27T06:43:21.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.465+0000 7f5fd5ec1640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5fbc0a3610 0x7f5fbc0a5ad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc02c900 tx=0x7f5fcc005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.469+0000 7f5fc6ffd640 1 -- 172.21.15.16:0/2035293042 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5fc009ccb0 con 0x7f5fd0102ee0 2024-03-27T06:43:21.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:21 smithi016 ceph-mon[31908]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:43:21.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:21 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1872575068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.656+0000 7f5fd8125640 1 -- 172.21.15.16:0/2035293042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5fd00627f0 con 0x7f5fd0102ee0 2024-03-27T06:43:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.657+0000 7f5fc6ffd640 1 -- 172.21.15.16:0/2035293042 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f5fc009c5b0 con 0x7f5fd0102ee0 2024-03-27T06:43:21.651 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:43:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.658+0000 7f5fd8125640 1 -- 172.21.15.16:0/2035293042 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5fbc0a3610 msgr2=0x7f5fbc0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.658+0000 7f5fd8125640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5fbc0a3610 0x7f5fbc0a5ad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc02c900 tx=0x7f5fcc005fd0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.658+0000 7f5fd8125640 1 -- 172.21.15.16:0/2035293042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0102ee0 msgr2=0x7f5fd0198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.658+0000 7f5fd8125640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0102ee0 0x7f5fd0198510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000e9f0 tx=0x7f5fc000edf0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.658+0000 7f5fd8125640 1 -- 172.21.15.16:0/2035293042 shutdown_connections 2024-03-27T06:43:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.658+0000 7f5fd8125640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f5fbc0a3610 0x7f5fbc0a5ad0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.658+0000 7f5fd8125640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd01038a0 0x7f5fd019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.658+0000 7f5fd8125640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0102ee0 0x7f5fd0198510 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.659+0000 7f5fd8125640 1 --2- 172.21.15.16:0/2035293042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fd00ffe70 0x7f5fd0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.659+0000 7f5fd8125640 1 -- 172.21.15.16:0/2035293042 >> 172.21.15.16:0/2035293042 conn(0x7f5fd00fb780 msgr2=0x7f5fd0106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.659+0000 7f5fd8125640 1 -- 172.21.15.16:0/2035293042 shutdown_connections 2024-03-27T06:43:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.659+0000 7f5fd8125640 1 -- 172.21.15.16:0/2035293042 wait complete. 2024-03-27T06:43:21.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T06:43:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.776+0000 7f786794b640 1 -- 172.21.15.16:0/586608023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860103720 msgr2=0x7f7860109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.776+0000 7f786794b640 1 --2- 172.21.15.16:0/586608023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860103720 0x7f7860109fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7858009ca0 tx=0x7f785802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.776+0000 7f786794b640 1 -- 172.21.15.16:0/586608023 shutdown_connections 2024-03-27T06:43:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.776+0000 7f786794b640 1 --2- 172.21.15.16:0/586608023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860103720 0x7f7860109fb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.776+0000 7f786794b640 1 --2- 172.21.15.16:0/586608023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7860102d60 0x7f78601031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.776+0000 7f786794b640 1 --2- 172.21.15.16:0/586608023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7860101b70 0x7f7860101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.776+0000 7f786794b640 1 -- 172.21.15.16:0/586608023 >> 172.21.15.16:0/586608023 conn(0x7f78600fd0e0 msgr2=0x7f78600ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f786794b640 1 -- 172.21.15.16:0/586608023 shutdown_connections 2024-03-27T06:43:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f786794b640 1 -- 172.21.15.16:0/586608023 wait complete. 2024-03-27T06:43:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f786794b640 1 Processor -- start 2024-03-27T06:43:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f786794b640 1 -- start start 2024-03-27T06:43:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f786794b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7860101b70 0x7f7860075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f786794b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860102d60 0x7f7860073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f786794b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7860103720 0x7f7860074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f786794b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f786010d920 con 0x7f7860102d60 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f786794b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f786010d7a0 con 0x7f7860101b70 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f786794b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f786010daa0 con 0x7f7860103720 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f7864ee6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860102d60 0x7f7860073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f7864ee6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860102d60 0x7f7860073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47840/0 (socket says 172.21.15.16:47840) 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f7864ee6640 1 -- 172.21.15.16:0/1904875182 learned_addr learned my addr 172.21.15.16:0/1904875182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f7865ee8640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7860103720 0x7f7860074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.777+0000 7f78656e7640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7860101b70 0x7f7860075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.778+0000 7f7864ee6640 1 -- 172.21.15.16:0/1904875182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7860103720 msgr2=0x7f7860074480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.778+0000 7f7864ee6640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7860103720 0x7f7860074480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.778+0000 7f7864ee6640 1 -- 172.21.15.16:0/1904875182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7860101b70 msgr2=0x7f7860075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.778+0000 7f7864ee6640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7860101b70 0x7f7860075900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.778+0000 7f7864ee6640 1 -- 172.21.15.16:0/1904875182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7860074d10 con 0x7f7860102d60 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.778+0000 7f7864ee6640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860102d60 0x7f7860073f40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f785000d950 tx=0x7f785000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.778+0000 7f78567fc640 1 -- 172.21.15.16:0/1904875182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850017030 con 0x7f7860102d60 2024-03-27T06:43:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.778+0000 7f786794b640 1 -- 172.21.15.16:0/1904875182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78601a1fd0 con 0x7f7860102d60 2024-03-27T06:43:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.778+0000 7f78567fc640 1 -- 172.21.15.16:0/1904875182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7850004020 con 0x7f7860102d60 2024-03-27T06:43:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.778+0000 7f78567fc640 1 -- 172.21.15.16:0/1904875182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850004a00 con 0x7f7860102d60 2024-03-27T06:43:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.778+0000 7f786794b640 1 -- 172.21.15.16:0/1904875182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78601a2430 con 0x7f7860102d60 2024-03-27T06:43:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.779+0000 7f786794b640 1 -- 172.21.15.16:0/1904875182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7860108e30 con 0x7f7860102d60 2024-03-27T06:43:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.780+0000 7f78567fc640 1 -- 172.21.15.16:0/1904875182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f78500041d0 con 0x7f7860102d60 2024-03-27T06:43:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.780+0000 7f78567fc640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f784c0a3610 0x7f784c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.781+0000 7f78656e7640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f784c0a3610 0x7f784c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.781+0000 7f78567fc640 1 -- 172.21.15.16:0/1904875182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f78500cfcd0 con 0x7f7860102d60 2024-03-27T06:43:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.781+0000 7f78656e7640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f784c0a3610 0x7f784c0a5ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7860102bc0 tx=0x7f785c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:21.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.784+0000 7f78567fc640 1 -- 172.21.15.16:0/1904875182 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f785009c850 con 0x7f7860102d60 2024-03-27T06:43:21.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:21 smithi170 ceph-mon[36438]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:43:21.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:21 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1872575068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:21.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.968+0000 7f786794b640 1 -- 172.21.15.16:0/1904875182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f786002d5c0 con 0x7f7860102d60 2024-03-27T06:43:21.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.969+0000 7f78567fc640 1 -- 172.21.15.16:0/1904875182 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f785009c150 con 0x7f7860102d60 2024-03-27T06:43:21.964 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-03-27T06:43:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 -- 172.21.15.16:0/1904875182 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f784c0a3610 msgr2=0x7f784c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f784c0a3610 0x7f784c0a5ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7860102bc0 tx=0x7f785c005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 -- 172.21.15.16:0/1904875182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860102d60 msgr2=0x7f7860073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860102d60 0x7f7860073f40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f785000d950 tx=0x7f785000de10 comp rx=0 tx=0).stop 2024-03-27T06:43:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 -- 172.21.15.16:0/1904875182 shutdown_connections 2024-03-27T06:43:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f784c0a3610 0x7f784c0a5ad0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7860103720 0x7f7860074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860102d60 0x7f7860073f40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 --2- 172.21.15.16:0/1904875182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7860101b70 0x7f7860075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 -- 172.21.15.16:0/1904875182 >> 172.21.15.16:0/1904875182 conn(0x7f78600fd0e0 msgr2=0x7f7860106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 -- 172.21.15.16:0/1904875182 shutdown_connections 2024-03-27T06:43:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:21.971+0000 7f786794b640 1 -- 172.21.15.16:0/1904875182 wait complete. 2024-03-27T06:43:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:21 smithi156 ceph-mon[36261]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-27T06:43:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:21 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1872575068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:21.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T06:43:22.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.087+0000 7f16d09c7640 1 -- 172.21.15.16:0/3882592526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc102600 msgr2=0x7f16cc108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.087+0000 7f16d09c7640 1 --2- 172.21.15.16:0/3882592526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc102600 0x7f16cc108e90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009ca0 tx=0x7f16bc02c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.087+0000 7f16d09c7640 1 -- 172.21.15.16:0/3882592526 shutdown_connections 2024-03-27T06:43:22.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.087+0000 7f16d09c7640 1 --2- 172.21.15.16:0/3882592526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc102600 0x7f16cc108e90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.087+0000 7f16d09c7640 1 --2- 172.21.15.16:0/3882592526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16cc101c40 0x7f16cc1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.087+0000 7f16d09c7640 1 --2- 172.21.15.16:0/3882592526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16cc100a50 0x7f16cc100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.087+0000 7f16d09c7640 1 -- 172.21.15.16:0/3882592526 >> 172.21.15.16:0/3882592526 conn(0x7f16cc0fbfe0 msgr2=0x7f16cc0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:22.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.087+0000 7f16d09c7640 1 -- 172.21.15.16:0/3882592526 shutdown_connections 2024-03-27T06:43:22.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.087+0000 7f16d09c7640 1 -- 172.21.15.16:0/3882592526 wait complete. 2024-03-27T06:43:22.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16d09c7640 1 Processor -- start 2024-03-27T06:43:22.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16d09c7640 1 -- start start 2024-03-27T06:43:22.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16d09c7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16cc100a50 0x7f16cc069820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16d09c7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16cc101c40 0x7f16cc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16d09c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc102600 0x7f16cc19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16d09c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc10c800 con 0x7f16cc102600 2024-03-27T06:43:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16d09c7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc10c680 con 0x7f16cc101c40 2024-03-27T06:43:22.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16d09c7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc10c980 con 0x7f16cc100a50 2024-03-27T06:43:22.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc102600 0x7f16cc19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc102600 0x7f16cc19f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47866/0 (socket says 172.21.15.16:47866) 2024-03-27T06:43:22.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16cad9d640 1 -- 172.21.15.16:0/1989740925 learned_addr learned my addr 172.21.15.16:0/1989740925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:22.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16ca59c640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16cc100a50 0x7f16cc069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16c9d9b640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16cc101c40 0x7f16cc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16cad9d640 1 -- 172.21.15.16:0/1989740925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16cc100a50 msgr2=0x7f16cc069820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.088+0000 7f16cad9d640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16cc100a50 0x7f16cc069820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.089+0000 7f16cad9d640 1 -- 172.21.15.16:0/1989740925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16cc101c40 msgr2=0x7f16cc19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.089+0000 7f16cad9d640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16cc101c40 0x7f16cc19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.089+0000 7f16cad9d640 1 -- 172.21.15.16:0/1989740925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc19fb90 con 0x7f16cc102600 2024-03-27T06:43:22.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.089+0000 7f16cad9d640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc102600 0x7f16cc19f280 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f16bc02c700 tx=0x7f16bc0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:22.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.089+0000 7f16c37fe640 1 -- 172.21.15.16:0/1989740925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc039040 con 0x7f16cc102600 2024-03-27T06:43:22.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.089+0000 7f16c37fe640 1 -- 172.21.15.16:0/1989740925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16bc044070 con 0x7f16cc102600 2024-03-27T06:43:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.089+0000 7f16d09c7640 1 -- 172.21.15.16:0/1989740925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc19fdc0 con 0x7f16cc102600 2024-03-27T06:43:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.089+0000 7f16c37fe640 1 -- 172.21.15.16:0/1989740925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc0043c0 con 0x7f16cc102600 2024-03-27T06:43:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.089+0000 7f16d09c7640 1 -- 172.21.15.16:0/1989740925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc1a0250 con 0x7f16cc102600 2024-03-27T06:43:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.090+0000 7f16d09c7640 1 -- 172.21.15.16:0/1989740925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16cc107d10 con 0x7f16cc102600 2024-03-27T06:43:22.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.091+0000 7f16c37fe640 1 -- 172.21.15.16:0/1989740925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f16bc0045a0 con 0x7f16cc102600 2024-03-27T06:43:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.092+0000 7f16c37fe640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f16b80a3610 0x7f16b80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.092+0000 7f16ca59c640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f16b80a3610 0x7f16b80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.092+0000 7f16c37fe640 1 -- 172.21.15.16:0/1989740925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f16bc0f22f0 con 0x7f16cc102600 2024-03-27T06:43:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.092+0000 7f16ca59c640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f16b80a3610 0x7f16b80a5ad0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f16cc101aa0 tx=0x7f16c4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:22.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.097+0000 7f16c37fe640 1 -- 172.21.15.16:0/1989740925 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f16bc0beed0 con 0x7f16cc102600 2024-03-27T06:43:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.281+0000 7f16d09c7640 1 -- 172.21.15.16:0/1989740925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f16cc062850 con 0x7f16cc102600 2024-03-27T06:43:22.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.282+0000 7f16c37fe640 1 -- 172.21.15.16:0/1989740925 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f16bc0be7d0 con 0x7f16cc102600 2024-03-27T06:43:22.277 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-03-27T06:43:22.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 -- 172.21.15.16:0/1989740925 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f16b80a3610 msgr2=0x7f16b80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f16b80a3610 0x7f16b80a5ad0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f16cc101aa0 tx=0x7f16c4005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:22.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 -- 172.21.15.16:0/1989740925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc102600 msgr2=0x7f16cc19f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc102600 0x7f16cc19f280 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f16bc02c700 tx=0x7f16bc0346d0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 -- 172.21.15.16:0/1989740925 shutdown_connections 2024-03-27T06:43:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f16b80a3610 0x7f16b80a5ad0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc102600 0x7f16cc19f280 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16cc101c40 0x7f16cc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 --2- 172.21.15.16:0/1989740925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16cc100a50 0x7f16cc069820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 -- 172.21.15.16:0/1989740925 >> 172.21.15.16:0/1989740925 conn(0x7f16cc0fbfe0 msgr2=0x7f16cc104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 -- 172.21.15.16:0/1989740925 shutdown_connections 2024-03-27T06:43:22.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.284+0000 7f16d09c7640 1 -- 172.21.15.16:0/1989740925 wait complete. 2024-03-27T06:43:22.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:43:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.402+0000 7fec2091e640 1 -- 172.21.15.16:0/3420578907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c102630 msgr2=0x7fec1c108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.402+0000 7fec2091e640 1 --2- 172.21.15.16:0/3420578907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c102630 0x7fec1c108ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fec14009ca0 tx=0x7fec1402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.402+0000 7fec2091e640 1 -- 172.21.15.16:0/3420578907 shutdown_connections 2024-03-27T06:43:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.402+0000 7fec2091e640 1 --2- 172.21.15.16:0/3420578907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c102630 0x7fec1c108ec0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.402+0000 7fec2091e640 1 --2- 172.21.15.16:0/3420578907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec1c101c70 0x7fec1c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.402+0000 7fec2091e640 1 --2- 172.21.15.16:0/3420578907 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec1c100a80 0x7fec1c100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.402+0000 7fec2091e640 1 -- 172.21.15.16:0/3420578907 >> 172.21.15.16:0/3420578907 conn(0x7fec1c0fc030 msgr2=0x7fec1c0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.402+0000 7fec2091e640 1 -- 172.21.15.16:0/3420578907 shutdown_connections 2024-03-27T06:43:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.402+0000 7fec2091e640 1 -- 172.21.15.16:0/3420578907 wait complete. 2024-03-27T06:43:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.402+0000 7fec2091e640 1 Processor -- start 2024-03-27T06:43:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec2091e640 1 -- start start 2024-03-27T06:43:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec2091e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c100a80 0x7fec1c104a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec2091e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec1c101c70 0x7fec1c103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec2091e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec1c102630 0x7fec1c103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec2091e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c10c830 con 0x7fec1c100a80 2024-03-27T06:43:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec2091e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c10c6b0 con 0x7fec1c102630 2024-03-27T06:43:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec2091e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c10c9b0 con 0x7fec1c101c70 2024-03-27T06:43:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec1a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c100a80 0x7fec1c104a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec1a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c100a80 0x7fec1c104a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47896/0 (socket says 172.21.15.16:47896) 2024-03-27T06:43:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec1a59c640 1 -- 172.21.15.16:0/1157053638 learned_addr learned my addr 172.21.15.16:0/1157053638 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec1ad9d640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec1c102630 0x7fec1c103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec13fff640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec1c101c70 0x7fec1c103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec1ad9d640 1 -- 172.21.15.16:0/1157053638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec1c101c70 msgr2=0x7fec1c103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec1ad9d640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec1c101c70 0x7fec1c103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec1ad9d640 1 -- 172.21.15.16:0/1157053638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c100a80 msgr2=0x7fec1c104a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec1ad9d640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c100a80 0x7fec1c104a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec1ad9d640 1 -- 172.21.15.16:0/1157053638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c103e20 con 0x7fec1c102630 2024-03-27T06:43:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec1a59c640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c100a80 0x7fec1c104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec13fff640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec1c101c70 0x7fec1c103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec1ad9d640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec1c102630 0x7fec1c103590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec1402c630 tx=0x7fec140346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.403+0000 7fec137fe640 1 -- 172.21.15.16:0/1157053638 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14039040 con 0x7fec1c102630 2024-03-27T06:43:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.404+0000 7fec2091e640 1 -- 172.21.15.16:0/1157053638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c1a1fd0 con 0x7fec1c102630 2024-03-27T06:43:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.404+0000 7fec137fe640 1 -- 172.21.15.16:0/1157053638 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fec14044070 con 0x7fec1c102630 2024-03-27T06:43:22.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.404+0000 7fec137fe640 1 -- 172.21.15.16:0/1157053638 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14004500 con 0x7fec1c102630 2024-03-27T06:43:22.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.404+0000 7fec2091e640 1 -- 172.21.15.16:0/1157053638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c1a2490 con 0x7fec1c102630 2024-03-27T06:43:22.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.404+0000 7fec2091e640 1 -- 172.21.15.16:0/1157053638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c1a2780 con 0x7fec1c102630 2024-03-27T06:43:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.406+0000 7fec137fe640 1 -- 172.21.15.16:0/1157053638 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fec140046a0 con 0x7fec1c102630 2024-03-27T06:43:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.406+0000 7fec137fe640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fec080a3610 0x7fec080a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.406+0000 7fec1a59c640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fec080a3610 0x7fec080a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.406+0000 7fec137fe640 1 -- 172.21.15.16:0/1157053638 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fec140f2cd0 con 0x7fec1c102630 2024-03-27T06:43:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.407+0000 7fec1a59c640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fec080a3610 0x7fec080a5ad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0095f0 tx=0x7fec0c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:22.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.414+0000 7fec137fe640 1 -- 172.21.15.16:0/1157053638 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fec140bf8b0 con 0x7fec1c102630 2024-03-27T06:43:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:22 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2035293042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:22 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1904875182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:22.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:22 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1989740925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.643+0000 7fec2091e640 1 -- 172.21.15.16:0/1157053638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec1c1a2950 con 0x7fec1c102630 2024-03-27T06:43:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.644+0000 7fec137fe640 1 -- 172.21.15.16:0/1157053638 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7fec140bf1b0 con 0x7fec1c102630 2024-03-27T06:43:22.639 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:22.639 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:41:35.466274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24472,24493,24500,24527,24467]},"id":1}]} 2024-03-27T06:43:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-03-27T06:43:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 -- 172.21.15.16:0/1157053638 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fec080a3610 msgr2=0x7fec080a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fec080a3610 0x7fec080a5ad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0095f0 tx=0x7fec0c005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 -- 172.21.15.16:0/1157053638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec1c102630 msgr2=0x7fec1c103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec1c102630 0x7fec1c103590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec1402c630 tx=0x7fec140346f0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 -- 172.21.15.16:0/1157053638 shutdown_connections 2024-03-27T06:43:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fec080a3610 0x7fec080a5ad0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec1c102630 0x7fec1c103590 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec1c101c70 0x7fec1c103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 --2- 172.21.15.16:0/1157053638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c100a80 0x7fec1c104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 -- 172.21.15.16:0/1157053638 >> 172.21.15.16:0/1157053638 conn(0x7fec1c0fc030 msgr2=0x7fec1c0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 -- 172.21.15.16:0/1157053638 shutdown_connections 2024-03-27T06:43:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.646+0000 7fec2091e640 1 -- 172.21.15.16:0/1157053638 wait complete. 2024-03-27T06:43:22.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:43:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.760+0000 7fc77d3d7640 1 -- 172.21.15.16:0/4259881825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc778102d60 msgr2=0x7fc7781031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.760+0000 7fc77d3d7640 1 --2- 172.21.15.16:0/4259881825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc778102d60 0x7fc7781031e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc760009900 tx=0x7fc76002bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.760+0000 7fc77d3d7640 1 -- 172.21.15.16:0/4259881825 shutdown_connections 2024-03-27T06:43:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.760+0000 7fc77d3d7640 1 --2- 172.21.15.16:0/4259881825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778103720 0x7fc778109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.760+0000 7fc77d3d7640 1 --2- 172.21.15.16:0/4259881825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc778102d60 0x7fc7781031e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.760+0000 7fc77d3d7640 1 --2- 172.21.15.16:0/4259881825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc778101b70 0x7fc778101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.760+0000 7fc77d3d7640 1 -- 172.21.15.16:0/4259881825 >> 172.21.15.16:0/4259881825 conn(0x7fc7780fd100 msgr2=0x7fc7780ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.760+0000 7fc77d3d7640 1 -- 172.21.15.16:0/4259881825 shutdown_connections 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.760+0000 7fc77d3d7640 1 -- 172.21.15.16:0/4259881825 wait complete. 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc77d3d7640 1 Processor -- start 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc77d3d7640 1 -- start start 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc77d3d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778101b70 0x7fc778197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc77d3d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc778102d60 0x7fc778198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc77d3d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc778103720 0x7fc77819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc77d3d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc77810d8d0 con 0x7fc778103720 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc77d3d7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fc77810d750 con 0x7fc778102d60 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc77d3d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc77810da50 con 0x7fc778101b70 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc778103720 0x7fc77819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc778103720 0x7fc77819f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47914/0 (socket says 172.21.15.16:47914) 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc7777fe640 1 -- 172.21.15.16:0/1867172318 learned_addr learned my addr 172.21.15.16:0/1867172318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.761+0000 7fc776ffd640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778101b70 0x7fc778197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc7767fc640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc778102d60 0x7fc778198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc7777fe640 1 -- 172.21.15.16:0/1867172318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778101b70 msgr2=0x7fc778197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc7777fe640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778101b70 0x7fc778197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc7777fe640 1 -- 172.21.15.16:0/1867172318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc778102d60 msgr2=0x7fc778198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc7777fe640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc778102d60 0x7fc778198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc7777fe640 1 -- 172.21.15.16:0/1867172318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77819fb30 con 0x7fc778103720 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc7777fe640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc778103720 0x7fc77819f4c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc77000e970 tx=0x7fc77000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc76ffff640 1 -- 172.21.15.16:0/1867172318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77000ccc0 con 0x7fc778103720 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc77d3d7640 1 -- 172.21.15.16:0/1867172318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77819fe20 con 0x7fc778103720 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc76ffff640 1 -- 172.21.15.16:0/1867172318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc77000ce60 con 0x7fc778103720 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc76ffff640 1 -- 172.21.15.16:0/1867172318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770010690 con 0x7fc778103720 2024-03-27T06:43:22.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.762+0000 7fc77d3d7640 1 -- 172.21.15.16:0/1867172318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7781a02e0 con 0x7fc778103720 2024-03-27T06:43:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.763+0000 7fc77d3d7640 1 -- 172.21.15.16:0/1867172318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc778108e30 con 0x7fc778103720 2024-03-27T06:43:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.764+0000 7fc76ffff640 1 -- 172.21.15.16:0/1867172318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc7700046c0 con 0x7fc778103720 2024-03-27T06:43:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.765+0000 7fc76ffff640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7640a3610 0x7fc7640a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.765+0000 7fc776ffd640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7640a3610 0x7fc7640a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:22.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.765+0000 7fc76ffff640 1 -- 172.21.15.16:0/1867172318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc7700d0100 con 0x7fc778103720 2024-03-27T06:43:22.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.765+0000 7fc776ffd640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7640a3610 0x7fc7640a5ad0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc778102bc0 tx=0x7fc768009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.769+0000 7fc76ffff640 1 -- 172.21.15.16:0/1867172318 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc77009cc80 con 0x7fc778103720 2024-03-27T06:43:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:22 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2035293042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:22 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1904875182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:22 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1989740925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.957+0000 7fc77d3d7640 1 -- 172.21.15.16:0/1867172318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc778062750 con 0x7fc778103720 2024-03-27T06:43:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.958+0000 7fc76ffff640 1 -- 172.21.15.16:0/1867172318 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7fc77009c580 con 0x7fc778103720 2024-03-27T06:43:22.953 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:22.953 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:42:20.647546+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:43:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.960+0000 7fc77d3d7640 1 -- 172.21.15.16:0/1867172318 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7640a3610 msgr2=0x7fc7640a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.960+0000 7fc77d3d7640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7640a3610 0x7fc7640a5ad0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc778102bc0 tx=0x7fc768009500 comp rx=0 tx=0).stop 2024-03-27T06:43:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.960+0000 7fc77d3d7640 1 -- 172.21.15.16:0/1867172318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc778103720 msgr2=0x7fc77819f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.960+0000 7fc77d3d7640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc778103720 0x7fc77819f4c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc77000e970 tx=0x7fc77000ed70 comp rx=0 tx=0).stop 2024-03-27T06:43:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.960+0000 7fc77d3d7640 1 -- 172.21.15.16:0/1867172318 shutdown_connections 2024-03-27T06:43:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.960+0000 7fc77d3d7640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc7640a3610 0x7fc7640a5ad0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.960+0000 7fc77d3d7640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc778103720 0x7fc77819f4c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.960+0000 7fc77d3d7640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc778102d60 0x7fc778198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.960+0000 7fc77d3d7640 1 --2- 172.21.15.16:0/1867172318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778101b70 0x7fc778197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.960+0000 7fc77d3d7640 1 -- 172.21.15.16:0/1867172318 >> 172.21.15.16:0/1867172318 conn(0x7fc7780fd100 msgr2=0x7fc778106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.961+0000 7fc77d3d7640 1 -- 172.21.15.16:0/1867172318 shutdown_connections 2024-03-27T06:43:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:22.961+0000 7fc77d3d7640 1 -- 172.21.15.16:0/1867172318 wait complete. 2024-03-27T06:43:22.967 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T06:41:26.393978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T06:41:27.707469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:43:22.967 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T06:43:22.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:22 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2035293042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:22.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:22 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1904875182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:22.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:22 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1989740925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.077+0000 7f512c16c640 1 -- 172.21.15.16:0/1303810510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5124102620 msgr2=0x7f5124108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.077+0000 7f512c16c640 1 --2- 172.21.15.16:0/1303810510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5124102620 0x7f5124108eb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5118009ca0 tx=0x7f511802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.077+0000 7f512c16c640 1 -- 172.21.15.16:0/1303810510 shutdown_connections 2024-03-27T06:43:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.077+0000 7f512c16c640 1 --2- 172.21.15.16:0/1303810510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5124102620 0x7f5124108eb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.077+0000 7f512c16c640 1 --2- 172.21.15.16:0/1303810510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5124101c60 0x7f51241020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.077+0000 7f512c16c640 1 --2- 172.21.15.16:0/1303810510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5124100a70 0x7f5124100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.077+0000 7f512c16c640 1 -- 172.21.15.16:0/1303810510 >> 172.21.15.16:0/1303810510 conn(0x7f51240fc000 msgr2=0x7f51240fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.077+0000 7f512c16c640 1 -- 172.21.15.16:0/1303810510 shutdown_connections 2024-03-27T06:43:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.077+0000 7f512c16c640 1 -- 172.21.15.16:0/1303810510 wait complete. 2024-03-27T06:43:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f512c16c640 1 Processor -- start 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f512c16c640 1 -- start start 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f512c16c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5124100a70 0x7f512406acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f512c16c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5124101c60 0x7f512419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f512c16c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5124102620 0x7f512419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f512c16c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f512410c820 con 0x7f5124100a70 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f512c16c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f512410c6a0 con 0x7f5124101c60 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f512c16c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f512410c9a0 con 0x7f5124102620 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f5129f08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5124100a70 0x7f512406acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f5129f08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5124100a70 0x7f512406acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47932/0 (socket says 172.21.15.16:47932) 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f5129f08640 1 -- 172.21.15.16:0/2626382435 learned_addr learned my addr 172.21.15.16:0/2626382435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f5129707640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5124101c60 0x7f512419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f512a709640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5124102620 0x7f512419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f5129f08640 1 -- 172.21.15.16:0/2626382435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5124102620 msgr2=0x7f512419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f5129f08640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5124102620 0x7f512419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f5129f08640 1 -- 172.21.15.16:0/2626382435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5124101c60 msgr2=0x7f512419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f5129f08640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5124101c60 0x7f512419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.078+0000 7f5129f08640 1 -- 172.21.15.16:0/2626382435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512419fae0 con 0x7f5124100a70 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.079+0000 7f5129f08640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5124100a70 0x7f512406acb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f511c00e9c0 tx=0x7f511c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.079+0000 7f5116ffd640 1 -- 172.21.15.16:0/2626382435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c00cd80 con 0x7f5124100a70 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.079+0000 7f5116ffd640 1 -- 172.21.15.16:0/2626382435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f511c0035f0 con 0x7f5124100a70 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.079+0000 7f512c16c640 1 -- 172.21.15.16:0/2626382435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512419fdd0 con 0x7f5124100a70 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.079+0000 7f5116ffd640 1 -- 172.21.15.16:0/2626382435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c010690 con 0x7f5124100a70 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.079+0000 7f512c16c640 1 -- 172.21.15.16:0/2626382435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51241a02e0 con 0x7f5124100a70 2024-03-27T06:43:23.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.080+0000 7f512c16c640 1 -- 172.21.15.16:0/2626382435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5124107d30 con 0x7f5124100a70 2024-03-27T06:43:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.081+0000 7f5116ffd640 1 -- 172.21.15.16:0/2626382435 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f511c0037a0 con 0x7f5124100a70 2024-03-27T06:43:23.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.081+0000 7f5116ffd640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51100a3690 0x7f51100a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.082+0000 7f5129707640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51100a3690 0x7f51100a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.082+0000 7f5116ffd640 1 -- 172.21.15.16:0/2626382435 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f511c0d00a0 con 0x7f5124100a70 2024-03-27T06:43:23.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.082+0000 7f5129707640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51100a3690 0x7f51100a5b50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f512406aa20 tx=0x7f5120005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:23.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.086+0000 7f5116ffd640 1 -- 172.21.15.16:0/2626382435 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f511c09cc80 con 0x7f5124100a70 2024-03-27T06:43:23.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.269+0000 7f512c16c640 1 -- 172.21.15.16:0/2626382435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51240627f0 con 0x7f5124100a70 2024-03-27T06:43:23.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.270+0000 7f5116ffd640 1 -- 172.21.15.16:0/2626382435 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f511c09c580 con 0x7f5124100a70 2024-03-27T06:43:23.264 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:43:23.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.271+0000 7f512c16c640 1 -- 172.21.15.16:0/2626382435 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51100a3690 msgr2=0x7f51100a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.271+0000 7f512c16c640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51100a3690 0x7f51100a5b50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f512406aa20 tx=0x7f5120005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:23.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.271+0000 7f512c16c640 1 -- 172.21.15.16:0/2626382435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5124100a70 msgr2=0x7f512406acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.271+0000 7f512c16c640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5124100a70 0x7f512406acb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f511c00e9c0 tx=0x7f511c00ee80 comp rx=0 tx=0).stop 2024-03-27T06:43:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.272+0000 7f512c16c640 1 -- 172.21.15.16:0/2626382435 shutdown_connections 2024-03-27T06:43:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.272+0000 7f512c16c640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f51100a3690 0x7f51100a5b50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.272+0000 7f512c16c640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5124102620 0x7f512419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.272+0000 7f512c16c640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5124101c60 0x7f512419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.272+0000 7f512c16c640 1 --2- 172.21.15.16:0/2626382435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5124100a70 0x7f512406acb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.272+0000 7f512c16c640 1 -- 172.21.15.16:0/2626382435 >> 172.21.15.16:0/2626382435 conn(0x7f51240fc000 msgr2=0x7f5124104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.272+0000 7f512c16c640 1 -- 172.21.15.16:0/2626382435 shutdown_connections 2024-03-27T06:43:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.272+0000 7f512c16c640 1 -- 172.21.15.16:0/2626382435 wait complete. 2024-03-27T06:43:23.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T06:43:23.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.388+0000 7fd01e09a640 1 -- 172.21.15.16:0/3834812830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018109a10 msgr2=0x7fd01810be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.388+0000 7fd01e09a640 1 --2- 172.21.15.16:0/3834812830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018109a10 0x7fd01810be80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd010009900 tx=0x7fd01002bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:23.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.388+0000 7fd01e09a640 1 -- 172.21.15.16:0/3834812830 shutdown_connections 2024-03-27T06:43:23.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.388+0000 7fd01e09a640 1 --2- 172.21.15.16:0/3834812830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018109a10 0x7fd01810be80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.388+0000 7fd01e09a640 1 --2- 172.21.15.16:0/3834812830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd018074570 0x7fd0180749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.388+0000 7fd01e09a640 1 --2- 172.21.15.16:0/3834812830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd018075b50 0x7fd018073fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.388+0000 7fd01e09a640 1 -- 172.21.15.16:0/3834812830 >> 172.21.15.16:0/3834812830 conn(0x7fd0180fd0f0 msgr2=0x7fd0180ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.388+0000 7fd01e09a640 1 -- 172.21.15.16:0/3834812830 shutdown_connections 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.388+0000 7fd01e09a640 1 -- 172.21.15.16:0/3834812830 wait complete. 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.388+0000 7fd01e09a640 1 Processor -- start 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd01e09a640 1 -- start start 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd01e09a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd018074570 0x7fd01819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd01e09a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd018075b50 0x7fd01819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd01e09a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018109a10 0x7fd0181a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd01e09a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810fab0 con 0x7fd018109a10 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd01e09a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810f930 con 0x7fd018074570 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd01e09a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810fc30 con 0x7fd018075b50 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd017fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018109a10 0x7fd0181a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd017fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018109a10 0x7fd0181a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47956/0 (socket says 172.21.15.16:47956) 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd017fff640 1 -- 172.21.15.16:0/4037509844 learned_addr learned my addr 172.21.15.16:0/4037509844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd016ffd640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd018075b50 0x7fd01819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd0177fe640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd018074570 0x7fd01819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd017fff640 1 -- 172.21.15.16:0/4037509844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd018075b50 msgr2=0x7fd01819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd017fff640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd018075b50 0x7fd01819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd017fff640 1 -- 172.21.15.16:0/4037509844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd018074570 msgr2=0x7fd01819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd017fff640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd018074570 0x7fd01819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd017fff640 1 -- 172.21.15.16:0/4037509844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0181a1ce0 con 0x7fd018109a10 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd0177fe640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd018074570 0x7fd01819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd017fff640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018109a10 0x7fd0181a1670 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd010005ab0 tx=0x7fd010004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd014ff9640 1 -- 172.21.15.16:0/4037509844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010043020 con 0x7fd018109a10 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd014ff9640 1 -- 172.21.15.16:0/4037509844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd0100042d0 con 0x7fd018109a10 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd01e09a640 1 -- 172.21.15.16:0/4037509844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0181a1f70 con 0x7fd018109a10 2024-03-27T06:43:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd014ff9640 1 -- 172.21.15.16:0/4037509844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010003c50 con 0x7fd018109a10 2024-03-27T06:43:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.389+0000 7fd01e09a640 1 -- 172.21.15.16:0/4037509844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0181a2480 con 0x7fd018109a10 2024-03-27T06:43:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.390+0000 7fd01e09a640 1 -- 172.21.15.16:0/4037509844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd018108790 con 0x7fd018109a10 2024-03-27T06:43:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.391+0000 7fd014ff9640 1 -- 172.21.15.16:0/4037509844 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fd010046020 con 0x7fd018109a10 2024-03-27T06:43:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.392+0000 7fd014ff9640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd0000a3690 0x7fd0000a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.392+0000 7fd0177fe640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd0000a3690 0x7fd0000a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.392+0000 7fd014ff9640 1 -- 172.21.15.16:0/4037509844 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd0100f1e50 con 0x7fd018109a10 2024-03-27T06:43:23.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.392+0000 7fd0177fe640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd0000a3690 0x7fd0000a5b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd008005fd0 tx=0x7fd008009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:23.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.397+0000 7fd014ff9640 1 -- 172.21.15.16:0/4037509844 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd01004d020 con 0x7fd018109a10 2024-03-27T06:43:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.582+0000 7fd01e09a640 1 -- 172.21.15.16:0/4037509844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd0180627f0 con 0x7fd018109a10 2024-03-27T06:43:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.582+0000 7fd014ff9640 1 -- 172.21.15.16:0/4037509844 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fd0100beac0 con 0x7fd018109a10 2024-03-27T06:43:23.577 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-03-27T06:43:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 -- 172.21.15.16:0/4037509844 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd0000a3690 msgr2=0x7fd0000a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd0000a3690 0x7fd0000a5b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd008005fd0 tx=0x7fd008009500 comp rx=0 tx=0).stop 2024-03-27T06:43:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 -- 172.21.15.16:0/4037509844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018109a10 msgr2=0x7fd0181a1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018109a10 0x7fd0181a1670 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd010005ab0 tx=0x7fd010004ad0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 -- 172.21.15.16:0/4037509844 shutdown_connections 2024-03-27T06:43:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd0000a3690 0x7fd0000a5b50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018109a10 0x7fd0181a1670 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd018075b50 0x7fd01819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 --2- 172.21.15.16:0/4037509844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd018074570 0x7fd01819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 -- 172.21.15.16:0/4037509844 >> 172.21.15.16:0/4037509844 conn(0x7fd0180fd0f0 msgr2=0x7fd018105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 -- 172.21.15.16:0/4037509844 shutdown_connections 2024-03-27T06:43:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.584+0000 7fd01e09a640 1 -- 172.21.15.16:0/4037509844 wait complete. 2024-03-27T06:43:23.589 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:23 smithi016 ceph-mon[31908]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:43:23.589 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:23 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1157053638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:23.589 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:23 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1867172318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:23.589 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:23 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2626382435' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:23.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T06:43:23.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.701+0000 7fe114669640 1 -- 172.21.15.16:0/715273521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c1078b0 msgr2=0x7fe10c109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.701+0000 7fe114669640 1 --2- 172.21.15.16:0/715273521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c1078b0 0x7fe10c109ca0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe104009ca0 tx=0x7fe10402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.701+0000 7fe114669640 1 -- 172.21.15.16:0/715273521 shutdown_connections 2024-03-27T06:43:23.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.701+0000 7fe114669640 1 --2- 172.21.15.16:0/715273521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c1078b0 0x7fe10c109ca0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.701+0000 7fe114669640 1 --2- 172.21.15.16:0/715273521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe10c104f80 0x7fe10c107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.701+0000 7fe114669640 1 --2- 172.21.15.16:0/715273521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10c102650 0x7fe10c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.701+0000 7fe114669640 1 -- 172.21.15.16:0/715273521 >> 172.21.15.16:0/715273521 conn(0x7fe10c0fc000 msgr2=0x7fe10c0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.701+0000 7fe114669640 1 -- 172.21.15.16:0/715273521 shutdown_connections 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.701+0000 7fe114669640 1 -- 172.21.15.16:0/715273521 wait complete. 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe114669640 1 Processor -- start 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe114669640 1 -- start start 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe114669640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe10c102650 0x7fe10c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe114669640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c104f80 0x7fe10c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe114669640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10c1078b0 0x7fe10c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe114669640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10d940 con 0x7fe10c104f80 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe114669640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10d7c0 con 0x7fe10c1078b0 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe114669640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10dac0 con 0x7fe10c102650 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe111c04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c104f80 0x7fe10c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe111c04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c104f80 0x7fe10c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47970/0 (socket says 172.21.15.16:47970) 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe111c04640 1 -- 172.21.15.16:0/3681444873 learned_addr learned my addr 172.21.15.16:0/3681444873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe112405640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe10c102650 0x7fe10c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe112c06640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10c1078b0 0x7fe10c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe111c04640 1 -- 172.21.15.16:0/3681444873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe10c102650 msgr2=0x7fe10c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe111c04640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe10c102650 0x7fe10c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe111c04640 1 -- 172.21.15.16:0/3681444873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10c1078b0 msgr2=0x7fe10c19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe111c04640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10c1078b0 0x7fe10c19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe111c04640 1 -- 172.21.15.16:0/3681444873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c19fb50 con 0x7fe10c104f80 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe111c04640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c104f80 0x7fe10c198470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00e9f0 tx=0x7fe0fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.702+0000 7fe1037fe640 1 -- 172.21.15.16:0/3681444873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc00ccc0 con 0x7fe10c104f80 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.703+0000 7fe1037fe640 1 -- 172.21.15.16:0/3681444873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe0fc005520 con 0x7fe10c104f80 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.703+0000 7fe114669640 1 -- 172.21.15.16:0/3681444873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10c19fe40 con 0x7fe10c104f80 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.703+0000 7fe1037fe640 1 -- 172.21.15.16:0/3681444873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc010690 con 0x7fe10c104f80 2024-03-27T06:43:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.703+0000 7fe114669640 1 -- 172.21.15.16:0/3681444873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10c1a02b0 con 0x7fe10c104f80 2024-03-27T06:43:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.703+0000 7fe114669640 1 -- 172.21.15.16:0/3681444873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10c0fdaf0 con 0x7fe10c104f80 2024-03-27T06:43:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.705+0000 7fe1037fe640 1 -- 172.21.15.16:0/3681444873 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fe0fc004f70 con 0x7fe10c104f80 2024-03-27T06:43:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.705+0000 7fe1037fe640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe0f80a3610 0x7fe0f80a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.705+0000 7fe112405640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe0f80a3610 0x7fe0f80a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.705+0000 7fe1037fe640 1 -- 172.21.15.16:0/3681444873 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe0fc0d01a0 con 0x7fe10c104f80 2024-03-27T06:43:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.706+0000 7fe112405640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe0f80a3610 0x7fe0f80a5ad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe108009bd0 tx=0x7fe108005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:23.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.709+0000 7fe1037fe640 1 -- 172.21.15.16:0/3681444873 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fe0fc053cb0 con 0x7fe10c104f80 2024-03-27T06:43:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:23 smithi170 ceph-mon[36438]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:43:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:23 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1157053638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:23 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1867172318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:23 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2626382435' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.896+0000 7fe114669640 1 -- 172.21.15.16:0/3681444873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe10c02d5c0 con 0x7fe10c104f80 2024-03-27T06:43:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.897+0000 7fe1037fe640 1 -- 172.21.15.16:0/3681444873 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fe0fc09cd80 con 0x7fe10c104f80 2024-03-27T06:43:23.892 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-03-27T06:43:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 -- 172.21.15.16:0/3681444873 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe0f80a3610 msgr2=0x7fe0f80a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe0f80a3610 0x7fe0f80a5ad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe108009bd0 tx=0x7fe108005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 -- 172.21.15.16:0/3681444873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c104f80 msgr2=0x7fe10c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c104f80 0x7fe10c198470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00e9f0 tx=0x7fe0fc00eeb0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 -- 172.21.15.16:0/3681444873 shutdown_connections 2024-03-27T06:43:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fe0f80a3610 0x7fe0f80a5ad0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe10c1078b0 0x7fe10c19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe10c104f80 0x7fe10c198470 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 --2- 172.21.15.16:0/3681444873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe10c102650 0x7fe10c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 -- 172.21.15.16:0/3681444873 >> 172.21.15.16:0/3681444873 conn(0x7fe10c0fc000 msgr2=0x7fe10c102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 -- 172.21.15.16:0/3681444873 shutdown_connections 2024-03-27T06:43:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:23.899+0000 7fe114669640 1 -- 172.21.15.16:0/3681444873 wait complete. 2024-03-27T06:43:23.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:43:23.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:23 smithi156 ceph-mon[36261]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:43:23.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:23 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1157053638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:23.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:23 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1867172318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:23.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:23 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2626382435' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.013+0000 7f47b873e640 1 -- 172.21.15.16:0/441583749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0100af0 msgr2=0x7f47b0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.013+0000 7f47b873e640 1 --2- 172.21.15.16:0/441583749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0100af0 0x7f47b0100f10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009900 tx=0x7f47ac02bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.013+0000 7f47b873e640 1 -- 172.21.15.16:0/441583749 shutdown_connections 2024-03-27T06:43:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.013+0000 7f47b873e640 1 --2- 172.21.15.16:0/441583749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47b01026a0 0x7f47b0108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.013+0000 7f47b873e640 1 --2- 172.21.15.16:0/441583749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47b0101ce0 0x7f47b0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.013+0000 7f47b873e640 1 --2- 172.21.15.16:0/441583749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0100af0 0x7f47b0100f10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.013+0000 7f47b873e640 1 -- 172.21.15.16:0/441583749 >> 172.21.15.16:0/441583749 conn(0x7f47b00fc080 msgr2=0x7f47b00fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.013+0000 7f47b873e640 1 -- 172.21.15.16:0/441583749 shutdown_connections 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.013+0000 7f47b873e640 1 -- 172.21.15.16:0/441583749 wait complete. 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b873e640 1 Processor -- start 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b873e640 1 -- start start 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b873e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47b0100af0 0x7f47b006acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b873e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101ce0 0x7f47b019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b873e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47b01026a0 0x7f47b019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b873e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47b010c7e0 con 0x7f47b0101ce0 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b873e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f47b010c660 con 0x7f47b01026a0 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b873e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f47b010c960 con 0x7f47b0100af0 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b5cd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101ce0 0x7f47b019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b5cd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101ce0 0x7f47b019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47976/0 (socket says 172.21.15.16:47976) 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b5cd9640 1 -- 172.21.15.16:0/1444010349 learned_addr learned my addr 172.21.15.16:0/1444010349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b6cdb640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47b01026a0 0x7f47b019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b64da640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47b0100af0 0x7f47b006acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b5cd9640 1 -- 172.21.15.16:0/1444010349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47b0100af0 msgr2=0x7f47b006acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b5cd9640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47b0100af0 0x7f47b006acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b5cd9640 1 -- 172.21.15.16:0/1444010349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47b01026a0 msgr2=0x7f47b019f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b5cd9640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47b01026a0 0x7f47b019f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b5cd9640 1 -- 172.21.15.16:0/1444010349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b019fc00 con 0x7f47b0101ce0 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.014+0000 7f47b5cd9640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101ce0 0x7f47b019ccd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f47a000e9f0 tx=0x7f47a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.015+0000 7f47a77fe640 1 -- 172.21.15.16:0/1444010349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a000cd80 con 0x7f47b0101ce0 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.015+0000 7f47a77fe640 1 -- 172.21.15.16:0/1444010349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f47a0005520 con 0x7f47b0101ce0 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.015+0000 7f47b873e640 1 -- 172.21.15.16:0/1444010349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b019fe90 con 0x7f47b0101ce0 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.015+0000 7f47a77fe640 1 -- 172.21.15.16:0/1444010349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0010690 con 0x7f47b0101ce0 2024-03-27T06:43:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.015+0000 7f47b873e640 1 -- 172.21.15.16:0/1444010349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b01a0300 con 0x7f47b0101ce0 2024-03-27T06:43:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.016+0000 7f47b873e640 1 -- 172.21.15.16:0/1444010349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b0107e30 con 0x7f47b0101ce0 2024-03-27T06:43:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.017+0000 7f47a77fe640 1 -- 172.21.15.16:0/1444010349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f47a00046c0 con 0x7f47b0101ce0 2024-03-27T06:43:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.018+0000 7f47a77fe640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f479c0a3610 0x7f479c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.018+0000 7f47b64da640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f479c0a3610 0x7f479c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.018+0000 7f47a77fe640 1 -- 172.21.15.16:0/1444010349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f47a00d00d0 con 0x7f47b0101ce0 2024-03-27T06:43:24.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.018+0000 7f47b64da640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f479c0a3610 0x7f479c0a5ad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f47ac000c00 tx=0x7f47ac005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.022+0000 7f47a77fe640 1 -- 172.21.15.16:0/1444010349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f47a009ccb0 con 0x7f47b0101ce0 2024-03-27T06:43:24.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.249+0000 7f47b873e640 1 -- 172.21.15.16:0/1444010349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47b002d370 con 0x7f47b0101ce0 2024-03-27T06:43:24.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.250+0000 7f47a77fe640 1 -- 172.21.15.16:0/1444010349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7f47a009c5b0 con 0x7f47b0101ce0 2024-03-27T06:43:24.245 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:24.245 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:41:35.466274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24472,24493,24500,24527,24467]},"id":1}]} 2024-03-27T06:43:24.245 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-03-27T06:43:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 -- 172.21.15.16:0/1444010349 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f479c0a3610 msgr2=0x7f479c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f479c0a3610 0x7f479c0a5ad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f47ac000c00 tx=0x7f47ac005fd0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 -- 172.21.15.16:0/1444010349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101ce0 msgr2=0x7f47b019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101ce0 0x7f47b019ccd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f47a000e9f0 tx=0x7f47a000edf0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 -- 172.21.15.16:0/1444010349 shutdown_connections 2024-03-27T06:43:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f479c0a3610 0x7f479c0a5ad0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47b01026a0 0x7f47b019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101ce0 0x7f47b019ccd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 --2- 172.21.15.16:0/1444010349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47b0100af0 0x7f47b006acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 -- 172.21.15.16:0/1444010349 >> 172.21.15.16:0/1444010349 conn(0x7f47b00fc080 msgr2=0x7f47b0105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 -- 172.21.15.16:0/1444010349 shutdown_connections 2024-03-27T06:43:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.252+0000 7f47b873e640 1 -- 172.21.15.16:0/1444010349 wait complete. 2024-03-27T06:43:24.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:43:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.366+0000 7f898475e640 1 -- 172.21.15.16:0/3019130621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f897c101b80 msgr2=0x7f897c101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.366+0000 7f898475e640 1 --2- 172.21.15.16:0/3019130621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f897c101b80 0x7f897c101fa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8974009900 tx=0x7f897402bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.366+0000 7f898475e640 1 -- 172.21.15.16:0/3019130621 shutdown_connections 2024-03-27T06:43:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.366+0000 7f898475e640 1 --2- 172.21.15.16:0/3019130621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f897c103730 0x7f897c109fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.366+0000 7f898475e640 1 --2- 172.21.15.16:0/3019130621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f897c102d70 0x7f897c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.366+0000 7f898475e640 1 --2- 172.21.15.16:0/3019130621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f897c101b80 0x7f897c101fa0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.366+0000 7f898475e640 1 -- 172.21.15.16:0/3019130621 >> 172.21.15.16:0/3019130621 conn(0x7f897c0fd130 msgr2=0x7f897c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.366+0000 7f898475e640 1 -- 172.21.15.16:0/3019130621 shutdown_connections 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.366+0000 7f898475e640 1 -- 172.21.15.16:0/3019130621 wait complete. 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f898475e640 1 Processor -- start 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f898475e640 1 -- start start 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f898475e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f897c101b80 0x7f897c075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f898475e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f897c102d70 0x7f897c073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f898475e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f897c103730 0x7f897c074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f898475e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10d8e0 con 0x7f897c102d70 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f898475e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10d760 con 0x7f897c101b80 2024-03-27T06:43:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f898475e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10da60 con 0x7f897c103730 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f8981cf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f897c102d70 0x7f897c073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f8981cf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f897c102d70 0x7f897c073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47996/0 (socket says 172.21.15.16:47996) 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f8981cf9640 1 -- 172.21.15.16:0/2257265683 learned_addr learned my addr 172.21.15.16:0/2257265683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f8982cfb640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f897c103730 0x7f897c074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.367+0000 7f89824fa640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f897c101b80 0x7f897c075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f8981cf9640 1 -- 172.21.15.16:0/2257265683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f897c103730 msgr2=0x7f897c074480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f8981cf9640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f897c103730 0x7f897c074480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f8981cf9640 1 -- 172.21.15.16:0/2257265683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f897c101b80 msgr2=0x7f897c075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f8981cf9640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f897c101b80 0x7f897c075900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f8981cf9640 1 -- 172.21.15.16:0/2257265683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c074d10 con 0x7f897c102d70 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f89824fa640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f897c101b80 0x7f897c075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f8981cf9640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f897c102d70 0x7f897c073f40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f897000e9f0 tx=0x7f897000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f897b7fe640 1 -- 172.21.15.16:0/2257265683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897000cd80 con 0x7f897c102d70 2024-03-27T06:43:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f897b7fe640 1 -- 172.21.15.16:0/2257265683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8970005520 con 0x7f897c102d70 2024-03-27T06:43:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f898475e640 1 -- 172.21.15.16:0/2257265683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c1a20d0 con 0x7f897c102d70 2024-03-27T06:43:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f897b7fe640 1 -- 172.21.15.16:0/2257265683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970010690 con 0x7f897c102d70 2024-03-27T06:43:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.368+0000 7f898475e640 1 -- 172.21.15.16:0/2257265683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c1a25d0 con 0x7f897c102d70 2024-03-27T06:43:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.369+0000 7f898475e640 1 -- 172.21.15.16:0/2257265683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c108ef0 con 0x7f897c102d70 2024-03-27T06:43:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.370+0000 7f897b7fe640 1 -- 172.21.15.16:0/2257265683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f89700046c0 con 0x7f897c102d70 2024-03-27T06:43:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.370+0000 7f897b7fe640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f89640a3610 0x7f89640a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.371+0000 7f89824fa640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f89640a3610 0x7f89640a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.371+0000 7f897b7fe640 1 -- 172.21.15.16:0/2257265683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f89700d0480 con 0x7f897c102d70 2024-03-27T06:43:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.371+0000 7f89824fa640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f89640a3610 0x7f89640a5ad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f897402c900 tx=0x7f897402c690 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.374+0000 7f897b7fe640 1 -- 172.21.15.16:0/2257265683 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8970053cb0 con 0x7f897c102d70 2024-03-27T06:43:24.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.562+0000 7f898475e640 1 -- 172.21.15.16:0/2257265683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f897c02d5c0 con 0x7f897c102d70 2024-03-27T06:43:24.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.563+0000 7f897b7fe640 1 -- 172.21.15.16:0/2257265683 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7f897009d000 con 0x7f897c102d70 2024-03-27T06:43:24.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:24.559 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:42:20.647546+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 -- 172.21.15.16:0/2257265683 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f89640a3610 msgr2=0x7f89640a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f89640a3610 0x7f89640a5ad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f897402c900 tx=0x7f897402c690 comp rx=0 tx=0).stop 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 -- 172.21.15.16:0/2257265683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f897c102d70 msgr2=0x7f897c073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f897c102d70 0x7f897c073f40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f897000e9f0 tx=0x7f897000edf0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 -- 172.21.15.16:0/2257265683 shutdown_connections 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f89640a3610 0x7f89640a5ad0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f897c103730 0x7f897c074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f897c102d70 0x7f897c073f40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 --2- 172.21.15.16:0/2257265683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f897c101b80 0x7f897c075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 -- 172.21.15.16:0/2257265683 >> 172.21.15.16:0/2257265683 conn(0x7f897c0fd130 msgr2=0x7f897c1060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 -- 172.21.15.16:0/2257265683 shutdown_connections 2024-03-27T06:43:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.566+0000 7f898475e640 1 -- 172.21.15.16:0/2257265683 wait complete. 2024-03-27T06:43:24.570 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:24 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/4037509844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:24.570 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:24 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3681444873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:24.571 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:24 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1444010349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:24.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-27T06:41:28.746131+0000', 'modified': '2024-03-27T06:41:35.466274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24500, 'mds_2': 24493, 'mds_3': 24472, 'mds_4': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/420981926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 420981926}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 420981926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/2365316099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 2365316099}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 2365316099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/1368007304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 1368007304}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 1368007304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6833/521338694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6832', 'nonce': 521338694}, {'type': 'v1', 'addr': '172.21.15.156:6833', 'nonce': 521338694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/4279306199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 4279306199}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 4279306199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.156:6835/4256913032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 4256913032}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 4256913032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.170:6837/2408489138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 2408489138}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2408489138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.170:6839/246078270', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 246078270}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 246078270}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.156:6837/3016592835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 3016592835}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 3016592835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6839/479168397', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 479168397}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 479168397}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24472, 24493, 24500, 24527, 24467]} 2024-03-27T06:43:24.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-27T06:43:24.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-27T06:43:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.682+0000 7fec0a35c640 1 -- 172.21.15.16:0/2181348487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec04103710 msgr2=0x7fec04109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.682+0000 7fec0a35c640 1 --2- 172.21.15.16:0/2181348487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec04103710 0x7fec04109fa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7febfc009ca0 tx=0x7febfc02c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.682+0000 7fec0a35c640 1 -- 172.21.15.16:0/2181348487 shutdown_connections 2024-03-27T06:43:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.682+0000 7fec0a35c640 1 --2- 172.21.15.16:0/2181348487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec04103710 0x7fec04109fa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.682+0000 7fec0a35c640 1 --2- 172.21.15.16:0/2181348487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec04102d50 0x7fec041031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.682+0000 7fec0a35c640 1 --2- 172.21.15.16:0/2181348487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec04101b60 0x7fec04101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.682+0000 7fec0a35c640 1 -- 172.21.15.16:0/2181348487 >> 172.21.15.16:0/2181348487 conn(0x7fec040fd0f0 msgr2=0x7fec040ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.682+0000 7fec0a35c640 1 -- 172.21.15.16:0/2181348487 shutdown_connections 2024-03-27T06:43:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.682+0000 7fec0a35c640 1 -- 172.21.15.16:0/2181348487 wait complete. 2024-03-27T06:43:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec0a35c640 1 Processor -- start 2024-03-27T06:43:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec0a35c640 1 -- start start 2024-03-27T06:43:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec0a35c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec04101b60 0x7fec04075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec0a35c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec04102d50 0x7fec04073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec0a35c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec04103710 0x7fec04074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec0a35c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec0410d910 con 0x7fec04101b60 2024-03-27T06:43:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec0a35c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fec0410d790 con 0x7fec04102d50 2024-03-27T06:43:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec0a35c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fec0410da90 con 0x7fec04103710 2024-03-27T06:43:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec04101b60 0x7fec04075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec04101b60 0x7fec04075900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48020/0 (socket says 172.21.15.16:48020) 2024-03-27T06:43:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec03fff640 1 -- 172.21.15.16:0/1703950538 learned_addr learned my addr 172.21.15.16:0/1703950538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec037fe640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec04102d50 0x7fec04073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec088f9640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec04103710 0x7fec04074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec03fff640 1 -- 172.21.15.16:0/1703950538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec04103710 msgr2=0x7fec04074480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec03fff640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec04103710 0x7fec04074480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec03fff640 1 -- 172.21.15.16:0/1703950538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec04102d50 msgr2=0x7fec04073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec03fff640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec04102d50 0x7fec04073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.683+0000 7fec03fff640 1 -- 172.21.15.16:0/1703950538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec04074d10 con 0x7fec04101b60 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.684+0000 7fec03fff640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec04101b60 0x7fec04075900 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7febf400e9c0 tx=0x7febf400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.684+0000 7fec017fa640 1 -- 172.21.15.16:0/1703950538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf400cd80 con 0x7fec04101b60 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.684+0000 7fec017fa640 1 -- 172.21.15.16:0/1703950538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febf40035f0 con 0x7fec04101b60 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.684+0000 7fec0a35c640 1 -- 172.21.15.16:0/1703950538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec041a1fb0 con 0x7fec04101b60 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.684+0000 7fec017fa640 1 -- 172.21.15.16:0/1703950538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4010670 con 0x7fec04101b60 2024-03-27T06:43:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.684+0000 7fec0a35c640 1 -- 172.21.15.16:0/1703950538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041a23c0 con 0x7fec04101b60 2024-03-27T06:43:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec017fa640 1 -- 172.21.15.16:0/1703950538 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7febf40037a0 con 0x7fec04101b60 2024-03-27T06:43:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec0d01a740 1 Processor -- start 2024-03-27T06:43:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec0d01a740 1 -- start start 2024-03-27T06:43:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec0d01a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a8d327010 con 0x564a8d3c1fd0 2024-03-27T06:43:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec0d01a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x564a8d2cf770 con 0x564a8d3c7b30 2024-03-27T06:43:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec0d01a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564a8d322fc0 con 0x564a8d3c29c0 2024-03-27T06:43:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec017fa640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7febec0a3610 0x7febec0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec088f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec088f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3c23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48030/0 (socket says 172.21.15.16:48030) 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec088f9640 1 -- 172.21.15.16:0/1622423645 learned_addr learned my addr 172.21.15.16:0/1622423645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.686+0000 7fec03fff640 1 --2- 172.21.15.16:0/1622423645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7fec037fe640 1 --2- 172.21.15.16:0/1622423645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7fec088f9640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7febec0a3610 0x7febec0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7fec017fa640 1 -- 172.21.15.16:0/1703950538 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7febf40d00d0 con 0x7fec04101b60 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7fec088f9640 1 -- 172.21.15.16:0/1622423645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 msgr2=0x564a8d3c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7fec088f9640 1 --2- 172.21.15.16:0/1622423645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7fec088f9640 1 -- 172.21.15.16:0/1622423645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 msgr2=0x564a8d3c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7fec088f9640 1 --2- 172.21.15.16:0/1622423645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7fec088f9640 1 -- 172.21.15.16:0/1622423645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a8d2ee330 con 0x564a8d3c1fd0 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7fec037fe640 1 --2- 172.21.15.16:0/1622423645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7fec088f9640 1 --2- 172.21.15.16:0/1622423645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3c23f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7febfc0325a0 tx=0x7febfc009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a65f519cde68ad7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7febea7fc640 1 -- 172.21.15.16:0/1622423645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc050070 con 0x564a8d3c1fd0 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7febea7fc640 1 -- 172.21.15.16:0/1622423645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febfc032ab0 con 0x564a8d3c1fd0 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7febea7fc640 1 -- 172.21.15.16:0/1622423645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc039040 con 0x564a8d3c1fd0 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.687+0000 7fec088f9640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7febec0a3610 0x7febec0a5ad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7febfc04d000 tx=0x7febfc048000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.688+0000 7fec0d01a740 1 -- 172.21.15.16:0/1622423645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 msgr2=0x564a8d3c23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.688+0000 7fec0d01a740 1 --2- 172.21.15.16:0/1622423645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3c23f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7febfc0325a0 tx=0x7febfc009a20 comp rx=0 tx=0).stop 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.688+0000 7fec0d01a740 1 -- 172.21.15.16:0/1622423645 shutdown_connections 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.688+0000 7fec0d01a740 1 --2- 172.21.15.16:0/1622423645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.688+0000 7fec0d01a740 1 --2- 172.21.15.16:0/1622423645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.688+0000 7fec0d01a740 1 --2- 172.21.15.16:0/1622423645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3c23f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.688+0000 7fec0d01a740 1 -- 172.21.15.16:0/1622423645 >> 172.21.15.16:0/1622423645 conn(0x564a8d389b20 msgr2=0x564a8d3bd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.688+0000 7fec0d01a740 1 -- 172.21.15.16:0/1622423645 shutdown_connections 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.688+0000 7fec0d01a740 1 -- 172.21.15.16:0/1622423645 wait complete. 2024-03-27T06:43:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.688+0000 7fec0d01a740 1 -- start start 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.688+0000 7fec0d01a740 1 Processor -- start 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c1fd0 0x564a8d3cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c7b30 0x564a8d3d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec0d01a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a8d3caae0 con 0x564a8d3c7b30 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec0d01a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x564a8d2e2b30 con 0x564a8d3c1fd0 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec0d01a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564a8d3cac60 con 0x564a8d3c29c0 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec0d01a740 10 client.0 _ll_register_callbacks cb 0x564a8d31e1a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec088f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c7b30 0x564a8d3d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec088f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c7b30 0x564a8d3d3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48034/0 (socket says 172.21.15.16:48034) 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec088f9640 1 -- 172.21.15.16:0/561480846 learned_addr learned my addr 172.21.15.16:0/561480846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec037fe640 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec03fff640 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c1fd0 0x564a8d3cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec088f9640 1 -- 172.21.15.16:0/561480846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 msgr2=0x564a8d3cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec088f9640 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec088f9640 1 -- 172.21.15.16:0/561480846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c1fd0 msgr2=0x564a8d3cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec088f9640 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c1fd0 0x564a8d3cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec088f9640 1 -- 172.21.15.16:0/561480846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a8d03b2d0 con 0x564a8d3c7b30 2024-03-27T06:43:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec088f9640 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c7b30 0x564a8d3d3a60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7febfc02c700 tx=0x7febfc034e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d47ac5c0402cd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7febe37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7febe37fe640 1 -- 172.21.15.16:0/561480846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc058020 con 0x564a8d3c7b30 2024-03-27T06:43:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7febe37fe640 1 -- 172.21.15.16:0/561480846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7febfc004040 con 0x564a8d3c7b30 2024-03-27T06:43:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.689+0000 7fec0d01a740 1 -- 172.21.15.16:0/561480846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febfc032ab0 con 0x564a8d3c7b30 2024-03-27T06:43:24.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7febe37fe640 1 -- 172.21.15.16:0/561480846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7febfc039070 con 0x564a8d3c7b30 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7fec0d01a740 10 client.15255 fetch_fsmap learned FSMap version 24 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7fec0d01a740 1 -- 172.21.15.16:0/561480846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564a8d1056b0 con 0x564a8d3c7b30 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7febe37fe640 1 -- 172.21.15.16:0/561480846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febfc032ab0 con 0x564a8d3c7b30 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7febe37fe640 1 -- 172.21.15.16:0/561480846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7febfc03f070 con 0x564a8d3c7b30 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7fec0d01a740 10 client.15255 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7fec0d01a740 10 client.15255 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7fec0d01a740 20 client.15255 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7fec0d01a740 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x564a8d3d8ae0 0x564a8d3d9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7fec0d01a740 4 client.15255 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7fec0d01a740 1 -- 172.21.15.16:0/561480846 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a8d334550 con 0x564a8d3d8ae0 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.690+0000 7fec03fff640 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x564a8d3d8ae0 0x564a8d3d9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7febe37fe640 10 client.15255 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7fec03fff640 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x564a8d3d8ae0 0x564a8d3d9f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bea55abeb93ec8f5 server_cookie=c490b1355c84b26f in_seq=0 out_seq=0 2024-03-27T06:43:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7febe37fe640 1 -- 172.21.15.16:0/561480846 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564a8d334550 con 0x564a8d3d8ae0 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7febe37fe640 10 client.15255 handle_command_reply: tid=0 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7fec0d01a740 1 client.15255 shutdown 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7fec0d01a740 1 -- 172.21.15.16:0/561480846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c7b30 msgr2=0x564a8d3d3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7fec0d01a740 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c7b30 0x564a8d3d3a60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7febfc02c700 tx=0x7febfc034e50 comp rx=0 tx=0).stop 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7fec0d01a740 1 -- 172.21.15.16:0/561480846 shutdown_connections 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7fec0d01a740 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x564a8d3d8ae0 0x564a8d3d9f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7fec0d01a740 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c7b30 0x564a8d3d3a60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7fec0d01a740 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7febe37fe640 0 client.15255 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7fec0d01a740 1 --2- 172.21.15.16:0/561480846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c1fd0 0x564a8d3cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.691+0000 7fec0d01a740 1 -- 172.21.15.16:0/561480846 >> 172.21.15.16:0/561480846 conn(0x564a8d389b20 msgr2=0x564a8d3bcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.692+0000 7fec0d01a740 1 -- 172.21.15.16:0/561480846 shutdown_connections 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.692+0000 7fec0d01a740 1 -- 172.21.15.16:0/561480846 wait complete. 2024-03-27T06:43:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.692+0000 7fec0d01a740 20 client.15255 trim_cache size 0 max 16384 2024-03-27T06:43:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.709+0000 7fec0d01a740 1 Processor -- start 2024-03-27T06:43:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.709+0000 7fec0d01a740 1 -- start start 2024-03-27T06:43:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec0d01a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a8d3d8290 con 0x564a8d3c1fd0 2024-03-27T06:43:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec0d01a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x564a8d3d8960 con 0x564a8d3c7b30 2024-03-27T06:43:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec0d01a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564a8d3d9030 con 0x564a8d3c29c0 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48036/0 (socket says 172.21.15.16:48036) 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec03fff640 1 -- 172.21.15.16:0/1555908389 learned_addr learned my addr 172.21.15.16:0/1555908389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec037fe640 1 --2- 172.21.15.16:0/1555908389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec088f9640 1 --2- 172.21.15.16:0/1555908389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec03fff640 1 -- 172.21.15.16:0/1555908389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 msgr2=0x564a8d3d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec03fff640 1 --2- 172.21.15.16:0/1555908389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec03fff640 1 -- 172.21.15.16:0/1555908389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 msgr2=0x564a8d3cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec03fff640 1 --2- 172.21.15.16:0/1555908389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec03fff640 1 -- 172.21.15.16:0/1555908389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a8d3bdee0 con 0x564a8d3c1fd0 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7fec03fff640 1 --2- 172.21.15.16:0/1555908389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3cc0f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7febf4098a10 tx=0x7febf40cdab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec957cbee222ccb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7febe9ffb640 1 -- 172.21.15.16:0/1555908389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf40b0e60 con 0x564a8d3c1fd0 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7febe9ffb640 1 -- 172.21.15.16:0/1555908389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febf40a0960 con 0x564a8d3c1fd0 2024-03-27T06:43:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.710+0000 7febe9ffb640 1 -- 172.21.15.16:0/1555908389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4003cc0 con 0x564a8d3c1fd0 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.711+0000 7fec0d01a740 1 -- 172.21.15.16:0/1555908389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 msgr2=0x564a8d3cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.711+0000 7fec0d01a740 1 --2- 172.21.15.16:0/1555908389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3cc0f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7febf4098a10 tx=0x7febf40cdab0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.711+0000 7fec0d01a740 1 -- 172.21.15.16:0/1555908389 shutdown_connections 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.711+0000 7fec0d01a740 1 --2- 172.21.15.16:0/1555908389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.711+0000 7fec0d01a740 1 --2- 172.21.15.16:0/1555908389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c29c0 0x564a8d3d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.711+0000 7fec0d01a740 1 --2- 172.21.15.16:0/1555908389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c1fd0 0x564a8d3cc0f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.711+0000 7fec0d01a740 1 -- 172.21.15.16:0/1555908389 >> 172.21.15.16:0/1555908389 conn(0x564a8d389b20 msgr2=0x564a8d3c33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.711+0000 7fec0d01a740 1 -- 172.21.15.16:0/1555908389 shutdown_connections 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.711+0000 7fec0d01a740 1 -- 172.21.15.16:0/1555908389 wait complete. 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.711+0000 7fec0d01a740 1 -- start start 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.711+0000 7fec0d01a740 1 Processor -- start 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c1fd0 0x564a8d3d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c29c0 0x564a8d3d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec0d01a740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec0d01a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a8d3d9dd0 con 0x564a8d3c29c0 2024-03-27T06:43:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec0d01a740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x564a8d3d9700 con 0x564a8d3c7b30 2024-03-27T06:43:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec0d01a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564a8d3da4a0 con 0x564a8d3c1fd0 2024-03-27T06:43:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec0d01a740 10 client.0 _ll_register_callbacks cb 0x7febfc04e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c29c0 0x564a8d3d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c29c0 0x564a8d3d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48048/0 (socket says 172.21.15.16:48048) 2024-03-27T06:43:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec037fe640 1 -- 172.21.15.16:0/4031568383 learned_addr learned my addr 172.21.15.16:0/4031568383 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec03fff640 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c1fd0 0x564a8d3d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec088f9640 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec037fe640 1 -- 172.21.15.16:0/4031568383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c1fd0 msgr2=0x564a8d3d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec037fe640 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c1fd0 0x564a8d3d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec037fe640 1 -- 172.21.15.16:0/4031568383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 msgr2=0x564a8d3cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec037fe640 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec037fe640 1 -- 172.21.15.16:0/4031568383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a8d3be690 con 0x564a8d3c29c0 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec037fe640 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c29c0 0x564a8d3d3860 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x564a8d3be890 tx=0x7febf8012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4b97d6b28f6466 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7febe37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7febe37fe640 1 -- 172.21.15.16:0/4031568383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8023060 con 0x564a8d3c29c0 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7febe37fe640 1 -- 172.21.15.16:0/4031568383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7febf8004250 con 0x564a8d3c29c0 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec0d01a740 1 -- 172.21.15.16:0/4031568383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf40a0960 con 0x564a8d3c29c0 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7febe37fe640 1 -- 172.21.15.16:0/4031568383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7febf8004450 con 0x564a8d3c29c0 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec0d01a740 10 client.15261 fetch_fsmap learned FSMap version 24 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.712+0000 7fec0d01a740 1 -- 172.21.15.16:0/4031568383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564a8d43f3d0 con 0x564a8d3c29c0 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.713+0000 7febe37fe640 1 -- 172.21.15.16:0/4031568383 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7febf40a0960 con 0x564a8d3c29c0 2024-03-27T06:43:24.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.713+0000 7febe37fe640 1 -- 172.21.15.16:0/4031568383 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7febf8010430 con 0x564a8d3c29c0 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.713+0000 7fec0d01a740 10 client.15261 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.713+0000 7fec0d01a740 10 client.15261 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.713+0000 7fec0d01a740 20 client.15261 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.713+0000 7fec0d01a740 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x564a8d444020 0x564a8d442d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.713+0000 7fec0d01a740 4 client.15261 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.713+0000 7fec0d01a740 1 -- 172.21.15.16:0/4031568383 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "status"}) -- 0x564a8d32e8a0 con 0x564a8d444020 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.713+0000 7fec03fff640 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x564a8d444020 0x564a8d442d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.713+0000 7febe37fe640 10 client.15261 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.713+0000 7fec03fff640 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x564a8d444020 0x564a8d442d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2da72397398740c6 server_cookie=7cfdacca70c15a7d in_seq=0 out_seq=0 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7febe37fe640 1 -- 172.21.15.16:0/4031568383 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564a8d32e8a0 con 0x564a8d444020 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7febe37fe640 10 client.15261 handle_command_reply: tid=0 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 1 client.15261 shutdown 2024-03-27T06:43:24.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 1 -- 172.21.15.16:0/4031568383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c29c0 msgr2=0x564a8d3d3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c29c0 0x564a8d3d3860 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x564a8d3be890 tx=0x7febf8012bd0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 1 -- 172.21.15.16:0/4031568383 shutdown_connections 2024-03-27T06:43:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x564a8d444020 0x564a8d442d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a8d3c7b30 0x564a8d3cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a8d3c29c0 0x564a8d3d3860 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7febe37fe640 0 client.15261 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:43:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 1 --2- 172.21.15.16:0/4031568383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a8d3c1fd0 0x564a8d3d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 1 -- 172.21.15.16:0/4031568383 >> 172.21.15.16:0/4031568383 conn(0x564a8d389b20 msgr2=0x564a8d3cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 1 -- 172.21.15.16:0/4031568383 shutdown_connections 2024-03-27T06:43:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 1 -- 172.21.15.16:0/4031568383 wait complete. 2024-03-27T06:43:24.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0d01a740 20 client.15261 trim_cache size 0 max 16384 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24467, 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 110.205618935, 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 140.14099367200001 2024-03-27T06:43:24.717 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:43:24.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0a35c640 1 -- 172.21.15.16:0/1703950538 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7febec0a3610 msgr2=0x7febec0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0a35c640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7febec0a3610 0x7febec0a5ad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7febfc04d000 tx=0x7febfc048000 comp rx=0 tx=0).stop 2024-03-27T06:43:24.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0a35c640 1 -- 172.21.15.16:0/1703950538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec04101b60 msgr2=0x7fec04075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.714+0000 7fec0a35c640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec04101b60 0x7fec04075900 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7febf400e9c0 tx=0x7febf400ee80 comp rx=0 tx=0).stop 2024-03-27T06:43:24.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.715+0000 7fec0a35c640 1 -- 172.21.15.16:0/1703950538 shutdown_connections 2024-03-27T06:43:24.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.715+0000 7fec0a35c640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7febec0a3610 0x7febec0a5ad0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.715+0000 7fec0a35c640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec04103710 0x7fec04074480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.715+0000 7fec0a35c640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec04102d50 0x7fec04073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.715+0000 7fec0a35c640 1 --2- 172.21.15.16:0/1703950538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec04101b60 0x7fec04075900 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.715+0000 7fec0a35c640 1 -- 172.21.15.16:0/1703950538 >> 172.21.15.16:0/1703950538 conn(0x7fec040fd0f0 msgr2=0x7fec04105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.715+0000 7fec0a35c640 1 -- 172.21.15.16:0/1703950538 shutdown_connections 2024-03-27T06:43:24.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.715+0000 7fec0a35c640 1 -- 172.21.15.16:0/1703950538 wait complete. 2024-03-27T06:43:24.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-27T06:43:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.829+0000 7fc0ada39640 1 -- 172.21.15.16:0/43293056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a8100da0 msgr2=0x7fc0a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.829+0000 7fc0ada39640 1 --2- 172.21.15.16:0/43293056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a8100da0 0x7fc0a8101200 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc090009900 tx=0x7fc09002bcd0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.829+0000 7fc0ada39640 1 -- 172.21.15.16:0/43293056 shutdown_connections 2024-03-27T06:43:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.829+0000 7fc0ada39640 1 --2- 172.21.15.16:0/43293056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0a8101740 0x7fc0a8101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.829+0000 7fc0ada39640 1 --2- 172.21.15.16:0/43293056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a8100da0 0x7fc0a8101200 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.829+0000 7fc0ada39640 1 --2- 172.21.15.16:0/43293056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a8100440 0x7fc0a8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.829+0000 7fc0ada39640 1 -- 172.21.15.16:0/43293056 >> 172.21.15.16:0/43293056 conn(0x7fc0a80fbfe0 msgr2=0x7fc0a80fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.829+0000 7fc0ada39640 1 -- 172.21.15.16:0/43293056 shutdown_connections 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.829+0000 7fc0ada39640 1 -- 172.21.15.16:0/43293056 wait complete. 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0ada39640 1 Processor -- start 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0ada39640 1 -- start start 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0ada39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a8100440 0x7fc0a819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0ada39640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0a8100da0 0x7fc0a819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0ada39640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a8101740 0x7fc0a81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0ada39640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810faf0 con 0x7fc0a8100440 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0ada39640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810f970 con 0x7fc0a8100da0 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0ada39640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810fc70 con 0x7fc0a8101740 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a8100440 0x7fc0a819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a8100440 0x7fc0a819a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48064/0 (socket says 172.21.15.16:48064) 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0a6ffd640 1 -- 172.21.15.16:0/2182867690 learned_addr learned my addr 172.21.15.16:0/2182867690 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc09ffff640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0a8100da0 0x7fc0a819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0a77fe640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a8101740 0x7fc0a81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0a6ffd640 1 -- 172.21.15.16:0/2182867690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a8101740 msgr2=0x7fc0a81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a8101740 0x7fc0a81a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0a6ffd640 1 -- 172.21.15.16:0/2182867690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0a8100da0 msgr2=0x7fc0a819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0a8100da0 0x7fc0a819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.830+0000 7fc0a6ffd640 1 -- 172.21.15.16:0/2182867690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a81a1d80 con 0x7fc0a8100440 2024-03-27T06:43:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.831+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a8100440 0x7fc0a819a160 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc09800ea60 tx=0x7fc09800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.831+0000 7fc0aca37640 1 -- 172.21.15.16:0/2182867690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09800cde0 con 0x7fc0a8100440 2024-03-27T06:43:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.831+0000 7fc0aca37640 1 -- 172.21.15.16:0/2182867690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc0980035f0 con 0x7fc0a8100440 2024-03-27T06:43:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.831+0000 7fc0ada39640 1 -- 172.21.15.16:0/2182867690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a81a2070 con 0x7fc0a8100440 2024-03-27T06:43:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.831+0000 7fc0aca37640 1 -- 172.21.15.16:0/2182867690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098010720 con 0x7fc0a8100440 2024-03-27T06:43:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.831+0000 7fc0ada39640 1 -- 172.21.15.16:0/2182867690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a81a2590 con 0x7fc0a8100440 2024-03-27T06:43:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.833+0000 7fc0aca37640 1 -- 172.21.15.16:0/2182867690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fc0980037a0 con 0x7fc0a8100440 2024-03-27T06:43:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.833+0000 7fc0b061d740 1 Processor -- start 2024-03-27T06:43:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.833+0000 7fc0b061d740 1 -- start start 2024-03-27T06:43:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.833+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb1683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb16d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb16ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0b061d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561abb0cd010 con 0x561abb1689c0 2024-03-27T06:43:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0b061d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x561abb075770 con 0x561abb167fd0 2024-03-27T06:43:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0b061d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561abb0c8fc0 con 0x561abb16db30 2024-03-27T06:43:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb16d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0a77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb1683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0a77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb1683f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:38970/0 (socket says 172.21.15.16:38970) 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0a77fe640 1 -- 172.21.15.16:0/788398384 learned_addr learned my addr 172.21.15.16:0/788398384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb16d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48076/0 (socket says 172.21.15.16:48076) 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0aca37640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0940a3610 0x7fc0940a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc09ffff640 1 --2- 172.21.15.16:0/788398384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb16ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0a77fe640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0940a3610 0x7fc0940a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc09ffff640 1 -- 172.21.15.16:0/788398384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 msgr2=0x561abb1683f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc09ffff640 1 --2- 172.21.15.16:0/788398384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb1683f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0aca37640 1 -- 172.21.15.16:0/2182867690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc0980d0060 con 0x7fc0a8100440 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc09ffff640 1 -- 172.21.15.16:0/788398384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 msgr2=0x561abb16d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc09ffff640 1 --2- 172.21.15.16:0/788398384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb16d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc09ffff640 1 -- 172.21.15.16:0/788398384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561abb094330 con 0x561abb16db30 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/788398384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb16d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:24.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0a77fe640 1 --2- 172.21.15.16:0/788398384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb1683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc09ffff640 1 --2- 172.21.15.16:0/788398384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb16ffc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc090009230 tx=0x7fc090009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfca5cdafe05ba39 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc0a77fe640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0940a3610 0x7fc0940a5ad0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x561abb169220 tx=0x7fc0a0004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc09dffb640 1 -- 172.21.15.16:0/788398384 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09002c670 con 0x561abb16db30 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc09dffb640 1 -- 172.21.15.16:0/788398384 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc09002cb70 con 0x561abb16db30 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.834+0000 7fc09dffb640 1 -- 172.21.15.16:0/788398384 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900046c0 con 0x561abb16db30 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.835+0000 7fc0b061d740 1 -- 172.21.15.16:0/788398384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 msgr2=0x561abb16ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.835+0000 7fc0b061d740 1 --2- 172.21.15.16:0/788398384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb16ffc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc090009230 tx=0x7fc090009670 comp rx=0 tx=0).stop 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.835+0000 7fc0b061d740 1 -- 172.21.15.16:0/788398384 shutdown_connections 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.835+0000 7fc0b061d740 1 --2- 172.21.15.16:0/788398384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb16ffc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.835+0000 7fc0b061d740 1 --2- 172.21.15.16:0/788398384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb16d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.835+0000 7fc0b061d740 1 --2- 172.21.15.16:0/788398384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb1683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.835+0000 7fc0b061d740 1 -- 172.21.15.16:0/788398384 >> 172.21.15.16:0/788398384 conn(0x561abb12fb20 msgr2=0x561abb163940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.835+0000 7fc0b061d740 1 -- 172.21.15.16:0/788398384 shutdown_connections 2024-03-27T06:43:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.835+0000 7fc0b061d740 1 -- 172.21.15.16:0/788398384 wait complete. 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.835+0000 7fc0b061d740 1 -- start start 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0b061d740 1 Processor -- start 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb172480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb1689c0 0x561abb1729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb16db30 0x561abb179990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0b061d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561abb170a80 con 0x561abb16db30 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0b061d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x561abb088b30 con 0x561abb167fd0 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0b061d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561abb170c00 con 0x561abb1689c0 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0b061d740 10 client.0 _ll_register_callbacks cb 0x561abb0c41a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb16db30 0x561abb179990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc09ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb1689c0 0x561abb1729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb16db30 0x561abb179990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48078/0 (socket says 172.21.15.16:48078) 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0a77fe640 1 -- 172.21.15.16:0/868249775 learned_addr learned my addr 172.21.15.16:0/868249775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0a6ffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb172480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0a77fe640 1 -- 172.21.15.16:0/868249775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb1689c0 msgr2=0x561abb1729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0a77fe640 1 --2- 172.21.15.16:0/868249775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb1689c0 0x561abb1729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0a77fe640 1 -- 172.21.15.16:0/868249775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 msgr2=0x561abb172480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0a77fe640 1 --2- 172.21.15.16:0/868249775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb172480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0a77fe640 1 -- 172.21.15.16:0/868249775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561abade12d0 con 0x561abb16db30 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc0a77fe640 1 --2- 172.21.15.16:0/868249775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb16db30 0x561abb179990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00133f0 tx=0x7fc0a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c828fb54d4932ed server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.836+0000 7fc086ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc086ffd640 1 -- 172.21.15.16:0/868249775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0026070 con 0x561abb16db30 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc086ffd640 1 -- 172.21.15.16:0/868249775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc0a0010d40 con 0x561abb16db30 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc0b061d740 1 -- 172.21.15.16:0/868249775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc09002cb70 con 0x561abb16db30 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc086ffd640 1 -- 172.21.15.16:0/868249775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc0a0031050 con 0x561abb16db30 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc0b061d740 10 client.15273 fetch_fsmap learned FSMap version 24 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc0b061d740 1 -- 172.21.15.16:0/868249775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561abaeab6b0 con 0x561abb16db30 2024-03-27T06:43:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc086ffd640 1 -- 172.21.15.16:0/868249775 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc09002cb70 con 0x561abb16db30 2024-03-27T06:43:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc086ffd640 1 -- 172.21.15.16:0/868249775 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fc0a001f050 con 0x561abb16db30 2024-03-27T06:43:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc0b061d740 10 client.15273 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:43:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc0b061d740 10 client.15273 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-27T06:43:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc0b061d740 20 client.15273 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc0b061d740 1 --2- 172.21.15.16:0/868249775 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561abb17eee0 0x561abb180350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc0b061d740 4 client.15273 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:43:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc0b061d740 1 -- 172.21.15.16:0/868249775 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561abb0da550 con 0x561abb17eee0 2024-03-27T06:43:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.837+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/868249775 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561abb17eee0 0x561abb180350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.838+0000 7fc086ffd640 10 client.15273 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:43:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.838+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/868249775 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561abb17eee0 0x561abb180350 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a52b3bbbde12ed4f server_cookie=138367cee21b4481 in_seq=0 out_seq=0 2024-03-27T06:43:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.838+0000 7fc086ffd640 1 -- 172.21.15.16:0/868249775 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561abb0da550 con 0x561abb17eee0 2024-03-27T06:43:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.838+0000 7fc086ffd640 10 client.15273 handle_command_reply: tid=0 2024-03-27T06:43:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 1 client.15273 shutdown 2024-03-27T06:43:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 1 -- 172.21.15.16:0/868249775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb16db30 msgr2=0x561abb179990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 1 --2- 172.21.15.16:0/868249775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb16db30 0x561abb179990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00133f0 tx=0x7fc0a0004a40 comp rx=0 tx=0).stop 2024-03-27T06:43:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 1 -- 172.21.15.16:0/868249775 shutdown_connections 2024-03-27T06:43:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 1 --2- 172.21.15.16:0/868249775 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561abb17eee0 0x561abb180350 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 1 --2- 172.21.15.16:0/868249775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb16db30 0x561abb179990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 1 --2- 172.21.15.16:0/868249775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb1689c0 0x561abb1729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc086ffd640 0 client.15273 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:43:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 1 --2- 172.21.15.16:0/868249775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb172480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 1 -- 172.21.15.16:0/868249775 >> 172.21.15.16:0/868249775 conn(0x561abb12fb20 msgr2=0x561abb162b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 1 -- 172.21.15.16:0/868249775 shutdown_connections 2024-03-27T06:43:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 1 -- 172.21.15.16:0/868249775 wait complete. 2024-03-27T06:43:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.839+0000 7fc0b061d740 20 client.15273 trim_cache size 0 max 16384 2024-03-27T06:43:24.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:24 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/4037509844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:24.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:24 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3681444873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:24.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:24 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1444010349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.857+0000 7fc0b061d740 1 Processor -- start 2024-03-27T06:43:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.857+0000 7fc0b061d740 1 -- start start 2024-03-27T06:43:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.857+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb167fd0 0x561abb188630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.857+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb1689c0 0x561abb179950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.857+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb169ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.857+0000 7fc0b061d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561abb17e690 con 0x561abb167fd0 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.857+0000 7fc0b061d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x561abb17ed60 con 0x561abb1689c0 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.857+0000 7fc0b061d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561abb17f430 con 0x561abb16db30 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.857+0000 7fc0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb167fd0 0x561abb188630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb167fd0 0x561abb188630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48084/0 (socket says 172.21.15.16:48084) 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc0a6ffd640 1 -- 172.21.15.16:0/3164165685 learned_addr learned my addr 172.21.15.16:0/3164165685 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc09ffff640 1 --2- 172.21.15.16:0/3164165685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb1689c0 0x561abb179950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc0a77fe640 1 --2- 172.21.15.16:0/3164165685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb169ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc0a6ffd640 1 -- 172.21.15.16:0/3164165685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 msgr2=0x561abb169ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/3164165685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb169ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc0a6ffd640 1 -- 172.21.15.16:0/3164165685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb1689c0 msgr2=0x561abb179950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/3164165685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb1689c0 0x561abb179950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc0a6ffd640 1 -- 172.21.15.16:0/3164165685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561abb168da0 con 0x561abb167fd0 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/3164165685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb167fd0 0x561abb188630 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc098098f10 tx=0x7fc0980986c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74f0dfdb10672c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc09d7fa640 1 -- 172.21.15.16:0/3164165685 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09809c080 con 0x561abb167fd0 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc09d7fa640 1 -- 172.21.15.16:0/3164165685 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc098003bc0 con 0x561abb167fd0 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.858+0000 7fc09d7fa640 1 -- 172.21.15.16:0/3164165685 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09809bea0 con 0x561abb167fd0 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 -- 172.21.15.16:0/3164165685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb167fd0 msgr2=0x561abb188630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 --2- 172.21.15.16:0/3164165685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb167fd0 0x561abb188630 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc098098f10 tx=0x7fc0980986c0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 -- 172.21.15.16:0/3164165685 shutdown_connections 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 --2- 172.21.15.16:0/3164165685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb169ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 --2- 172.21.15.16:0/3164165685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb1689c0 0x561abb179950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 --2- 172.21.15.16:0/3164165685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb167fd0 0x561abb188630 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 -- 172.21.15.16:0/3164165685 >> 172.21.15.16:0/3164165685 conn(0x561abb12fb20 msgr2=0x561abb172480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 -- 172.21.15.16:0/3164165685 shutdown_connections 2024-03-27T06:43:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 -- 172.21.15.16:0/3164165685 wait complete. 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 -- start start 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 Processor -- start 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb1752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb175820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb178840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561abb1801d0 con 0x561abb1689c0 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x561abb17fb00 con 0x561abb167fd0 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.859+0000 7fc0b061d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561abb1808a0 con 0x561abb16db30 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc0b061d740 10 client.0 _ll_register_callbacks cb 0x7fc0a0003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc09ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb175820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc09ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb175820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48096/0 (socket says 172.21.15.16:48096) 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc09ffff640 1 -- 172.21.15.16:0/3920508745 learned_addr learned my addr 172.21.15.16:0/3920508745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb1752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc0a77fe640 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb178840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc09ffff640 1 -- 172.21.15.16:0/3920508745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 msgr2=0x561abb178840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc09ffff640 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb178840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc09ffff640 1 -- 172.21.15.16:0/3920508745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 msgr2=0x561abb1752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc09ffff640 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb1752e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc09ffff640 1 -- 172.21.15.16:0/3920508745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561abb178e30 con 0x561abb1689c0 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc09ffff640 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb175820 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc090009230 tx=0x7fc0900058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab65e853315a5dfe server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc086ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc086ffd640 1 -- 172.21.15.16:0/3920508745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090004310 con 0x561abb1689c0 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc086ffd640 1 -- 172.21.15.16:0/3920508745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc090005030 con 0x561abb1689c0 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc0b061d740 1 -- 172.21.15.16:0/3920508745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561abb0d48a0 con 0x561abb1689c0 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc086ffd640 1 -- 172.21.15.16:0/3920508745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc0900044b0 con 0x561abb1689c0 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc0b061d740 10 client.15279 fetch_fsmap learned FSMap version 24 2024-03-27T06:43:24.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.860+0000 7fc0b061d740 1 -- 172.21.15.16:0/3920508745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561abb1e4ef0 con 0x561abb1689c0 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.861+0000 7fc086ffd640 1 -- 172.21.15.16:0/3920508745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x561abb0d48a0 con 0x561abb1689c0 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.861+0000 7fc086ffd640 1 -- 172.21.15.16:0/3920508745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fc0900035f0 con 0x561abb1689c0 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.861+0000 7fc0b061d740 10 client.15279 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.861+0000 7fc0b061d740 10 client.15279 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.861+0000 7fc0b061d740 20 client.15279 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.861+0000 7fc0b061d740 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561abb1e9700 0x561abb1e8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.861+0000 7fc0b061d740 4 client.15279 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.861+0000 7fc0b061d740 1 -- 172.21.15.16:0/3920508745 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "status"}) -- 0x561abb052a00 con 0x561abb1e9700 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.861+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561abb1e9700 0x561abb1e8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.861+0000 7fc0a6ffd640 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561abb1e9700 0x561abb1e8410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ee8531fd1b1c89b server_cookie=a83576f1fb54f1af in_seq=0 out_seq=0 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.861+0000 7fc086ffd640 10 client.15279 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc086ffd640 1 -- 172.21.15.16:0/3920508745 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561abb052a00 con 0x561abb1e9700 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc086ffd640 10 client.15279 handle_command_reply: tid=0 2024-03-27T06:43:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 1 client.15279 shutdown 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 1 -- 172.21.15.16:0/3920508745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 msgr2=0x561abb175820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb175820 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc090009230 tx=0x7fc0900058e0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 1 -- 172.21.15.16:0/3920508745 shutdown_connections 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561abb1e9700 0x561abb1e8410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561abb16db30 0x561abb178840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561abb1689c0 0x561abb175820 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc086ffd640 0 client.15279 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 1 --2- 172.21.15.16:0/3920508745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561abb167fd0 0x561abb1752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 1 -- 172.21.15.16:0/3920508745 >> 172.21.15.16:0/3920508745 conn(0x561abb12fb20 msgr2=0x561abb172480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 1 -- 172.21.15.16:0/3920508745 shutdown_connections 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 1 -- 172.21.15.16:0/3920508745 wait complete. 2024-03-27T06:43:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0b061d740 20 client.15279 trim_cache size 0 max 16384 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24472, 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 111.369511843, 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 134.78958454599999 2024-03-27T06:43:24.864 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:43:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 -- 172.21.15.16:0/2182867690 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0940a3610 msgr2=0x7fc0940a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0940a3610 0x7fc0940a5ad0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x561abb169220 tx=0x7fc0a0004ab0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 -- 172.21.15.16:0/2182867690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a8100440 msgr2=0x7fc0a819a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a8100440 0x7fc0a819a160 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc09800ea60 tx=0x7fc09800ef20 comp rx=0 tx=0).stop 2024-03-27T06:43:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 -- 172.21.15.16:0/2182867690 shutdown_connections 2024-03-27T06:43:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fc0940a3610 0x7fc0940a5ad0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a8101740 0x7fc0a81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0a8100da0 0x7fc0a819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 --2- 172.21.15.16:0/2182867690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a8100440 0x7fc0a819a160 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 -- 172.21.15.16:0/2182867690 >> 172.21.15.16:0/2182867690 conn(0x7fc0a80fbfe0 msgr2=0x7fc0a80fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 -- 172.21.15.16:0/2182867690 shutdown_connections 2024-03-27T06:43:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.862+0000 7fc0ada39640 1 -- 172.21.15.16:0/2182867690 wait complete. 2024-03-27T06:43:24.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-27T06:43:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.977+0000 7fca4a676640 1 -- 172.21.15.16:0/602306813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44101b60 msgr2=0x7fca44101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.977+0000 7fca4a676640 1 --2- 172.21.15.16:0/602306813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44101b60 0x7fca44101f80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fca34009900 tx=0x7fca3402bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.977+0000 7fca4a676640 1 -- 172.21.15.16:0/602306813 shutdown_connections 2024-03-27T06:43:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.977+0000 7fca4a676640 1 --2- 172.21.15.16:0/602306813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca44103710 0x7fca44109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.977+0000 7fca4a676640 1 --2- 172.21.15.16:0/602306813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca44102d50 0x7fca441031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.977+0000 7fca4a676640 1 --2- 172.21.15.16:0/602306813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44101b60 0x7fca44101f80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.977+0000 7fca4a676640 1 -- 172.21.15.16:0/602306813 >> 172.21.15.16:0/602306813 conn(0x7fca440fd0f0 msgr2=0x7fca440ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.977+0000 7fca4a676640 1 -- 172.21.15.16:0/602306813 shutdown_connections 2024-03-27T06:43:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.977+0000 7fca4a676640 1 -- 172.21.15.16:0/602306813 wait complete. 2024-03-27T06:43:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.977+0000 7fca4a676640 1 Processor -- start 2024-03-27T06:43:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.977+0000 7fca4a676640 1 -- start start 2024-03-27T06:43:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca4a676640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca44101b60 0x7fca4419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca4a676640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca44102d50 0x7fca4419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca4a676640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44103710 0x7fca441a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca4a676640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca4410d8c0 con 0x7fca44103710 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca4a676640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fca4410d740 con 0x7fca44102d50 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca4a676640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fca4410da40 con 0x7fca44101b60 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca48c13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44103710 0x7fca441a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca43fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca44101b60 0x7fca4419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca48c13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44103710 0x7fca441a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48112/0 (socket says 172.21.15.16:48112) 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca48c13640 1 -- 172.21.15.16:0/3595444777 learned_addr learned my addr 172.21.15.16:0/3595444777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca437fe640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca44102d50 0x7fca4419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca48c13640 1 -- 172.21.15.16:0/3595444777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca44101b60 msgr2=0x7fca4419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca48c13640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca44101b60 0x7fca4419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca48c13640 1 -- 172.21.15.16:0/3595444777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca44102d50 msgr2=0x7fca4419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca48c13640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca44102d50 0x7fca4419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca48c13640 1 -- 172.21.15.16:0/3595444777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca441a1df0 con 0x7fca44103710 2024-03-27T06:43:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca48c13640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44103710 0x7fca441a1780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00eb10 tx=0x7fca3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.978+0000 7fca417fa640 1 -- 172.21.15.16:0/3595444777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c004b20 con 0x7fca44103710 2024-03-27T06:43:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.979+0000 7fca417fa640 1 -- 172.21.15.16:0/3595444777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fca3c004cc0 con 0x7fca44103710 2024-03-27T06:43:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.979+0000 7fca4a676640 1 -- 172.21.15.16:0/3595444777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca441a20e0 con 0x7fca44103710 2024-03-27T06:43:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.979+0000 7fca417fa640 1 -- 172.21.15.16:0/3595444777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0107f0 con 0x7fca44103710 2024-03-27T06:43:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.979+0000 7fca4a676640 1 -- 172.21.15.16:0/3595444777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca441a25e0 con 0x7fca44103710 2024-03-27T06:43:24.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:24 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/4037509844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:24.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:24 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3681444873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:24.977 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:24 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1444010349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca417fa640 1 -- 172.21.15.16:0/3595444777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fca3c005020 con 0x7fca44103710 2024-03-27T06:43:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca4d264740 1 Processor -- start 2024-03-27T06:43:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca4d264740 1 -- start start 2024-03-27T06:43:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca4d264740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea0c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca4d264740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea0c9c0 0x5619eea114c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca4d264740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea11b30 0x5619eea13fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca4d264740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5619ee971010 con 0x5619eea11b30 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca4d264740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5619ee919770 con 0x5619eea0bfd0 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca4d264740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5619ee96cfc0 con 0x5619eea0c9c0 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea11b30 0x5619eea13fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea11b30 0x5619eea13fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48128/0 (socket says 172.21.15.16:48128) 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca437fe640 1 -- 172.21.15.16:0/4199805103 learned_addr learned my addr 172.21.15.16:0/4199805103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca43fff640 1 --2- 172.21.15.16:0/4199805103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea0c9c0 0x5619eea114c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca437fe640 1 --2- 172.21.15.16:0/4199805103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea0c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.981+0000 7fca417fa640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fca300a3610 0x7fca300a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca48c13640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fca300a3610 0x7fca300a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca437fe640 1 -- 172.21.15.16:0/4199805103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea0c9c0 msgr2=0x5619eea114c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca437fe640 1 --2- 172.21.15.16:0/4199805103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea0c9c0 0x5619eea114c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca437fe640 1 -- 172.21.15.16:0/4199805103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 msgr2=0x5619eea0c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca437fe640 1 --2- 172.21.15.16:0/4199805103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea0c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca437fe640 1 -- 172.21.15.16:0/4199805103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ee938330 con 0x5619eea11b30 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca417fa640 1 -- 172.21.15.16:0/3595444777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fca3c0d0280 con 0x7fca44103710 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca437fe640 1 --2- 172.21.15.16:0/4199805103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea11b30 0x5619eea13fc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0117e0 tx=0x7fca2c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcfe4281fd91dda server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca3a7fc640 1 -- 172.21.15.16:0/4199805103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c004030 con 0x5619eea11b30 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca48c13640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fca300a3610 0x7fca300a5ad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fca3c09c6f0 tx=0x7fca3c087040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca3a7fc640 1 -- 172.21.15.16:0/4199805103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fca2c00fcc0 con 0x5619eea11b30 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.982+0000 7fca3a7fc640 1 -- 172.21.15.16:0/4199805103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c003a50 con 0x5619eea11b30 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 -- 172.21.15.16:0/4199805103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea11b30 msgr2=0x5619eea13fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 --2- 172.21.15.16:0/4199805103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea11b30 0x5619eea13fc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0117e0 tx=0x7fca2c0097d0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 -- 172.21.15.16:0/4199805103 shutdown_connections 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 --2- 172.21.15.16:0/4199805103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea11b30 0x5619eea13fc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 --2- 172.21.15.16:0/4199805103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea0c9c0 0x5619eea114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 --2- 172.21.15.16:0/4199805103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea0c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 -- 172.21.15.16:0/4199805103 >> 172.21.15.16:0/4199805103 conn(0x5619ee9d3b20 msgr2=0x5619eea07940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 -- 172.21.15.16:0/4199805103 shutdown_connections 2024-03-27T06:43:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 -- 172.21.15.16:0/4199805103 wait complete. 2024-03-27T06:43:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 -- start start 2024-03-27T06:43:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 Processor -- start 2024-03-27T06:43:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea16450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea16990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5619eea14ae0 con 0x5619eea0c9c0 2024-03-27T06:43:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5619ee92cb30 con 0x5619eea0bfd0 2024-03-27T06:43:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5619eea14c60 con 0x5619eea11b30 2024-03-27T06:43:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.983+0000 7fca4d264740 10 client.0 _ll_register_callbacks cb 0x5619ee9681a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea16990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea16990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48136/0 (socket says 172.21.15.16:48136) 2024-03-27T06:43:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca43fff640 1 -- 172.21.15.16:0/3740580968 learned_addr learned my addr 172.21.15.16:0/3740580968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca437fe640 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea16450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca437fe640 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca43fff640 1 -- 172.21.15.16:0/3740580968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 msgr2=0x5619eea1d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca43fff640 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca43fff640 1 -- 172.21.15.16:0/3740580968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 msgr2=0x5619eea16450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca43fff640 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea16450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca43fff640 1 -- 172.21.15.16:0/3740580968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ee6852d0 con 0x5619eea0c9c0 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca437fe640 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca43fff640 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea16990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fca34009900 tx=0x7fca3402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ae7ad3f74cb572 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca237fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca237fe640 1 -- 172.21.15.16:0/3740580968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34004250 con 0x5619eea0c9c0 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca237fe640 1 -- 172.21.15.16:0/3740580968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fca340043f0 con 0x5619eea0c9c0 2024-03-27T06:43:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca4d264740 1 -- 172.21.15.16:0/3740580968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2c00fcc0 con 0x5619eea0c9c0 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca237fe640 1 -- 172.21.15.16:0/3740580968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fca34005030 con 0x5619eea0c9c0 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca4d264740 10 client.15291 fetch_fsmap learned FSMap version 24 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca4d264740 1 -- 172.21.15.16:0/3740580968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5619ee74f6b0 con 0x5619eea0c9c0 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.984+0000 7fca237fe640 1 -- 172.21.15.16:0/3740580968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fca2c00fcc0 con 0x5619eea0c9c0 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.985+0000 7fca237fe640 1 -- 172.21.15.16:0/3740580968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fca34003680 con 0x5619eea0c9c0 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.985+0000 7fca4d264740 10 client.15291 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.985+0000 7fca4d264740 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.985+0000 7fca4d264740 20 client.15291 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.985+0000 7fca4d264740 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5619eea22f20 0x5619eea24390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.985+0000 7fca4d264740 4 client.15291 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.985+0000 7fca4d264740 1 -- 172.21.15.16:0/3740580968 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619ee97e550 con 0x5619eea22f20 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.985+0000 7fca437fe640 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5619eea22f20 0x5619eea24390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.985+0000 7fca437fe640 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5619eea22f20 0x5619eea24390 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31b317fc030b991f server_cookie=f62390fdd97db209 in_seq=0 out_seq=0 2024-03-27T06:43:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.985+0000 7fca237fe640 10 client.15291 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca237fe640 1 -- 172.21.15.16:0/3740580968 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5619ee97e550 con 0x5619eea22f20 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca237fe640 10 client.15291 handle_command_reply: tid=0 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 1 client.15291 shutdown 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 1 -- 172.21.15.16:0/3740580968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 msgr2=0x5619eea16990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea16990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fca34009900 tx=0x7fca3402c800 comp rx=0 tx=0).stop 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 1 -- 172.21.15.16:0/3740580968 shutdown_connections 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5619eea22f20 0x5619eea24390 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea16990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 1 --2- 172.21.15.16:0/3740580968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea16450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca237fe640 0 client.15291 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 1 -- 172.21.15.16:0/3740580968 >> 172.21.15.16:0/3740580968 conn(0x5619ee9d3b20 msgr2=0x5619eea06b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 1 -- 172.21.15.16:0/3740580968 shutdown_connections 2024-03-27T06:43:24.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 1 -- 172.21.15.16:0/3740580968 wait complete. 2024-03-27T06:43:24.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:24.986+0000 7fca4d264740 20 client.15291 trim_cache size 0 max 16384 2024-03-27T06:43:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.004+0000 7fca4d264740 1 Processor -- start 2024-03-27T06:43:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.004+0000 7fca4d264740 1 -- start start 2024-03-27T06:43:25.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.004+0000 7fca4d264740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea0de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.004+0000 7fca4d264740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea0e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.004+0000 7fca4d264740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.004+0000 7fca4d264740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5619eea226d0 con 0x5619eea0c9c0 2024-03-27T06:43:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.004+0000 7fca4d264740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5619eea22da0 con 0x5619eea0bfd0 2024-03-27T06:43:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.004+0000 7fca4d264740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5619eea23470 con 0x5619eea11b30 2024-03-27T06:43:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.004+0000 7fca43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea0e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.004+0000 7fca437fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea0de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.004+0000 7fca43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea0e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48142/0 (socket says 172.21.15.16:48142) 2024-03-27T06:43:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.005+0000 7fca43fff640 1 -- 172.21.15.16:0/3076408029 learned_addr learned my addr 172.21.15.16:0/3076408029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.005+0000 7fca437fe640 1 --2- 172.21.15.16:0/3076408029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.005+0000 7fca43fff640 1 -- 172.21.15.16:0/3076408029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 msgr2=0x5619eea1d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.005+0000 7fca43fff640 1 --2- 172.21.15.16:0/3076408029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.005+0000 7fca43fff640 1 -- 172.21.15.16:0/3076408029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 msgr2=0x5619eea0de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.005+0000 7fca43fff640 1 --2- 172.21.15.16:0/3076408029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea0de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.005+0000 7fca43fff640 1 -- 172.21.15.16:0/3076408029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619eea0cda0 con 0x5619eea0c9c0 2024-03-27T06:43:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.005+0000 7fca43fff640 1 --2- 172.21.15.16:0/3076408029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea0e3a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fca3402c170 tx=0x7fca34009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cef5fd69b793b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.005+0000 7fca39ffb640 1 -- 172.21.15.16:0/3076408029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3405a070 con 0x5619eea0c9c0 2024-03-27T06:43:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.005+0000 7fca39ffb640 1 -- 172.21.15.16:0/3076408029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fca340056d0 con 0x5619eea0c9c0 2024-03-27T06:43:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.005+0000 7fca39ffb640 1 -- 172.21.15.16:0/3076408029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34004920 con 0x5619eea0c9c0 2024-03-27T06:43:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 -- 172.21.15.16:0/3076408029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 msgr2=0x5619eea0e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 --2- 172.21.15.16:0/3076408029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea0e3a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fca3402c170 tx=0x7fca34009230 comp rx=0 tx=0).stop 2024-03-27T06:43:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 -- 172.21.15.16:0/3076408029 shutdown_connections 2024-03-27T06:43:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 --2- 172.21.15.16:0/3076408029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 --2- 172.21.15.16:0/3076408029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea0e3a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 --2- 172.21.15.16:0/3076408029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea0de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 -- 172.21.15.16:0/3076408029 >> 172.21.15.16:0/3076408029 conn(0x5619ee9d3b20 msgr2=0x5619eea07460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 -- 172.21.15.16:0/3076408029 shutdown_connections 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 -- 172.21.15.16:0/3076408029 wait complete. 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 -- start start 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 Processor -- start 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea19980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea19ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5619eea24210 con 0x5619eea0c9c0 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5619eea23b40 con 0x5619eea0bfd0 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5619eea248e0 con 0x5619eea11b30 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca4d264740 10 client.0 _ll_register_callbacks cb 0x7fca34009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea19ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.006+0000 7fca43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea19ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48150/0 (socket says 172.21.15.16:48150) 2024-03-27T06:43:25.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca43fff640 1 -- 172.21.15.16:0/4085602603 learned_addr learned my addr 172.21.15.16:0/4085602603 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca437fe640 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea19980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca437fe640 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca43fff640 1 -- 172.21.15.16:0/4085602603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 msgr2=0x5619eea1cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca43fff640 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca43fff640 1 -- 172.21.15.16:0/4085602603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 msgr2=0x5619eea19980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca43fff640 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea19980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca43fff640 1 -- 172.21.15.16:0/4085602603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619eea1d4d0 con 0x5619eea0c9c0 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca43fff640 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea19ec0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fca34003b10 tx=0x7fca340044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22027012140e4163 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca237fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca237fe640 1 -- 172.21.15.16:0/4085602603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3405a070 con 0x5619eea0c9c0 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca237fe640 1 -- 172.21.15.16:0/4085602603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fca340046c0 con 0x5619eea0c9c0 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca4d264740 1 -- 172.21.15.16:0/4085602603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca340056d0 con 0x5619eea0c9c0 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca237fe640 1 -- 172.21.15.16:0/4085602603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fca340035f0 con 0x5619eea0c9c0 2024-03-27T06:43:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca4d264740 10 client.15297 fetch_fsmap learned FSMap version 24 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca4d264740 1 -- 172.21.15.16:0/4085602603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5619eea894b0 con 0x5619eea0c9c0 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.007+0000 7fca237fe640 1 -- 172.21.15.16:0/4085602603 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fca340056d0 con 0x5619eea0c9c0 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca237fe640 1 -- 172.21.15.16:0/4085602603 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fca34069070 con 0x5619eea0c9c0 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca4d264740 10 client.15297 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca4d264740 10 client.15297 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca4d264740 20 client.15297 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca4d264740 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5619eea8df30 0x5619eea8cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca4d264740 4 client.15297 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca4d264740 1 -- 172.21.15.16:0/4085602603 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "status"}) -- 0x5619ee9788a0 con 0x5619eea8df30 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca437fe640 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5619eea8df30 0x5619eea8cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca237fe640 10 client.15297 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca437fe640 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5619eea8df30 0x5619eea8cc40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e63a7c67fa94ec5b server_cookie=7d10a5f22bbcd821 in_seq=0 out_seq=0 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca237fe640 1 -- 172.21.15.16:0/4085602603 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5619ee9788a0 con 0x5619eea8df30 2024-03-27T06:43:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca237fe640 10 client.15297 handle_command_reply: tid=0 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.008+0000 7fca4d264740 1 client.15297 shutdown 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4d264740 1 -- 172.21.15.16:0/4085602603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 msgr2=0x5619eea19ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4d264740 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea19ec0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fca34003b10 tx=0x7fca340044e0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4d264740 1 -- 172.21.15.16:0/4085602603 shutdown_connections 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4d264740 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5619eea8df30 0x5619eea8cc40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4d264740 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619eea11b30 0x5619eea1cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4d264740 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619eea0c9c0 0x5619eea19ec0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca237fe640 0 client.15297 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4d264740 1 --2- 172.21.15.16:0/4085602603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619eea0bfd0 0x5619eea19980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4d264740 1 -- 172.21.15.16:0/4085602603 >> 172.21.15.16:0/4085602603 conn(0x5619ee9d3b20 msgr2=0x5619eea0de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4d264740 1 -- 172.21.15.16:0/4085602603 shutdown_connections 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4d264740 1 -- 172.21.15.16:0/4085602603 wait complete. 2024-03-27T06:43:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4d264740 20 client.15297 trim_cache size 0 max 16384 2024-03-27T06:43:25.012 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:43:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:43:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-03-27T06:43:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24493, 2024-03-27T06:43:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-03-27T06:43:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-03-27T06:43:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-03-27T06:43:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 112.521653598, 2024-03-27T06:43:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-03-27T06:43:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-03-27T06:43:25.012 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-03-27T06:43:25.013 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 129.28132473700001 2024-03-27T06:43:25.013 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:43:25.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4a676640 1 -- 172.21.15.16:0/3595444777 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fca300a3610 msgr2=0x7fca300a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4a676640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fca300a3610 0x7fca300a5ad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fca3c09c6f0 tx=0x7fca3c087040 comp rx=0 tx=0).stop 2024-03-27T06:43:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4a676640 1 -- 172.21.15.16:0/3595444777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44103710 msgr2=0x7fca441a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4a676640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44103710 0x7fca441a1780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00eb10 tx=0x7fca3c00c2d0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4a676640 1 -- 172.21.15.16:0/3595444777 shutdown_connections 2024-03-27T06:43:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4a676640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fca300a3610 0x7fca300a5ad0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4a676640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca44103710 0x7fca441a1780 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4a676640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca44102d50 0x7fca4419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4a676640 1 --2- 172.21.15.16:0/3595444777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca44101b60 0x7fca4419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4a676640 1 -- 172.21.15.16:0/3595444777 >> 172.21.15.16:0/3595444777 conn(0x7fca440fd0f0 msgr2=0x7fca44073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.009+0000 7fca4a676640 1 -- 172.21.15.16:0/3595444777 shutdown_connections 2024-03-27T06:43:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.010+0000 7fca4a676640 1 -- 172.21.15.16:0/3595444777 wait complete. 2024-03-27T06:43:25.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-27T06:43:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.124+0000 7fdd68aed640 1 -- 172.21.15.16:0/1858637050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64100a70 msgr2=0x7fdd64100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.124+0000 7fdd68aed640 1 --2- 172.21.15.16:0/1858637050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64100a70 0x7fdd64100e90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009900 tx=0x7fdd5402bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.124+0000 7fdd68aed640 1 -- 172.21.15.16:0/1858637050 shutdown_connections 2024-03-27T06:43:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.124+0000 7fdd68aed640 1 --2- 172.21.15.16:0/1858637050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd64102620 0x7fdd64108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.124+0000 7fdd68aed640 1 --2- 172.21.15.16:0/1858637050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd64101c60 0x7fdd641020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.124+0000 7fdd68aed640 1 --2- 172.21.15.16:0/1858637050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64100a70 0x7fdd64100e90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.124+0000 7fdd68aed640 1 -- 172.21.15.16:0/1858637050 >> 172.21.15.16:0/1858637050 conn(0x7fdd640fc000 msgr2=0x7fdd640fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.124+0000 7fdd68aed640 1 -- 172.21.15.16:0/1858637050 shutdown_connections 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.124+0000 7fdd68aed640 1 -- 172.21.15.16:0/1858637050 wait complete. 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd68aed640 1 Processor -- start 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd68aed640 1 -- start start 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd68aed640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd64100a70 0x7fdd64104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd68aed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64101c60 0x7fdd64103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd68aed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd64102620 0x7fdd641036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd68aed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6410c7d0 con 0x7fdd64101c60 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd68aed640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6410c650 con 0x7fdd64100a70 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd68aed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6410c950 con 0x7fdd64102620 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64101c60 0x7fdd64103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64101c60 0x7fdd64103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48160/0 (socket says 172.21.15.16:48160) 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd5bfff640 1 -- 172.21.15.16:0/972429694 learned_addr learned my addr 172.21.15.16:0/972429694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd6259c640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd64100a70 0x7fdd64104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd62d9d640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd64102620 0x7fdd641036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd5bfff640 1 -- 172.21.15.16:0/972429694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd64102620 msgr2=0x7fdd641036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd5bfff640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd64102620 0x7fdd641036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd5bfff640 1 -- 172.21.15.16:0/972429694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd64100a70 msgr2=0x7fdd64104b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd5bfff640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd64100a70 0x7fdd64104b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.125+0000 7fdd5bfff640 1 -- 172.21.15.16:0/972429694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd64103f30 con 0x7fdd64101c60 2024-03-27T06:43:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.126+0000 7fdd5bfff640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64101c60 0x7fdd64103160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00e9f0 tx=0x7fdd4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.126+0000 7fdd5b7fe640 1 -- 172.21.15.16:0/972429694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00cd80 con 0x7fdd64101c60 2024-03-27T06:43:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.126+0000 7fdd68aed640 1 -- 172.21.15.16:0/972429694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd641a20b0 con 0x7fdd64101c60 2024-03-27T06:43:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.126+0000 7fdd5b7fe640 1 -- 172.21.15.16:0/972429694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdd4c005520 con 0x7fdd64101c60 2024-03-27T06:43:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.126+0000 7fdd5b7fe640 1 -- 172.21.15.16:0/972429694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c010690 con 0x7fdd64101c60 2024-03-27T06:43:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.126+0000 7fdd68aed640 1 -- 172.21.15.16:0/972429694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd641a2600 con 0x7fdd64101c60 2024-03-27T06:43:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.128+0000 7fdd5b7fe640 1 -- 172.21.15.16:0/972429694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fdd4c0046c0 con 0x7fdd64101c60 2024-03-27T06:43:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.128+0000 7fdd6b7b5740 1 Processor -- start 2024-03-27T06:43:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.128+0000 7fdd6b7b5740 1 -- start start 2024-03-27T06:43:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a8b30 0x55b7c36aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7c3608010 con 0x55b7c36a2fd0 2024-03-27T06:43:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55b7c35b0770 con 0x55b7c36a8b30 2024-03-27T06:43:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b7c3603fc0 con 0x55b7c36a39c0 2024-03-27T06:43:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd5b7fe640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdd500a3610 0x7fdd500a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd62d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd62d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48164/0 (socket says 172.21.15.16:48164) 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd62d9d640 1 -- 172.21.15.16:0/2800444451 learned_addr learned my addr 172.21.15.16:0/2800444451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd6259c640 1 --2- 172.21.15.16:0/2800444451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd5bfff640 1 --2- 172.21.15.16:0/2800444451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a8b30 0x55b7c36aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd62d9d640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdd500a3610 0x7fdd500a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd5b7fe640 1 -- 172.21.15.16:0/972429694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fdd4c0d01c0 con 0x7fdd64101c60 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd62d9d640 1 -- 172.21.15.16:0/2800444451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 msgr2=0x55b7c36a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd62d9d640 1 --2- 172.21.15.16:0/2800444451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36a84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd62d9d640 1 -- 172.21.15.16:0/2800444451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a8b30 msgr2=0x55b7c36aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd62d9d640 1 --2- 172.21.15.16:0/2800444451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a8b30 0x55b7c36aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd62d9d640 1 -- 172.21.15.16:0/2800444451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c35cf330 con 0x55b7c36a2fd0 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd6259c640 1 --2- 172.21.15.16:0/2800444451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd62d9d640 1 --2- 172.21.15.16:0/2800444451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a33f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c011e20 tx=0x7fdd5c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8e9abf066bd879 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd62d9d640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdd500a3610 0x7fdd500a5ad0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00fc90 tx=0x7fdd5c01e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.129+0000 7fdd58ff9640 1 -- 172.21.15.16:0/2800444451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c01dcb0 con 0x55b7c36a2fd0 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.130+0000 7fdd58ff9640 1 -- 172.21.15.16:0/2800444451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdd5c01de50 con 0x55b7c36a2fd0 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.130+0000 7fdd58ff9640 1 -- 172.21.15.16:0/2800444451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c004330 con 0x55b7c36a2fd0 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.130+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2800444451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 msgr2=0x55b7c36a33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.130+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2800444451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a33f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c011e20 tx=0x7fdd5c00f790 comp rx=0 tx=0).stop 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.130+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2800444451 shutdown_connections 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.130+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2800444451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a8b30 0x55b7c36aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.130+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2800444451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.130+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2800444451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a33f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.130+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2800444451 >> 172.21.15.16:0/2800444451 conn(0x55b7c366ab20 msgr2=0x55b7c369e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.130+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2800444451 shutdown_connections 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.130+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2800444451 wait complete. 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd6b7b5740 1 -- start start 2024-03-27T06:43:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd6b7b5740 1 Processor -- start 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a2fd0 0x55b7c36ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36ada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a8b30 0x55b7c36b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7c36aba20 con 0x55b7c36a8b30 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55b7c35c3b30 con 0x55b7c36a2fd0 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b7c36abba0 con 0x55b7c36a39c0 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd6b7b5740 10 client.0 _ll_register_callbacks cb 0x55b7c35ff1a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd62d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a8b30 0x55b7c36b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a2fd0 0x55b7c36ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd62d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a8b30 0x55b7c36b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48166/0 (socket says 172.21.15.16:48166) 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd62d9d640 1 -- 172.21.15.16:0/2477156766 learned_addr learned my addr 172.21.15.16:0/2477156766 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd5bfff640 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd62d9d640 1 -- 172.21.15.16:0/2477156766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 msgr2=0x55b7c36ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd62d9d640 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd62d9d640 1 -- 172.21.15.16:0/2477156766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a2fd0 msgr2=0x55b7c36ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd62d9d640 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a2fd0 0x55b7c36ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd62d9d640 1 -- 172.21.15.16:0/2477156766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c331c2d0 con 0x55b7c36a8b30 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd5bfff640 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd62d9d640 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a8b30 0x55b7c36b4a40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c005c10 tx=0x7fdd5c01dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc4b55f37af7011 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.131+0000 7fdd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2477156766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c02a050 con 0x55b7c36a8b30 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2477156766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fdd5c003970 con 0x55b7c36a8b30 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2477156766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd5c01de50 con 0x55b7c36a8b30 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2477156766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdd5c003b70 con 0x55b7c36a8b30 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd6b7b5740 10 client.15309 fetch_fsmap learned FSMap version 24 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2477156766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b7c33e66b0 con 0x55b7c36a8b30 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2477156766 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdd5c01de50 con 0x55b7c36a8b30 2024-03-27T06:43:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2477156766 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fdd5c004fc0 con 0x55b7c36a8b30 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd6b7b5740 10 client.15309 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd6b7b5740 10 client.15309 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd6b7b5740 20 client.15309 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55b7c36b9df0 0x55b7c36bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd6b7b5740 4 client.15309 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2477156766 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7c3615550 con 0x55b7c36b9df0 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.132+0000 7fdd6259c640 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55b7c36b9df0 0x55b7c36bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.133+0000 7fdd6259c640 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55b7c36b9df0 0x55b7c36bb260 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4587fc062f4739a7 server_cookie=416d53775b514f1b in_seq=0 out_seq=0 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.133+0000 7fdd49ffb640 10 client.15309 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.133+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2477156766 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b7c3615550 con 0x55b7c36b9df0 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.133+0000 7fdd49ffb640 10 client.15309 handle_command_reply: tid=0 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.133+0000 7fdd6b7b5740 1 client.15309 shutdown 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2477156766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a8b30 msgr2=0x55b7c36b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a8b30 0x55b7c36b4a40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c005c10 tx=0x7fdd5c01dcb0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2477156766 shutdown_connections 2024-03-27T06:43:25.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55b7c36b9df0 0x55b7c36bb260 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a8b30 0x55b7c36b4a40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd49ffb640 0 client.15309 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:43:25.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2477156766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a2fd0 0x55b7c36ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2477156766 >> 172.21.15.16:0/2477156766 conn(0x55b7c366ab20 msgr2=0x55b7c369db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2477156766 shutdown_connections 2024-03-27T06:43:25.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2477156766 wait complete. 2024-03-27T06:43:25.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.134+0000 7fdd6b7b5740 20 client.15309 trim_cache size 0 max 16384 2024-03-27T06:43:25.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6b7b5740 1 Processor -- start 2024-03-27T06:43:25.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6b7b5740 1 -- start start 2024-03-27T06:43:25.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a39c0 0x55b7c36b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a8b30 0x55b7c36a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7c36b95a0 con 0x55b7c36a2fd0 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55b7c36b9c70 con 0x55b7c36a39c0 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b7c36ba340 con 0x55b7c36a8b30 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48172/0 (socket says 172.21.15.16:48172) 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6259c640 1 -- 172.21.15.16:0/73488106 learned_addr learned my addr 172.21.15.16:0/73488106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd62d9d640 1 --2- 172.21.15.16:0/73488106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a8b30 0x55b7c36a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd5bfff640 1 --2- 172.21.15.16:0/73488106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a39c0 0x55b7c36b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6259c640 1 -- 172.21.15.16:0/73488106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a8b30 msgr2=0x55b7c36a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6259c640 1 --2- 172.21.15.16:0/73488106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a8b30 0x55b7c36a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6259c640 1 -- 172.21.15.16:0/73488106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a39c0 msgr2=0x55b7c36b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6259c640 1 --2- 172.21.15.16:0/73488106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a39c0 0x55b7c36b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.152+0000 7fdd6259c640 1 -- 172.21.15.16:0/73488106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c36ab380 con 0x55b7c36a2fd0 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd6259c640 1 --2- 172.21.15.16:0/73488106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a1b80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdd54000c00 tx=0x7fdd54005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f4734743a7a5dd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd4bfff640 1 -- 172.21.15.16:0/73488106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5402cad0 con 0x55b7c36a2fd0 2024-03-27T06:43:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd4bfff640 1 -- 172.21.15.16:0/73488106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdd5404a070 con 0x55b7c36a2fd0 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd4bfff640 1 -- 172.21.15.16:0/73488106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd540055f0 con 0x55b7c36a2fd0 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/73488106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 msgr2=0x55b7c36a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/73488106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a1b80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdd54000c00 tx=0x7fdd54005fd0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/73488106 shutdown_connections 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/73488106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a8b30 0x55b7c36a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/73488106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a39c0 0x55b7c36b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/73488106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36a1b80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/73488106 >> 172.21.15.16:0/73488106 conn(0x55b7c366ab20 msgr2=0x55b7c369e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/73488106 shutdown_connections 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.153+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/73488106 wait complete. 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6b7b5740 1 -- start start 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6b7b5740 1 Processor -- start 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6b7b5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a8b30 0x55b7c36b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b7c36bb0e0 con 0x55b7c36a2fd0 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55b7c36baa10 con 0x55b7c36a8b30 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6b7b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b7c36bb7b0 con 0x55b7c36a39c0 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6b7b5740 10 client.0 _ll_register_callbacks cb 0x7fdd4c01cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48186/0 (socket says 172.21.15.16:48186) 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6259c640 1 -- 172.21.15.16:0/2649149027 learned_addr learned my addr 172.21.15.16:0/2649149027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd5bfff640 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd62d9d640 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a8b30 0x55b7c36b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6259c640 1 -- 172.21.15.16:0/2649149027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 msgr2=0x55b7c36b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6259c640 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6259c640 1 -- 172.21.15.16:0/2649149027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a8b30 msgr2=0x55b7c36b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6259c640 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a8b30 0x55b7c36b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6259c640 1 -- 172.21.15.16:0/2649149027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c36b46b0 con 0x55b7c36a2fd0 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd6259c640 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36b0b60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdd5402c900 tx=0x7fdd540055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6ecb96f435e0f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.154+0000 7fdd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2649149027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54054060 con 0x55b7c36a2fd0 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2649149027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fdd5402cce0 con 0x55b7c36a2fd0 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2649149027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd5404a070 con 0x55b7c36a2fd0 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2649149027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdd54000e60 con 0x55b7c36a2fd0 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd6b7b5740 10 client.15315 fetch_fsmap learned FSMap version 24 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2649149027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b7c37204e0 con 0x55b7c36a2fd0 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2649149027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdd5404a070 con 0x55b7c36a2fd0 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2649149027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fdd540035f0 con 0x55b7c36a2fd0 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd6b7b5740 10 client.15315 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:43:25.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd6b7b5740 10 client.15315 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd6b7b5740 20 client.15315 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55b7c3724d40 0x55b7c3723a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd6b7b5740 4 client.15315 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2649149027 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "status"}) -- 0x55b7c360f8a0 con 0x55b7c3724d40 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.155+0000 7fdd5bfff640 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55b7c3724d40 0x55b7c3723a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd49ffb640 10 client.15315 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd5bfff640 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55b7c3724d40 0x55b7c3723a50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60c16e2f734d9213 server_cookie=e134606a667dd5c7 in_seq=0 out_seq=0 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd49ffb640 1 -- 172.21.15.16:0/2649149027 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b7c360f8a0 con 0x55b7c3724d40 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd49ffb640 10 client.15315 handle_command_reply: tid=0 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 1 client.15315 shutdown 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2649149027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 msgr2=0x55b7c36b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36b0b60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdd5402c900 tx=0x7fdd540055f0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2649149027 shutdown_connections 2024-03-27T06:43:25.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55b7c3724d40 0x55b7c3723a50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b7c36a8b30 0x55b7c36b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b7c36a39c0 0x55b7c36b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd49ffb640 0 client.15315 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:43:25.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 1 --2- 172.21.15.16:0/2649149027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b7c36a2fd0 0x55b7c36b0b60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2649149027 >> 172.21.15.16:0/2649149027 conn(0x55b7c366ab20 msgr2=0x55b7c36a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2649149027 shutdown_connections 2024-03-27T06:43:25.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 1 -- 172.21.15.16:0/2649149027 wait complete. 2024-03-27T06:43:25.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.156+0000 7fdd6b7b5740 20 client.15315 trim_cache size 0 max 16384 2024-03-27T06:43:25.158 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:43:25.158 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:43:25.158 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-03-27T06:43:25.159 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24500, 2024-03-27T06:43:25.159 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-03-27T06:43:25.159 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-03-27T06:43:25.159 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-03-27T06:43:25.159 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 113.67903051899999, 2024-03-27T06:43:25.159 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-03-27T06:43:25.159 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-03-27T06:43:25.159 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-03-27T06:43:25.159 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 132.16852763700001 2024-03-27T06:43:25.159 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:43:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 -- 172.21.15.16:0/972429694 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdd500a3610 msgr2=0x7fdd500a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdd500a3610 0x7fdd500a5ad0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00fc90 tx=0x7fdd5c01e000 comp rx=0 tx=0).stop 2024-03-27T06:43:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 -- 172.21.15.16:0/972429694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64101c60 msgr2=0x7fdd64103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64101c60 0x7fdd64103160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00e9f0 tx=0x7fdd4c00edf0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 -- 172.21.15.16:0/972429694 shutdown_connections 2024-03-27T06:43:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdd500a3610 0x7fdd500a5ad0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd64102620 0x7fdd641036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64101c60 0x7fdd64103160 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 --2- 172.21.15.16:0/972429694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd64100a70 0x7fdd64104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 -- 172.21.15.16:0/972429694 >> 172.21.15.16:0/972429694 conn(0x7fdd640fc000 msgr2=0x7fdd6406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 -- 172.21.15.16:0/972429694 shutdown_connections 2024-03-27T06:43:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.157+0000 7fdd68aed640 1 -- 172.21.15.16:0/972429694 wait complete. 2024-03-27T06:43:25.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-27T06:43:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.271+0000 7f3283b8a640 1 -- 172.21.15.16:0/4226401216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327c101c50 msgr2=0x7f327c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.271+0000 7f3283b8a640 1 --2- 172.21.15.16:0/4226401216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327c101c50 0x7f327c1020d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f326c009900 tx=0x7f326c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.271+0000 7f3283b8a640 1 -- 172.21.15.16:0/4226401216 shutdown_connections 2024-03-27T06:43:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.271+0000 7f3283b8a640 1 --2- 172.21.15.16:0/4226401216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f327c102610 0x7f327c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.271+0000 7f3283b8a640 1 --2- 172.21.15.16:0/4226401216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327c101c50 0x7f327c1020d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.271+0000 7f3283b8a640 1 --2- 172.21.15.16:0/4226401216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c100a60 0x7f327c100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.271+0000 7f3283b8a640 1 -- 172.21.15.16:0/4226401216 >> 172.21.15.16:0/4226401216 conn(0x7f327c0fbff0 msgr2=0x7f327c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.271+0000 7f3283b8a640 1 -- 172.21.15.16:0/4226401216 shutdown_connections 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.271+0000 7f3283b8a640 1 -- 172.21.15.16:0/4226401216 wait complete. 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3283b8a640 1 Processor -- start 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3283b8a640 1 -- start start 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3283b8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327c100a60 0x7f327c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3283b8a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f327c101c50 0x7f327c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3283b8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c102610 0x7f327c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3283b8a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f327c10c7c0 con 0x7f327c100a60 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3283b8a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f327c10c640 con 0x7f327c101c50 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3283b8a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f327c10c940 con 0x7f327c102610 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3281926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327c100a60 0x7f327c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3281926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327c100a60 0x7f327c198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48194/0 (socket says 172.21.15.16:48194) 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3281926640 1 -- 172.21.15.16:0/589149804 learned_addr learned my addr 172.21.15.16:0/589149804 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3281125640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f327c101c50 0x7f327c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3282127640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c102610 0x7f327c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3281926640 1 -- 172.21.15.16:0/589149804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c102610 msgr2=0x7f327c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3281926640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c102610 0x7f327c19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3281926640 1 -- 172.21.15.16:0/589149804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f327c101c50 msgr2=0x7f327c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3281926640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f327c101c50 0x7f327c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.272+0000 7f3281926640 1 -- 172.21.15.16:0/589149804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c19fbc0 con 0x7f327c100a60 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.273+0000 7f3281926640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327c100a60 0x7f327c198010 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f327800e9c0 tx=0x7f327800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.273+0000 7f3272ffd640 1 -- 172.21.15.16:0/589149804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327800cd80 con 0x7f327c100a60 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.273+0000 7f3272ffd640 1 -- 172.21.15.16:0/589149804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32780035f0 con 0x7f327c100a60 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.273+0000 7f3283b8a640 1 -- 172.21.15.16:0/589149804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327c19feb0 con 0x7f327c100a60 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.273+0000 7f3272ffd640 1 -- 172.21.15.16:0/589149804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278010670 con 0x7f327c100a60 2024-03-27T06:43:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.273+0000 7f3283b8a640 1 -- 172.21.15.16:0/589149804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327c1a02c0 con 0x7f327c100a60 2024-03-27T06:43:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3272ffd640 1 -- 172.21.15.16:0/589149804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7f32780037a0 con 0x7f327c100a60 2024-03-27T06:43:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3286773740 1 Processor -- start 2024-03-27T06:43:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3286773740 1 -- start start 2024-03-27T06:43:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3286773740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e7363fd0 0x5650e73643f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3286773740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e73649c0 0x5650e73694c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3286773740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7369b30 0x5650e736bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3286773740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650e72c9010 con 0x5650e7369b30 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3286773740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5650e7271770 con 0x5650e7363fd0 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3286773740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5650e72c4fc0 con 0x5650e73649c0 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3281125640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7369b30 0x5650e736bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3281125640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7369b30 0x5650e736bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48208/0 (socket says 172.21.15.16:48208) 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3281125640 1 -- 172.21.15.16:0/1697300315 learned_addr learned my addr 172.21.15.16:0/1697300315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3282127640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e7363fd0 0x5650e73643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.276+0000 7f3281926640 1 --2- 172.21.15.16:0/1697300315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e73649c0 0x5650e73694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3272ffd640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32680a3610 0x7f32680a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3281125640 1 -- 172.21.15.16:0/1697300315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e73649c0 msgr2=0x5650e73694c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3281125640 1 --2- 172.21.15.16:0/1697300315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e73649c0 0x5650e73694c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3281125640 1 -- 172.21.15.16:0/1697300315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e7363fd0 msgr2=0x5650e73643f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3281125640 1 --2- 172.21.15.16:0/1697300315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e7363fd0 0x5650e73643f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3281125640 1 -- 172.21.15.16:0/1697300315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e7290330 con 0x5650e7369b30 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3282127640 1 --2- 172.21.15.16:0/1697300315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e7363fd0 0x5650e73643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3282127640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32680a3610 0x7f32680a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3272ffd640 1 -- 172.21.15.16:0/589149804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f32780cffe0 con 0x7f327c100a60 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3281125640 1 --2- 172.21.15.16:0/1697300315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7369b30 0x5650e736bfc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f326c009900 tx=0x7f326c009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca69ce1ca71189da server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3267fff640 1 -- 172.21.15.16:0/1697300315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c02c5d0 con 0x5650e7369b30 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3267fff640 1 -- 172.21.15.16:0/1697300315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f326c004040 con 0x5650e7369b30 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3267fff640 1 -- 172.21.15.16:0/1697300315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c0046c0 con 0x5650e7369b30 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.277+0000 7f3282127640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32680a3610 0x7f32680a5ad0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5650e7365220 tx=0x7f3274004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.278+0000 7f3286773740 1 -- 172.21.15.16:0/1697300315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7369b30 msgr2=0x5650e736bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.278+0000 7f3286773740 1 --2- 172.21.15.16:0/1697300315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7369b30 0x5650e736bfc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f326c009900 tx=0x7f326c009670 comp rx=0 tx=0).stop 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.278+0000 7f3286773740 1 -- 172.21.15.16:0/1697300315 shutdown_connections 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.278+0000 7f3286773740 1 --2- 172.21.15.16:0/1697300315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7369b30 0x5650e736bfc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.278+0000 7f3286773740 1 --2- 172.21.15.16:0/1697300315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e73649c0 0x5650e73694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.278+0000 7f3286773740 1 --2- 172.21.15.16:0/1697300315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e7363fd0 0x5650e73643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.278+0000 7f3286773740 1 -- 172.21.15.16:0/1697300315 >> 172.21.15.16:0/1697300315 conn(0x5650e732bb20 msgr2=0x5650e735f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.278+0000 7f3286773740 1 -- 172.21.15.16:0/1697300315 shutdown_connections 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.278+0000 7f3286773740 1 -- 172.21.15.16:0/1697300315 wait complete. 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.278+0000 7f3286773740 1 -- start start 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.278+0000 7f3286773740 1 Processor -- start 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3286773740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3286773740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e73649c0 0x5650e736e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3286773740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7369b30 0x5650e7375970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3286773740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650e736ca80 con 0x5650e7363fd0 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3286773740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5650e7284b30 con 0x5650e73649c0 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3286773740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5650e736cc00 con 0x5650e7369b30 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3286773740 10 client.0 _ll_register_callbacks cb 0x5650e72c01a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3281926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3281125640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e73649c0 0x5650e736e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3281926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48220/0 (socket says 172.21.15.16:48220) 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3281926640 1 -- 172.21.15.16:0/1763627751 learned_addr learned my addr 172.21.15.16:0/1763627751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3282127640 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7369b30 0x5650e7375970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3281926640 1 -- 172.21.15.16:0/1763627751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7369b30 msgr2=0x5650e7375970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3281926640 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7369b30 0x5650e7375970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3281926640 1 -- 172.21.15.16:0/1763627751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e73649c0 msgr2=0x5650e736e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3281926640 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e73649c0 0x5650e736e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3281926640 1 -- 172.21.15.16:0/1763627751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e6fdd2d0 con 0x5650e7363fd0 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3281926640 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3278098f30 tx=0x7f3278009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3c07730b2b5610 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f32657fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f32657fa640 1 -- 172.21.15.16:0/1763627751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327809c0a0 con 0x5650e7363fd0 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f32657fa640 1 -- 172.21.15.16:0/1763627751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3278003ba0 con 0x5650e7363fd0 2024-03-27T06:43:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.279+0000 7f3286773740 1 -- 172.21.15.16:0/1763627751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f326c004040 con 0x5650e7363fd0 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f32657fa640 1 -- 172.21.15.16:0/1763627751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f327809bec0 con 0x5650e7363fd0 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f3286773740 10 client.15327 fetch_fsmap learned FSMap version 24 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f3286773740 1 -- 172.21.15.16:0/1763627751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5650e70a76b0 con 0x5650e7363fd0 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f32657fa640 1 -- 172.21.15.16:0/1763627751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f326c004040 con 0x5650e7363fd0 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f32657fa640 1 -- 172.21.15.16:0/1763627751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f327809b9b0 con 0x5650e7363fd0 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f3286773740 10 client.15327 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f3286773740 10 client.15327 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f3286773740 20 client.15327 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f3286773740 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5650e737aef0 0x5650e737c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f3286773740 4 client.15327 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f3286773740 1 -- 172.21.15.16:0/1763627751 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650e72d6550 con 0x5650e737aef0 2024-03-27T06:43:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.280+0000 7f3281125640 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5650e737aef0 0x5650e737c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f32657fa640 10 client.15327 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3281125640 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5650e737aef0 0x5650e737c360 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e74c8d2a88614d2d server_cookie=d1d88e1ef95659b3 in_seq=0 out_seq=0 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f32657fa640 1 -- 172.21.15.16:0/1763627751 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5650e72d6550 con 0x5650e737aef0 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f32657fa640 10 client.15327 handle_command_reply: tid=0 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3286773740 1 client.15327 shutdown 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3286773740 1 -- 172.21.15.16:0/1763627751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 msgr2=0x5650e736e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3286773740 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3278098f30 tx=0x7f3278009740 comp rx=0 tx=0).stop 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3286773740 1 -- 172.21.15.16:0/1763627751 shutdown_connections 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3286773740 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5650e737aef0 0x5650e737c360 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3286773740 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7369b30 0x5650e7375970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3286773740 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e73649c0 0x5650e736e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f32657fa640 0 client.15327 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3286773740 1 --2- 172.21.15.16:0/1763627751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e3f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3286773740 1 -- 172.21.15.16:0/1763627751 >> 172.21.15.16:0/1763627751 conn(0x5650e732bb20 msgr2=0x5650e735eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3286773740 1 -- 172.21.15.16:0/1763627751 shutdown_connections 2024-03-27T06:43:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.281+0000 7f3286773740 1 -- 172.21.15.16:0/1763627751 wait complete. 2024-03-27T06:43:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.282+0000 7f3286773740 20 client.15327 trim_cache size 0 max 16384 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.299+0000 7f3286773740 1 Processor -- start 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.299+0000 7f3286773740 1 -- start start 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3286773740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3286773740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e73649c0 0x5650e7375900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3286773740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7369b30 0x5650e7362b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3286773740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650e737a6a0 con 0x5650e7363fd0 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3286773740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5650e737ad70 con 0x5650e73649c0 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3286773740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5650e737b440 con 0x5650e7369b30 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3281926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3281926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48232/0 (socket says 172.21.15.16:48232) 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3281926640 1 -- 172.21.15.16:0/3730650640 learned_addr learned my addr 172.21.15.16:0/3730650640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3281125640 1 --2- 172.21.15.16:0/3730650640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e73649c0 0x5650e7375900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3282127640 1 --2- 172.21.15.16:0/3730650640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7369b30 0x5650e7362b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3281926640 1 -- 172.21.15.16:0/3730650640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7369b30 msgr2=0x5650e7362b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3281926640 1 --2- 172.21.15.16:0/3730650640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7369b30 0x5650e7362b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3281926640 1 -- 172.21.15.16:0/3730650640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e73649c0 msgr2=0x5650e7375900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3281926640 1 --2- 172.21.15.16:0/3730650640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e73649c0 0x5650e7375900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3281926640 1 -- 172.21.15.16:0/3730650640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e73690f0 con 0x5650e7363fd0 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f3281926640 1 --2- 172.21.15.16:0/3730650640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e710 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f327809a810 tx=0x7f327809aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee6b9db3fcdcfc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f32677fe640 1 -- 172.21.15.16:0/3730650640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327809c0a0 con 0x5650e7363fd0 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f32677fe640 1 -- 172.21.15.16:0/3730650640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32780a0910 con 0x5650e7363fd0 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.300+0000 7f32677fe640 1 -- 172.21.15.16:0/3730650640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327809b9b0 con 0x5650e7363fd0 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.301+0000 7f3286773740 1 -- 172.21.15.16:0/3730650640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 msgr2=0x5650e736e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.301+0000 7f3286773740 1 --2- 172.21.15.16:0/3730650640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e710 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f327809a810 tx=0x7f327809aa70 comp rx=0 tx=0).stop 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.301+0000 7f3286773740 1 -- 172.21.15.16:0/3730650640 shutdown_connections 2024-03-27T06:43:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.301+0000 7f3286773740 1 --2- 172.21.15.16:0/3730650640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7369b30 0x5650e7362b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.301+0000 7f3286773740 1 --2- 172.21.15.16:0/3730650640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e73649c0 0x5650e7375900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.301+0000 7f3286773740 1 --2- 172.21.15.16:0/3730650640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e7363fd0 0x5650e736e710 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.301+0000 7f3286773740 1 -- 172.21.15.16:0/3730650640 >> 172.21.15.16:0/3730650640 conn(0x5650e732bb20 msgr2=0x5650e736e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.301+0000 7f3286773740 1 -- 172.21.15.16:0/3730650640 shutdown_connections 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.301+0000 7f3286773740 1 -- 172.21.15.16:0/3730650640 wait complete. 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.301+0000 7f3286773740 1 -- start start 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.301+0000 7f3286773740 1 Processor -- start 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3286773740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7363fd0 0x5650e73715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3286773740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e73649c0 0x5650e7371b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3286773740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e7369b30 0x5650e7374b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3286773740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650e737c1e0 con 0x5650e73649c0 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3286773740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5650e737bb10 con 0x5650e7369b30 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3286773740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5650e737c8b0 con 0x5650e7363fd0 2024-03-27T06:43:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3286773740 10 client.0 _ll_register_callbacks cb 0x7f3278003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3281125640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e73649c0 0x5650e7371b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3281926640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7363fd0 0x5650e73715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3281926640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7363fd0 0x5650e73715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:48978/0 (socket says 172.21.15.16:48978) 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3281125640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e73649c0 0x5650e7371b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48242/0 (socket says 172.21.15.16:48242) 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3282127640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e7369b30 0x5650e7374b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3281926640 1 -- 172.21.15.16:0/1743511117 learned_addr learned my addr 172.21.15.16:0/1743511117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3281125640 1 -- 172.21.15.16:0/1743511117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7363fd0 msgr2=0x5650e73715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3281125640 1 --2- 172.21.15.16:0/1743511117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7363fd0 0x5650e73715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3281125640 1 -- 172.21.15.16:0/1743511117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e7369b30 msgr2=0x5650e7374b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3281125640 1 --2- 172.21.15.16:0/1743511117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e7369b30 0x5650e7374b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3281125640 1 -- 172.21.15.16:0/1743511117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e7375140 con 0x5650e73649c0 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3281125640 1 --2- 172.21.15.16:0/1743511117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e73649c0 0x5650e7371b30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f326c02c900 tx=0x7f326c02c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41c776da15a5417 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f32657fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f32657fa640 1 -- 172.21.15.16:0/1743511117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c053d90 con 0x5650e73649c0 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f32657fa640 1 -- 172.21.15.16:0/1743511117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f326c004040 con 0x5650e73649c0 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3286773740 1 -- 172.21.15.16:0/1743511117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32780a0910 con 0x5650e73649c0 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f32657fa640 1 -- 172.21.15.16:0/1743511117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f326c004260 con 0x5650e73649c0 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3286773740 10 client.15333 fetch_fsmap learned FSMap version 24 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.302+0000 7f3286773740 1 -- 172.21.15.16:0/1743511117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5650e73e1220 con 0x5650e73649c0 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f32657fa640 1 -- 172.21.15.16:0/1743511117 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32780a0910 con 0x5650e73649c0 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f32657fa640 1 -- 172.21.15.16:0/1743511117 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f326c02cce0 con 0x5650e73649c0 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f3286773740 10 client.15333 fetch_fsmap finished waiting for FSMap version 24 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f3286773740 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f3286773740 20 client.15333 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f3286773740 1 --2- 172.21.15.16:0/1743511117 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5650e73e5960 0x5650e73e4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f3286773740 4 client.15333 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f3286773740 1 -- 172.21.15.16:0/1743511117 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "status"}) -- 0x5650e72d08a0 con 0x5650e73e5960 2024-03-27T06:43:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f3281926640 1 --2- 172.21.15.16:0/1743511117 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5650e73e5960 0x5650e73e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f32657fa640 10 client.15333 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f3281926640 1 --2- 172.21.15.16:0/1743511117 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5650e73e5960 0x5650e73e4670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e577d039243318fd server_cookie=1a5c36194e570c90 in_seq=0 out_seq=0 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f32657fa640 1 -- 172.21.15.16:0/1743511117 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5650e72d08a0 con 0x5650e73e5960 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f32657fa640 10 client.15333 handle_command_reply: tid=0 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.303+0000 7f3286773740 1 client.15333 shutdown 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3286773740 1 -- 172.21.15.16:0/1743511117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e73649c0 msgr2=0x5650e7371b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3286773740 1 --2- 172.21.15.16:0/1743511117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e73649c0 0x5650e7371b30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f326c02c900 tx=0x7f326c02c800 comp rx=0 tx=0).stop 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3286773740 1 -- 172.21.15.16:0/1743511117 shutdown_connections 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3286773740 1 --2- 172.21.15.16:0/1743511117 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5650e73e5960 0x5650e73e4670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3286773740 1 --2- 172.21.15.16:0/1743511117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5650e7369b30 0x5650e7374b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3286773740 1 --2- 172.21.15.16:0/1743511117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650e73649c0 0x5650e7371b30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3286773740 1 --2- 172.21.15.16:0/1743511117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650e7363fd0 0x5650e73715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f32657fa640 0 client.15333 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3286773740 1 -- 172.21.15.16:0/1743511117 >> 172.21.15.16:0/1743511117 conn(0x5650e732bb20 msgr2=0x5650e7363410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3286773740 1 -- 172.21.15.16:0/1743511117 shutdown_connections 2024-03-27T06:43:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3286773740 1 -- 172.21.15.16:0/1743511117 wait complete. 2024-03-27T06:43:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3286773740 20 client.15333 trim_cache size 0 max 16384 2024-03-27T06:43:25.306 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:43:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "c9095bc4-ec03-11ee-95d0-87774f69a715", 2024-03-27T06:43:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-03-27T06:43:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24527, 2024-03-27T06:43:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-03-27T06:43:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-03-27T06:43:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-03-27T06:43:25.306 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 115.920363364, 2024-03-27T06:43:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-03-27T06:43:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 88, 2024-03-27T06:43:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2024-03-27T06:43:25.307 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 121.05201621000001 2024-03-27T06:43:25.307 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 -- 172.21.15.16:0/589149804 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32680a3610 msgr2=0x7f32680a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32680a3610 0x7f32680a5ad0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5650e7365220 tx=0x7f3274004a40 comp rx=0 tx=0).stop 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 -- 172.21.15.16:0/589149804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327c100a60 msgr2=0x7f327c198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327c100a60 0x7f327c198010 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f327800e9c0 tx=0x7f327800ee80 comp rx=0 tx=0).stop 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 -- 172.21.15.16:0/589149804 shutdown_connections 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f32680a3610 0x7f32680a5ad0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c102610 0x7f327c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f327c101c50 0x7f327c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 --2- 172.21.15.16:0/589149804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f327c100a60 0x7f327c198010 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 -- 172.21.15.16:0/589149804 >> 172.21.15.16:0/589149804 conn(0x7f327c0fbff0 msgr2=0x7f327c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 -- 172.21.15.16:0/589149804 shutdown_connections 2024-03-27T06:43:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.304+0000 7f3283b8a640 1 -- 172.21.15.16:0/589149804 wait complete. 2024-03-27T06:43:25.310 INFO:teuthology.run_tasks:Running task sequential... 2024-03-27T06:43:25.320 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-27T06:43:25.320 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-27T06:43:25.320 INFO:teuthology.task.print:Enabling mgr modules 2024-03-27T06:43:25.320 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-03-27T06:43:25.321 INFO:teuthology.task.exec:Executing custom commands... 2024-03-27T06:43:25.321 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-03-27T06:43:25.322 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-27T06:43:25.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.429+0000 7fdbccbe2640 1 -- 172.21.15.16:0/2737948351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8102620 msgr2=0x7fdbc8108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.429+0000 7fdbccbe2640 1 --2- 172.21.15.16:0/2737948351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8102620 0x7fdbc8108eb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009ca0 tx=0x7fdbc002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.430+0000 7fdbccbe2640 1 -- 172.21.15.16:0/2737948351 shutdown_connections 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.430+0000 7fdbccbe2640 1 --2- 172.21.15.16:0/2737948351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8102620 0x7fdbc8108eb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.430+0000 7fdbccbe2640 1 --2- 172.21.15.16:0/2737948351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc8101c60 0x7fdbc81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.430+0000 7fdbccbe2640 1 --2- 172.21.15.16:0/2737948351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbc8100a70 0x7fdbc8100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.430+0000 7fdbccbe2640 1 -- 172.21.15.16:0/2737948351 >> 172.21.15.16:0/2737948351 conn(0x7fdbc80fbfe0 msgr2=0x7fdbc80fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.430+0000 7fdbccbe2640 1 -- 172.21.15.16:0/2737948351 shutdown_connections 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.430+0000 7fdbccbe2640 1 -- 172.21.15.16:0/2737948351 wait complete. 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.430+0000 7fdbccbe2640 1 Processor -- start 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.430+0000 7fdbccbe2640 1 -- start start 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.430+0000 7fdbccbe2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8100a70 0x7fdbc810b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbccbe2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbc8101c60 0x7fdbc8109a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbccbe2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc8102620 0x7fdbc8109f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbccbe2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc810d930 con 0x7fdbc8100a70 2024-03-27T06:43:25.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbccbe2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc810d7b0 con 0x7fdbc8102620 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbccbe2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc810dab0 con 0x7fdbc8101c60 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbc659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8100a70 0x7fdbc810b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbc659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8100a70 0x7fdbc810b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48262/0 (socket says 172.21.15.16:48262) 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbc659c640 1 -- 172.21.15.16:0/1382762486 learned_addr learned my addr 172.21.15.16:0/1382762486 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbc5d9b640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbc8101c60 0x7fdbc8109a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbc6d9d640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc8102620 0x7fdbc8109f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbc659c640 1 -- 172.21.15.16:0/1382762486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbc8101c60 msgr2=0x7fdbc8109a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbc659c640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbc8101c60 0x7fdbc8109a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbc659c640 1 -- 172.21.15.16:0/1382762486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc8102620 msgr2=0x7fdbc8109f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbc659c640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc8102620 0x7fdbc8109f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbc659c640 1 -- 172.21.15.16:0/1382762486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc810a800 con 0x7fdbc8100a70 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbc659c640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8100a70 0x7fdbc810b3f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800e9c0 tx=0x7fdbb800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbb77fe640 1 -- 172.21.15.16:0/1382762486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb800cd80 con 0x7fdbc8100a70 2024-03-27T06:43:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbb77fe640 1 -- 172.21.15.16:0/1382762486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdbb80035f0 con 0x7fdbc8100a70 2024-03-27T06:43:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbccbe2640 1 -- 172.21.15.16:0/1382762486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc81a1f80 con 0x7fdbc8100a70 2024-03-27T06:43:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbb77fe640 1 -- 172.21.15.16:0/1382762486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8010690 con 0x7fdbc8100a70 2024-03-27T06:43:25.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.431+0000 7fdbccbe2640 1 -- 172.21.15.16:0/1382762486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc81a2390 con 0x7fdbc8100a70 2024-03-27T06:43:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.432+0000 7fdbccbe2640 1 -- 172.21.15.16:0/1382762486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc8107cc0 con 0x7fdbc8100a70 2024-03-27T06:43:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.433+0000 7fdbb77fe640 1 -- 172.21.15.16:0/1382762486 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138370+0+0 (secure 0 0 0) 0x7fdbb80037a0 con 0x7fdbc8100a70 2024-03-27T06:43:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.434+0000 7fdbb77fe640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdbb00a36b0 0x7fdbb00a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.434+0000 7fdbc5d9b640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdbb00a36b0 0x7fdbb00a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.434+0000 7fdbb77fe640 1 -- 172.21.15.16:0/1382762486 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fdbb80d0190 con 0x7fdbc8100a70 2024-03-27T06:43:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.434+0000 7fdbc5d9b640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdbb00a36b0 0x7fdbb00a5b70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdbc810b160 tx=0x7fdbbc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:25.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.438+0000 7fdbb77fe640 1 -- 172.21.15.16:0/1382762486 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fdbb809cd70 con 0x7fdbc8100a70 2024-03-27T06:43:25.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:25.678+0000 7fdbccbe2640 1 -- 172.21.15.16:0/1382762486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fdbc802d370 con 0x7fdbc8100a70 2024-03-27T06:43:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:25 smithi170 ceph-mon[36438]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:43:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:25 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2257265683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:25.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:25 smithi016 ceph-mon[31908]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:43:25.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:25 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2257265683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:25.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:25 smithi156 ceph-mon[36261]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:43:25.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:25 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2257265683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:26.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.509+0000 7fdbb77fe640 1 -- 172.21.15.16:0/1382762486 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fdbb809c670 con 0x7fdbc8100a70 2024-03-27T06:43:26.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.512+0000 7fdbccbe2640 1 -- 172.21.15.16:0/1382762486 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdbb00a36b0 msgr2=0x7fdbb00a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:26.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.512+0000 7fdbccbe2640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdbb00a36b0 0x7fdbb00a5b70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdbc810b160 tx=0x7fdbbc005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:26.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.512+0000 7fdbccbe2640 1 -- 172.21.15.16:0/1382762486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8100a70 msgr2=0x7fdbc810b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:26.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.512+0000 7fdbccbe2640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8100a70 0x7fdbc810b3f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800e9c0 tx=0x7fdbb800ee80 comp rx=0 tx=0).stop 2024-03-27T06:43:26.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.512+0000 7fdbccbe2640 1 -- 172.21.15.16:0/1382762486 shutdown_connections 2024-03-27T06:43:26.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.512+0000 7fdbccbe2640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fdbb00a36b0 0x7fdbb00a5b70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.512+0000 7fdbccbe2640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc8102620 0x7fdbc8109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.512+0000 7fdbccbe2640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbc8101c60 0x7fdbc8109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.512+0000 7fdbccbe2640 1 --2- 172.21.15.16:0/1382762486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8100a70 0x7fdbc810b3f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.512+0000 7fdbccbe2640 1 -- 172.21.15.16:0/1382762486 >> 172.21.15.16:0/1382762486 conn(0x7fdbc80fbfe0 msgr2=0x7fdbc8104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:26.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.513+0000 7fdbccbe2640 1 -- 172.21.15.16:0/1382762486 shutdown_connections 2024-03-27T06:43:26.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.513+0000 7fdbccbe2640 1 -- 172.21.15.16:0/1382762486 wait complete. 2024-03-27T06:43:26.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-27T06:43:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.645+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/145941650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a38104fc0 msgr2=0x7f8a381073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.645+0000 7f8a3f2a4640 1 --2- 172.21.15.16:0/145941650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a38104fc0 0x7f8a381073b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009900 tx=0x7f8a3402bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.645+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/145941650 shutdown_connections 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.645+0000 7f8a3f2a4640 1 --2- 172.21.15.16:0/145941650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a381078f0 0x7f8a38109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.645+0000 7f8a3f2a4640 1 --2- 172.21.15.16:0/145941650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a38104fc0 0x7f8a381073b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.645+0000 7f8a3f2a4640 1 --2- 172.21.15.16:0/145941650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a38102690 0x7f8a38104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.645+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/145941650 >> 172.21.15.16:0/145941650 conn(0x7f8a380fbfe0 msgr2=0x7f8a380fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.645+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/145941650 shutdown_connections 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.645+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/145941650 wait complete. 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.645+0000 7f8a3f2a4640 1 Processor -- start 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.645+0000 7f8a3f2a4640 1 -- start start 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3f2a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a38102690 0x7f8a38197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3f2a4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a38104fc0 0x7f8a38198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3f2a4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a381078f0 0x7f8a3819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3f2a4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3810d8f0 con 0x7f8a38102690 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3f2a4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3810d770 con 0x7f8a38104fc0 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3f2a4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3810da70 con 0x7f8a381078f0 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3d040640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a38102690 0x7f8a38197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3d040640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a38102690 0x7f8a38197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35642/0 (socket says 172.21.15.16:35642) 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3d040640 1 -- 172.21.15.16:0/3681472989 learned_addr learned my addr 172.21.15.16:0/3681472989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3c83f640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a38104fc0 0x7f8a38198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3d841640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a381078f0 0x7f8a3819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3d040640 1 -- 172.21.15.16:0/3681472989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a381078f0 msgr2=0x7f8a3819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3d040640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a381078f0 0x7f8a3819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3d040640 1 -- 172.21.15.16:0/3681472989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a38104fc0 msgr2=0x7f8a38198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3d040640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a38104fc0 0x7f8a38198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3d040640 1 -- 172.21.15.16:0/3681472989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3819fb30 con 0x7f8a38102690 2024-03-27T06:43:26.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3d040640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a38102690 0x7f8a38197f10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800e9f0 tx=0x7f8a2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:26.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a2e7fc640 1 -- 172.21.15.16:0/3681472989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2800cd80 con 0x7f8a38102690 2024-03-27T06:43:26.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a2e7fc640 1 -- 172.21.15.16:0/3681472989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a28005520 con 0x7f8a38102690 2024-03-27T06:43:26.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/3681472989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3819fe20 con 0x7f8a38102690 2024-03-27T06:43:26.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a2e7fc640 1 -- 172.21.15.16:0/3681472989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28010690 con 0x7f8a38102690 2024-03-27T06:43:26.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.646+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/3681472989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a381a0280 con 0x7f8a38102690 2024-03-27T06:43:26.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.647+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/3681472989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a380fdad0 con 0x7f8a38102690 2024-03-27T06:43:26.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.648+0000 7f8a2e7fc640 1 -- 172.21.15.16:0/3681472989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 138387+0+0 (secure 0 0 0) 0x7f8a280046c0 con 0x7f8a38102690 2024-03-27T06:43:26.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.649+0000 7f8a2e7fc640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8a240a3780 0x7f8a240a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:26.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.649+0000 7f8a3c83f640 1 -- 172.21.15.16:0/3681472989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8a240a3780 msgr2=0x7f8a240a5c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4198806087 2024-03-27T06:43:26.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.649+0000 7f8a2e7fc640 1 -- 172.21.15.16:0/3681472989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8a280d00d0 con 0x7f8a38102690 2024-03-27T06:43:26.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.649+0000 7f8a3c83f640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8a240a3780 0x7f8a240a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:43:26.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.654+0000 7f8a2e7fc640 1 -- 172.21.15.16:0/3681472989 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f8a2809ccb0 con 0x7f8a38102690 2024-03-27T06:43:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.842+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/3681472989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f8a3802d370 con 0x7f8a38102690 2024-03-27T06:43:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.846+0000 7f8a2e7fc640 1 -- 172.21.15.16:0/3681472989 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) ==== 138+0+0 (secure 0 0 0) 0x7f8a2809c5b0 con 0x7f8a38102690 2024-03-27T06:43:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/3681472989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8a240a3780 msgr2=0x7f8a240a5c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:43:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8a240a3780 0x7f8a240a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/3681472989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a38102690 msgr2=0x7f8a38197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a38102690 0x7f8a38197f10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800e9f0 tx=0x7f8a2800edf0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/3681472989 shutdown_connections 2024-03-27T06:43:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f8a240a3780 0x7f8a240a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a381078f0 0x7f8a3819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a38104fc0 0x7f8a38198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 --2- 172.21.15.16:0/3681472989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a38102690 0x7f8a38197f10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/3681472989 >> 172.21.15.16:0/3681472989 conn(0x7f8a380fbfe0 msgr2=0x7f8a38103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/3681472989 shutdown_connections 2024-03-27T06:43:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.848+0000 7f8a3f2a4640 1 -- 172.21.15.16:0/3681472989 wait complete. 2024-03-27T06:43:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:26 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1382762486' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-27T06:43:26.844 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: ignoring --setuser ceph since I am not root 2024-03-27T06:43:26.844 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: ignoring --setgroup ceph since I am not root 2024-03-27T06:43:26.844 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:26 smithi170 ceph-mgr[38238]: -- 172.21.15.170:0/3040139041 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b19c2f45a0 con 0x55b19c2c8800 2024-03-27T06:43:26.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:26 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1382762486' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-27T06:43:26.853 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: ignoring --setuser ceph since I am not root 2024-03-27T06:43:26.853 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: ignoring --setgroup ceph since I am not root 2024-03-27T06:43:26.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-27T06:43:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.964+0000 7fd9881bf640 1 -- 172.21.15.16:0/3160253406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd980107850 msgr2=0x7fd980109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.964+0000 7fd9881bf640 1 --2- 172.21.15.16:0/3160253406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd980107850 0x7fd980109ca0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd978009ca0 tx=0x7fd97802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.964+0000 7fd9881bf640 1 -- 172.21.15.16:0/3160253406 shutdown_connections 2024-03-27T06:43:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.964+0000 7fd9881bf640 1 --2- 172.21.15.16:0/3160253406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd980107850 0x7fd980109ca0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.964+0000 7fd9881bf640 1 --2- 172.21.15.16:0/3160253406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd980104f20 0x7fd980107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.964+0000 7fd9881bf640 1 --2- 172.21.15.16:0/3160253406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9801025f0 0x7fd9801049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.964+0000 7fd9881bf640 1 -- 172.21.15.16:0/3160253406 >> 172.21.15.16:0/3160253406 conn(0x7fd9800fbfe0 msgr2=0x7fd9800fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.964+0000 7fd9881bf640 1 -- 172.21.15.16:0/3160253406 shutdown_connections 2024-03-27T06:43:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.964+0000 7fd9881bf640 1 -- 172.21.15.16:0/3160253406 wait complete. 2024-03-27T06:43:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd9881bf640 1 Processor -- start 2024-03-27T06:43:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd9881bf640 1 -- start start 2024-03-27T06:43:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd9881bf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9801025f0 0x7fd980197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd9881bf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd980104f20 0x7fd9801983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd9881bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd980107850 0x7fd98019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd9881bf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd98010d940 con 0x7fd980107850 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd9881bf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd98010d7c0 con 0x7fd9801025f0 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd9881bf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd98010dac0 con 0x7fd980104f20 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd98675c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd980107850 0x7fd98019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd985f5b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9801025f0 0x7fd980197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd98675c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd980107850 0x7fd98019f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35658/0 (socket says 172.21.15.16:35658) 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd98675c640 1 -- 172.21.15.16:0/1555854738 learned_addr learned my addr 172.21.15.16:0/1555854738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd98575a640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd980104f20 0x7fd9801983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd98675c640 1 -- 172.21.15.16:0/1555854738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd980104f20 msgr2=0x7fd9801983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd98675c640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd980104f20 0x7fd9801983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd98675c640 1 -- 172.21.15.16:0/1555854738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9801025f0 msgr2=0x7fd980197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd98675c640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9801025f0 0x7fd980197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd98675c640 1 -- 172.21.15.16:0/1555854738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98019fab0 con 0x7fd980107850 2024-03-27T06:43:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd98575a640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd980104f20 0x7fd9801983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:43:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.965+0000 7fd98675c640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd980107850 0x7fd98019f440 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd97802c700 tx=0x7fd9780346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.966+0000 7fd976ffd640 1 -- 172.21.15.16:0/1555854738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978039040 con 0x7fd980107850 2024-03-27T06:43:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.966+0000 7fd9881bf640 1 -- 172.21.15.16:0/1555854738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98019fd40 con 0x7fd980107850 2024-03-27T06:43:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.966+0000 7fd976ffd640 1 -- 172.21.15.16:0/1555854738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd978044070 con 0x7fd980107850 2024-03-27T06:43:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.966+0000 7fd976ffd640 1 -- 172.21.15.16:0/1555854738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9780043c0 con 0x7fd980107850 2024-03-27T06:43:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.966+0000 7fd9881bf640 1 -- 172.21.15.16:0/1555854738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9801a01d0 con 0x7fd980107850 2024-03-27T06:43:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.967+0000 7fd9881bf640 1 -- 172.21.15.16:0/1555854738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9800fdad0 con 0x7fd980107850 2024-03-27T06:43:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.968+0000 7fd976ffd640 1 -- 172.21.15.16:0/1555854738 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 138387+0+0 (secure 0 0 0) 0x7fd978004580 con 0x7fd980107850 2024-03-27T06:43:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.968+0000 7fd976ffd640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd96c0a3700 0x7fd96c0a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.968+0000 7fd985f5b640 1 -- 172.21.15.16:0/1555854738 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd96c0a3700 msgr2=0x7fd96c0a5bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4198806087 2024-03-27T06:43:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.968+0000 7fd976ffd640 1 -- 172.21.15.16:0/1555854738 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd9780f2430 con 0x7fd980107850 2024-03-27T06:43:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.968+0000 7fd985f5b640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd96c0a3700 0x7fd96c0a5bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:43:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:26.973+0000 7fd976ffd640 1 -- 172.21.15.16:0/1555854738 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd9780bf010 con 0x7fd980107850 2024-03-27T06:43:26.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:26 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1382762486' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-27T06:43:26.972 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: ignoring --setuser ceph since I am not root 2024-03-27T06:43:26.972 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: ignoring --setgroup ceph since I am not root 2024-03-27T06:43:26.972 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:26 smithi156 ceph-mgr[37650]: -- 172.21.15.156:0/1961972319 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ccc81c25a0 con 0x55ccc8196c00 2024-03-27T06:43:27.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.160+0000 7fd9881bf640 1 -- 172.21.15.16:0/1555854738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fd9800627c0 con 0x7fd980107850 2024-03-27T06:43:27.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.164+0000 7fd976ffd640 1 -- 172.21.15.16:0/1555854738 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7fd9780be910 con 0x7fd980107850 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 -- 172.21.15.16:0/1555854738 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd96c0a3700 msgr2=0x7fd96c0a5bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd96c0a3700 0x7fd96c0a5bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 -- 172.21.15.16:0/1555854738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd980107850 msgr2=0x7fd98019f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd980107850 0x7fd98019f440 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd97802c700 tx=0x7fd9780346d0 comp rx=0 tx=0).stop 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 -- 172.21.15.16:0/1555854738 shutdown_connections 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd96c0a3700 0x7fd96c0a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd980107850 0x7fd98019f440 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd980104f20 0x7fd9801983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 --2- 172.21.15.16:0/1555854738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9801025f0 0x7fd980197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 -- 172.21.15.16:0/1555854738 >> 172.21.15.16:0/1555854738 conn(0x7fd9800fbfe0 msgr2=0x7fd980102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 -- 172.21.15.16:0/1555854738 shutdown_connections 2024-03-27T06:43:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:27.166+0000 7fd9881bf640 1 -- 172.21.15.16:0/1555854738 wait complete. 2024-03-27T06:43:27.172 INFO:teuthology.run_tasks:Running task kclient... 2024-03-27T06:43:27.201 INFO:tasks.kclient:Mounting kernel clients... 2024-03-27T06:43:27.201 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-27T06:43:27.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:43:27.201 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-03-27T06:43:27.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:43:27.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-03-27T06:43:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-03-27T06:43:27.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:43:27.288 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'rbytes': False, 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-27T06:43:27.288 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-27T06:43:27.288 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-27T06:43:27.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-27T06:43:27.361 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:27.115+0000 7f7105e2f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T06:43:27.361 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:27.213+0000 7f7105e2f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T06:43:27.505 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:27 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:27.227+0000 7f292459a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T06:43:27.505 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:27 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:27.343+0000 7f292459a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T06:43:27.505 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:27 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:27.228+0000 7fdea0dbe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-27T06:43:27.506 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:27 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:27.322+0000 7fdea0dbe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-27T06:43:27.581 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:43:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:27 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1382762486' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-27T06:43:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:27 smithi170 ceph-mon[36438]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-27T06:43:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:27 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3681472989' entity='client.admin' 2024-03-27T06:43:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:27 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1555854738' entity='client.admin' 2024-03-27T06:43:27.846 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:27 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1382762486' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-27T06:43:27.846 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:27 smithi016 ceph-mon[31908]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-27T06:43:27.846 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:27 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3681472989' entity='client.admin' 2024-03-27T06:43:27.846 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:27 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1555854738' entity='client.admin' 2024-03-27T06:43:27.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:27 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1382762486' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-27T06:43:27.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:27 smithi156 ceph-mon[36261]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-27T06:43:27.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:27 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3681472989' entity='client.admin' 2024-03-27T06:43:27.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:27 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1555854738' entity='client.admin' 2024-03-27T06:43:28.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:27.897+0000 7f7105e2f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T06:43:28.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:27.993+0000 7f7105e2f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T06:43:28.344 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:28.075+0000 7f292459a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T06:43:28.344 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:28.188+0000 7f292459a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T06:43:28.461 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:28 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:28.149+0000 7fdea0dbe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-27T06:43:28.461 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:28 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:28.287+0000 7fdea0dbe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-27T06:43:28.601 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:28.148+0000 7f7105e2f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T06:43:28.601 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:28.246+0000 7f7105e2f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T06:43:28.601 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:28.332+0000 7f7105e2f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T06:43:28.722 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:28 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:28.466+0000 7fdea0dbe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T06:43:28.722 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:28 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:28.601+0000 7fdea0dbe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T06:43:28.722 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:28 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:28.686+0000 7fdea0dbe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T06:43:28.844 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:28.364+0000 7f292459a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-27T06:43:28.844 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:28.502+0000 7f292459a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-27T06:43:28.844 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:28.590+0000 7f292459a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-27T06:43:28.861 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:28.606+0000 7f7105e2f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T06:43:28.861 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:28.719+0000 7f7105e2f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T06:43:28.861 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:28.809+0000 7f7105e2f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T06:43:29.159 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:28.867+0000 7f292459a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T06:43:29.159 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:28.978+0000 7f292459a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T06:43:29.159 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:29 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:29.066+0000 7f292459a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T06:43:29.227 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:28.907+0000 7f7105e2f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T06:43:29.227 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:29 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:29.092+0000 7f7105e2f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T06:43:29.306 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:29 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:29.016+0000 7fdea0dbe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-27T06:43:29.306 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:29 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:29.183+0000 7fdea0dbe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-27T06:43:29.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.511+0000 7f8622573700 1 -- 172.21.15.16:0/276480307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f861c11be20 msgr2=0x7f861c11e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:29.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.511+0000 7f8622573700 1 --2- 172.21.15.16:0/276480307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f861c11be20 0x7f861c11e210 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f860c009990 tx=0x7f860c009ca0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.511+0000 7f8622573700 1 -- 172.21.15.16:0/276480307 shutdown_connections 2024-03-27T06:43:29.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.511+0000 7f8622573700 1 --2- 172.21.15.16:0/276480307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f861c11be20 0x7f861c11e210 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.511+0000 7f8622573700 1 --2- 172.21.15.16:0/276480307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c110da0 0x7f861c109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.511+0000 7f8622573700 1 --2- 172.21.15.16:0/276480307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f861c111210 0x7f861c109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.511+0000 7f8622573700 1 -- 172.21.15.16:0/276480307 >> 172.21.15.16:0/276480307 conn(0x7f861c06d780 msgr2=0x7f861c06db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:29.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.511+0000 7f8622573700 1 -- 172.21.15.16:0/276480307 shutdown_connections 2024-03-27T06:43:29.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.511+0000 7f8622573700 1 -- 172.21.15.16:0/276480307 wait complete. 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.512+0000 7f8622573700 1 Processor -- start 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.512+0000 7f8622573700 1 -- start start 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.512+0000 7f8622573700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f861c110da0 0x7f861c11a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.512+0000 7f8622573700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c11be20 0x7f861c11a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.512+0000 7f8622573700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f861c1133b0 0x7f861c113830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.513+0000 7f8622573700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f861c121900 con 0x7f861c11be20 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.513+0000 7f8622573700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f861c121780 con 0x7f861c1133b0 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.513+0000 7f8622573700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f861c121a80 con 0x7f861c110da0 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.513+0000 7f8620b10700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f861c1133b0 0x7f861c113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.513+0000 7f8620b10700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f861c1133b0 0x7f861c113830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:39980/0 (socket says 172.21.15.16:39980) 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.513+0000 7f8620b10700 1 -- 172.21.15.16:0/4021897979 learned_addr learned my addr 172.21.15.16:0/4021897979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.513+0000 7f861bfff700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f861c110da0 0x7f861c11a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.513+0000 7f861b7fe700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c11be20 0x7f861c11a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:29.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.514+0000 7f861b7fe700 1 -- 172.21.15.16:0/4021897979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f861c110da0 msgr2=0x7f861c11a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:29.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.514+0000 7f861b7fe700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f861c110da0 0x7f861c11a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.514+0000 7f861b7fe700 1 -- 172.21.15.16:0/4021897979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f861c1133b0 msgr2=0x7f861c113830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:29.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.514+0000 7f861b7fe700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f861c1133b0 0x7f861c113830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.514+0000 7f861b7fe700 1 -- 172.21.15.16:0/4021897979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c114130 con 0x7f861c11be20 2024-03-27T06:43:29.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.514+0000 7f861b7fe700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c11be20 0x7f861c11a7c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f860400e9c0 tx=0x7f860400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:29.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.514+0000 7f86197fa700 1 -- 172.21.15.16:0/4021897979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860400cbd0 con 0x7f861c11be20 2024-03-27T06:43:29.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.514+0000 7f8622573700 1 -- 172.21.15.16:0/4021897979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861c117e70 con 0x7f861c11be20 2024-03-27T06:43:29.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.514+0000 7f8622573700 1 -- 172.21.15.16:0/4021897979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861c118400 con 0x7f861c11be20 2024-03-27T06:43:29.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.515+0000 7f86197fa700 1 -- 172.21.15.16:0/4021897979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f860400cd70 con 0x7f861c11be20 2024-03-27T06:43:29.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.515+0000 7f86197fa700 1 -- 172.21.15.16:0/4021897979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604026060 con 0x7f861c11be20 2024-03-27T06:43:29.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.516+0000 7f8622573700 1 -- 172.21.15.16:0/4021897979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85fc005330 con 0x7f861c11be20 2024-03-27T06:43:29.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.518+0000 7f86197fa700 1 -- 172.21.15.16:0/4021897979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 138387+0+0 (secure 0 0 0) 0x7f8604010430 con 0x7f861c11be20 2024-03-27T06:43:29.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.519+0000 7f86197fa700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86080a3ae0 0x7f86080a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:29.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.519+0000 7f86197fa700 1 -- 172.21.15.16:0/4021897979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f860400c9d0 con 0x7f861c11be20 2024-03-27T06:43:29.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.520+0000 7f861bfff700 1 -- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86080a3ae0 msgr2=0x7f86080a5fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4198806087 2024-03-27T06:43:29.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.521+0000 7f861bfff700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86080a3ae0 0x7f86080a5fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:43:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.529+0000 7f86197fa700 1 -- 172.21.15.16:0/4021897979 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f860409c510 con 0x7f861c11be20 2024-03-27T06:43:29.594 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:29 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:29.162+0000 7f292459a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T06:43:29.594 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:29 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:29.376+0000 7f292459a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T06:43:29.650 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:29 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:29.311+0000 7fdea0dbe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-27T06:43:29.650 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:29 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:29.441+0000 7fdea0dbe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-27T06:43:29.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.722+0000 7f861bfff700 1 -- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86080a3ae0 msgr2=0x7f86080a5fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4198806087 2024-03-27T06:43:29.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.722+0000 7f861bfff700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86080a3ae0 0x7f86080a5fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-27T06:43:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.785+0000 7f8622573700 1 -- 172.21.15.16:0/4021897979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f85fc0058f0 con 0x7f861c11be20 2024-03-27T06:43:29.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.785+0000 7f86197fa700 1 -- 172.21.15.16:0/4021897979 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f86040a01c0 con 0x7f861c11be20 2024-03-27T06:43:29.780 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-27T06:43:29.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.787+0000 7f8622573700 1 -- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86080a3ae0 msgr2=0x7f86080a5fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:43:29.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.787+0000 7f8622573700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86080a3ae0 0x7f86080a5fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.787+0000 7f8622573700 1 -- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c11be20 msgr2=0x7f861c11a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:29.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.787+0000 7f8622573700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c11be20 0x7f861c11a7c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f860400e9c0 tx=0x7f860400ecd0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.788+0000 7f8622573700 1 -- 172.21.15.16:0/4021897979 shutdown_connections 2024-03-27T06:43:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.788+0000 7f8622573700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f861c1133b0 0x7f861c113830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.788+0000 7f8622573700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7f86080a3ae0 0x7f86080a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.788+0000 7f8622573700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f861c110da0 0x7f861c11a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.788+0000 7f8622573700 1 --2- 172.21.15.16:0/4021897979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c11be20 0x7f861c11a7c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.788+0000 7f8622573700 1 -- 172.21.15.16:0/4021897979 >> 172.21.15.16:0/4021897979 conn(0x7f861c06d780 msgr2=0x7f861c0711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.788+0000 7f8622573700 1 -- 172.21.15.16:0/4021897979 shutdown_connections 2024-03-27T06:43:29.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:29.788+0000 7f8622573700 1 -- 172.21.15.16:0/4021897979 wait complete. 2024-03-27T06:43:29.972 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:29 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:29.654+0000 7fdea0dbe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-27T06:43:29.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:29 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/4021897979' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-27T06:43:30.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:29 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/4021897979' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-27T06:43:30.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:29 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/4021897979' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-27T06:43:30.576 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:30.125+0000 7f7105e2f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T06:43:30.576 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:30.261+0000 7f7105e2f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T06:43:30.605 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-27T06:43:30.605 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-27T06:43:30.605 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-03-27T06:43:30.605 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-27T06:43:30.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-27T06:43:30.605 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-27T06:43:30.605 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-27T06:43:30.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-27T06:43:30.605 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-27T06:43:30.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:43:30.606 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-03-27T06:43:30.661 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-27T06:43:30.661 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-27T06:43:30.661 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-03-27T06:43:30.661 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-03-27T06:43:30.661 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-27T06:43:30.662 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42131sec preferred_lft 42131sec 2024-03-27T06:43:30.663 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-03-27T06:43:30.663 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-03-27T06:43:30.663 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-27T06:43:30.663 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-03-27T06:43:30.663 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-03-27T06:43:30.663 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-27T06:43:30.663 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-27T06:43:30.664 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-03-27T06:43:30.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-03-27T06:43:30.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-03-27T06:43:30.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-03-27T06:43:30.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-27T06:43:30.664 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-03-27T06:43:30.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-27T06:43:30.831 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:30.606+0000 7f7105e2f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T06:43:30.832 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:30.700+0000 7f7105e2f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T06:43:30.844 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:30 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:30.457+0000 7f292459a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T06:43:30.844 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:30 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:30.589+0000 7f292459a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T06:43:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T06:43:30.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:43:30.887 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-27T06:43:30.915 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-03-27T06:43:30.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:43:30.917 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-03-27T06:43:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-03-27T06:43:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-27T06:43:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-27T06:43:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-27T06:43:30.992 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-27T06:43:30.993 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-27T06:43:30.993 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-03-27T06:43:30.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-27T06:43:30.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-27T06:43:30.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-27T06:43:30.993 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-03-27T06:43:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-27T06:43:31.092 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:30.920+0000 7f7105e2f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:43:31.212 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:30 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:30.929+0000 7f292459a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T06:43:31.213 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:31.020+0000 7f292459a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T06:43:31.213 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:31.217+0000 7f292459a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:43:31.222 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:30 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:30.951+0000 7fdea0dbe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-27T06:43:31.222 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:31.083+0000 7fdea0dbe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-27T06:43:31.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T06:43:31.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:43:31.321 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-03-27T06:43:31.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:43:31.336 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-03-27T06:43:31.361 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:31 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:31.117+0000 7f7105e2f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T06:43:31.376 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-27T06:43:31.377 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-03-27T06:43:31.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-27T06:43:31.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-27T06:43:31.377 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-03-27T06:43:31.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-27T06:43:31.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T06:43:31.508 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-27T06:43:31.508 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-27T06:43:31.508 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-03-27T06:43:31.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-27T06:43:31.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-27T06:43:31.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-27T06:43:31.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-27T06:43:31.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-27T06:43:31.509 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-03-27T06:43:31.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-27T06:43:31.594 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:31.406+0000 7f292459a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T06:43:31.618 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:31 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:31.425+0000 7f7105e2f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:43:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T06:43:31.658 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-27T06:43:31.658 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-03-27T06:43:31.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-03-27T06:43:31.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-03-27T06:43:31.659 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-03-27T06:43:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-27T06:43:31.722 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:31.439+0000 7fdea0dbe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-27T06:43:31.722 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:31.532+0000 7fdea0dbe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-27T06:43:31.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T06:43:31.742 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-27T06:43:31.742 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-27T06:43:31.756 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-27T06:43:31.757 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-27T06:43:31.757 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-27T06:43:31.811 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-27T06:43:31.811 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-27T06:43:31.811 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a 2024-03-27T06:43:31.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:43:31.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-27T06:43:31.875 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:31 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:31.717+0000 7f7105e2f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T06:43:31.896 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-27T06:43:31.896 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-27T06:43:31.896 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-03-27T06:43:31.981 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:31.710+0000 7f292459a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:43:31.981 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:31.986+0000 7f292459a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T06:43:32.049 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-27T06:43:32.049 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-27T06:43:32.050 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-03-27T06:43:32.050 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.156:6789,172.21.15.170:6789" 2024-03-27T06:43:32.050 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.156:6789,172.21.15.170:6789:/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a 2024-03-27T06:43:32.050 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=c9095bc4-ec03-11ee-95d0-87774f69a715" will pass to kernel 2024-03-27T06:43:32.050 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-27T06:43:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-03-27T06:43:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-27T06:43:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-03-27T06:43:32.050 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-27T06:43:32.052 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-27T06:43:32.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:43:32.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-27T06:43:32.123 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-27T06:43:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-27T06:43:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T06:43:32.221 INFO:teuthology.orchestra.run.smithi016.stdout:client25223 2024-03-27T06:43:32.221 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:43:32.221 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-27T06:43:32.222 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:31.764+0000 7fdea0dbe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:43:32.222 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:31.954+0000 7fdea0dbe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-27T06:43:32.234 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:31 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:31.977+0000 7f7105e2f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T06:43:32.238 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-27T06:43:32.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-27T06:43:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T06:43:32.336 INFO:teuthology.orchestra.run.smithi016.stdout:client25223 2024-03-27T06:43:32.336 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-27T06:43:32.344 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:32.228+0000 7f292459a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T06:43:32.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-27T06:43:32.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T06:43:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:client25223 2024-03-27T06:43:32.393 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715.client25223/status') 2024-03-27T06:43:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-27T06:43:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.25223 (3)172.21.15.16:0/1870090402 2024-03-27T06:43:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-03-27T06:43:32.455 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-03-27T06:43:32.455 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-03-27T06:43:32.455 INFO:teuthology.orchestra.run.smithi016.stderr:71 bytes copied, 2.0703e-05 s, 3.4 MB/s 2024-03-27T06:43:32.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-27T06:43:32.462 INFO:teuthology.run_tasks:Running task exec... 2024-03-27T06:43:32.473 INFO:teuthology.task.exec:Executing custom commands... 2024-03-27T06:43:32.473 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-03-27T06:43:32.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-27T06:43:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.586+0000 7fafb417a640 1 -- 172.21.15.16:0/2950200600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafac102650 msgr2=0x7fafac104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.586+0000 7fafb417a640 1 --2- 172.21.15.16:0/2950200600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafac102650 0x7fafac104a40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009900 tx=0x7fafa802bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.586+0000 7fafb417a640 1 -- 172.21.15.16:0/2950200600 shutdown_connections 2024-03-27T06:43:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.586+0000 7fafb417a640 1 --2- 172.21.15.16:0/2950200600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafac1078b0 0x7fafac109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.586+0000 7fafb417a640 1 --2- 172.21.15.16:0/2950200600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac104f80 0x7fafac107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.586+0000 7fafb417a640 1 --2- 172.21.15.16:0/2950200600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafac102650 0x7fafac104a40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.586+0000 7fafb417a640 1 -- 172.21.15.16:0/2950200600 >> 172.21.15.16:0/2950200600 conn(0x7fafac0fbfe0 msgr2=0x7fafac0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.586+0000 7fafb417a640 1 -- 172.21.15.16:0/2950200600 shutdown_connections 2024-03-27T06:43:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.586+0000 7fafb417a640 1 -- 172.21.15.16:0/2950200600 wait complete. 2024-03-27T06:43:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb417a640 1 Processor -- start 2024-03-27T06:43:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb417a640 1 -- start start 2024-03-27T06:43:32.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb417a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafac102650 0x7fafac197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb417a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafac104f80 0x7fafac198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb417a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac1078b0 0x7fafac19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb417a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafac10d8d0 con 0x7fafac104f80 2024-03-27T06:43:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb417a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fafac10d750 con 0x7fafac1078b0 2024-03-27T06:43:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb417a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fafac10da50 con 0x7fafac102650 2024-03-27T06:43:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb1715640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafac104f80 0x7fafac198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb1715640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafac104f80 0x7fafac198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35688/0 (socket says 172.21.15.16:35688) 2024-03-27T06:43:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb1715640 1 -- 172.21.15.16:0/3894021781 learned_addr learned my addr 172.21.15.16:0/3894021781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb1f16640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafac102650 0x7fafac197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb2717640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac1078b0 0x7fafac19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb1715640 1 -- 172.21.15.16:0/3894021781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafac102650 msgr2=0x7fafac197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb1715640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafac102650 0x7fafac197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb1715640 1 -- 172.21.15.16:0/3894021781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac1078b0 msgr2=0x7fafac19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb1715640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac1078b0 0x7fafac19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb1715640 1 -- 172.21.15.16:0/3894021781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac19fb40 con 0x7fafac104f80 2024-03-27T06:43:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.587+0000 7fafb1715640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafac104f80 0x7fafac198460 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fafa000e9c0 tx=0x7fafa000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.588+0000 7faf9effd640 1 -- 172.21.15.16:0/3894021781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa000ccc0 con 0x7fafac104f80 2024-03-27T06:43:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.588+0000 7faf9effd640 1 -- 172.21.15.16:0/3894021781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fafa0005520 con 0x7fafac104f80 2024-03-27T06:43:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.588+0000 7faf9effd640 1 -- 172.21.15.16:0/3894021781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0010690 con 0x7fafac104f80 2024-03-27T06:43:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.588+0000 7fafb417a640 1 -- 172.21.15.16:0/3894021781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafac19fe30 con 0x7fafac104f80 2024-03-27T06:43:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.588+0000 7fafb417a640 1 -- 172.21.15.16:0/3894021781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafac1a0340 con 0x7fafac104f80 2024-03-27T06:43:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.589+0000 7fafb417a640 1 -- 172.21.15.16:0/3894021781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafac0fdad0 con 0x7fafac104f80 2024-03-27T06:43:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.590+0000 7faf9effd640 1 -- 172.21.15.16:0/3894021781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 138387+0+0 (secure 0 0 0) 0x7fafa0004e40 con 0x7fafac104f80 2024-03-27T06:43:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.590+0000 7faf9effd640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf980a3780 0x7faf980a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.591+0000 7fafb1f16640 1 -- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf980a3780 msgr2=0x7faf980a5c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4198806087 2024-03-27T06:43:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.591+0000 7faf9effd640 1 -- 172.21.15.16:0/3894021781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fafa00d0550 con 0x7fafac104f80 2024-03-27T06:43:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.591+0000 7fafb1f16640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf980a3780 0x7faf980a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:43:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.595+0000 7faf9effd640 1 -- 172.21.15.16:0/3894021781 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fafa009d130 con 0x7fafac104f80 2024-03-27T06:43:32.609 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:32 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:32.281+0000 7fdea0dbe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-27T06:43:32.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.792+0000 7fafb1f16640 1 -- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf980a3780 msgr2=0x7faf980a5c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4198806087 2024-03-27T06:43:32.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.792+0000 7fafb1f16640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf980a3780 0x7faf980a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-27T06:43:32.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.823+0000 7fafb417a640 1 -- 172.21.15.16:0/3894021781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fafac02d370 con 0x7fafac104f80 2024-03-27T06:43:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.875+0000 7faf9effd640 1 -- 172.21.15.16:0/3894021781 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7fafa009ca30 con 0x7fafac104f80 2024-03-27T06:43:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 -- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf980a3780 msgr2=0x7faf980a5c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:43:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf980a3780 0x7faf980a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 -- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafac104f80 msgr2=0x7fafac198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafac104f80 0x7fafac198460 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fafa000e9c0 tx=0x7fafa000edc0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 -- 172.21.15.16:0/3894021781 shutdown_connections 2024-03-27T06:43:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7faf980a3780 0x7faf980a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafac1078b0 0x7fafac19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafac104f80 0x7fafac198460 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 --2- 172.21.15.16:0/3894021781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafac102650 0x7fafac197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 -- 172.21.15.16:0/3894021781 >> 172.21.15.16:0/3894021781 conn(0x7fafac0fbfe0 msgr2=0x7fafac102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 -- 172.21.15.16:0/3894021781 shutdown_connections 2024-03-27T06:43:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:32.877+0000 7fafb417a640 1 -- 172.21.15.16:0/3894021781 wait complete. 2024-03-27T06:43:32.875 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:32 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:32.614+0000 7fdea0dbe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-27T06:43:32.875 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:32 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:32.860+0000 7fdea0dbe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-27T06:43:32.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-27T06:43:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.001+0000 7fd7b1871640 1 -- 172.21.15.16:0/409356406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ac100a80 msgr2=0x7fd7ac100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.001+0000 7fd7b1871640 1 --2- 172.21.15.16:0/409356406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ac100a80 0x7fd7ac100ea0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009900 tx=0x7fd79c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.001+0000 7fd7b1871640 1 -- 172.21.15.16:0/409356406 shutdown_connections 2024-03-27T06:43:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.001+0000 7fd7b1871640 1 --2- 172.21.15.16:0/409356406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac102630 0x7fd7ac108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.001+0000 7fd7b1871640 1 --2- 172.21.15.16:0/409356406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac101c70 0x7fd7ac1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.001+0000 7fd7b1871640 1 --2- 172.21.15.16:0/409356406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ac100a80 0x7fd7ac100ea0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.001+0000 7fd7b1871640 1 -- 172.21.15.16:0/409356406 >> 172.21.15.16:0/409356406 conn(0x7fd7ac0fc010 msgr2=0x7fd7ac0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.001+0000 7fd7b1871640 1 -- 172.21.15.16:0/409356406 shutdown_connections 2024-03-27T06:43:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.001+0000 7fd7b1871640 1 -- 172.21.15.16:0/409356406 wait complete. 2024-03-27T06:43:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7b1871640 1 Processor -- start 2024-03-27T06:43:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7b1871640 1 -- start start 2024-03-27T06:43:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7b1871640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac100a80 0x7fd7ac196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7b1871640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ac101c70 0x7fd7ac1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7b1871640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac102630 0x7fd7ac19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7b1871640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac10c7d0 con 0x7fd7ac101c70 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7b1871640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac10c650 con 0x7fd7ac102630 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7b1871640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac10c950 con 0x7fd7ac100a80 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ac101c70 0x7fd7ac1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ac101c70 0x7fd7ac1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35704/0 (socket says 172.21.15.16:35704) 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7a3fff640 1 -- 172.21.15.16:0/3156277678 learned_addr learned my addr 172.21.15.16:0/3156277678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7aaffd640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac100a80 0x7fd7ac196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.002+0000 7fd7ab7fe640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac102630 0x7fd7ac19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.003+0000 7fd7a3fff640 1 -- 172.21.15.16:0/3156277678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac100a80 msgr2=0x7fd7ac196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.003+0000 7fd7a3fff640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac100a80 0x7fd7ac196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.003+0000 7fd7a3fff640 1 -- 172.21.15.16:0/3156277678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac102630 msgr2=0x7fd7ac19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.003+0000 7fd7a3fff640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7ac102630 0x7fd7ac19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.003+0000 7fd7a3fff640 1 -- 172.21.15.16:0/3156277678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac19e9d0 con 0x7fd7ac101c70 2024-03-27T06:43:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.003+0000 7fd7a3fff640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ac101c70 0x7fd7ac1972f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd79400d950 tx=0x7fd79400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.003+0000 7fd7b086f640 1 -- 172.21.15.16:0/3156277678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794017070 con 0x7fd7ac101c70 2024-03-27T06:43:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.003+0000 7fd7b086f640 1 -- 172.21.15.16:0/3156277678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd794009c30 con 0x7fd7ac101c70 2024-03-27T06:43:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.003+0000 7fd7b1871640 1 -- 172.21.15.16:0/3156277678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ac19ecc0 con 0x7fd7ac101c70 2024-03-27T06:43:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.003+0000 7fd7b086f640 1 -- 172.21.15.16:0/3156277678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79400f740 con 0x7fd7ac101c70 2024-03-27T06:43:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.003+0000 7fd7b1871640 1 -- 172.21.15.16:0/3156277678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ac19f1c0 con 0x7fd7ac101c70 2024-03-27T06:43:33.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.004+0000 7fd7b1871640 1 -- 172.21.15.16:0/3156277678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ac107d40 con 0x7fd7ac101c70 2024-03-27T06:43:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.006+0000 7fd7b086f640 1 -- 172.21.15.16:0/3156277678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 138387+0+0 (secure 0 0 0) 0x7fd794010a80 con 0x7fd7ac101c70 2024-03-27T06:43:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.007+0000 7fd7b086f640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd7980a3700 0x7fd7980a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.007+0000 7fd7aaffd640 1 -- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd7980a3700 msgr2=0x7fd7980a5bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4198806087 2024-03-27T06:43:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.007+0000 7fd7b086f640 1 -- 172.21.15.16:0/3156277678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd7940cf910 con 0x7fd7ac101c70 2024-03-27T06:43:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.007+0000 7fd7aaffd640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd7980a3700 0x7fd7980a5bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-27T06:43:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.010+0000 7fd7b086f640 1 -- 172.21.15.16:0/3156277678 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd794009de0 con 0x7fd7ac101c70 2024-03-27T06:43:33.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:32 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3894021781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-27T06:43:33.112 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:32 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:32.844+0000 7f7105e2f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T06:43:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.208+0000 7fd7aaffd640 1 -- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd7980a3700 msgr2=0x7fd7980a5bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4198806087 2024-03-27T06:43:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.208+0000 7fd7aaffd640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd7980a3700 0x7fd7980a5bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-27T06:43:33.203 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:32 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3894021781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-27T06:43:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.225+0000 7fd7b1871640 1 -- 172.21.15.16:0/3156277678 --> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fd7ac0620f0 con 0x7fd7980a3700 2024-03-27T06:43:33.221 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:32 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3894021781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-27T06:43:33.478 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:33.207+0000 7f292459a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T06:43:33.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.609+0000 7fd7aaffd640 1 -- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd7980a3700 msgr2=0x7fd7980a5bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/4198806087 2024-03-27T06:43:33.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.609+0000 7fd7aaffd640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd7980a3700 0x7fd7980a5bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-27T06:43:33.762 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:33.435+0000 7f7105e2f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T06:43:33.762 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:33.538+0000 7f7105e2f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T06:43:33.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.790+0000 7fd7b086f640 1 -- 172.21.15.16:0/3156277678 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 20) ==== 138055+0+0 (secure 0 0 0) 0x7fd794098510 con 0x7fd7ac101c70 2024-03-27T06:43:33.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.791+0000 7fd7b086f640 1 -- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd7980a3700 msgr2=0x7fd7980a5bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-27T06:43:33.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:33.791+0000 7fd7b086f640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/4198806087,v1:172.21.15.16:6801/4198806087] conn(0x7fd7980a3700 0x7fd7980a5bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:34.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3894021781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-27T06:43:34.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:43:34.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: Active manager daemon x restarted 2024-03-27T06:43:34.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: Activating manager daemon x 2024-03-27T06:43:34.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: osdmap e89: 12 total, 12 up, 12 in 2024-03-27T06:43:34.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: mgrmap e20: x(active, starting, since 0.0123465s), standbys: z, y 2024-03-27T06:43:34.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:43:34.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:43:34.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:43:34.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-27T06:43:34.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-27T06:43:34.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T06:43:34.034 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T06:43:34.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T06:43:34.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T06:43:34.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:43:34.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T06:43:34.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T06:43:34.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T06:43:34.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T06:43:34.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T06:43:34.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-27T06:43:34.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-27T06:43:34.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-27T06:43:34.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:43:34.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:43:34.037 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:43:34.037 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:33.767+0000 7f7105e2f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T06:43:34.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:43:34.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:43:34.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:43:34.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:43:34.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:43:34.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:43:34.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:43:34.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:43:34.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:43:34.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-27T06:43:34.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-27T06:43:34.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-27T06:43:34.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: Manager daemon x is now available 2024-03-27T06:43:34.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:33 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:43:34.227 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:33.837+0000 7fdea0dbe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-27T06:43:34.472 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.037+0000 7f706483c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:34.472 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.037+0000 7f706483c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:34.472 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.037+0000 7f706483c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:34.473 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.037+0000 7f706483c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:34.473 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.037+0000 7f706483c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:34.473 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.038+0000 7f7067842700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:34.473 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.038+0000 7f7067842700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:34.473 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.038+0000 7f7067842700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:34.473 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.038+0000 7f7067842700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:34.473 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.038+0000 7f7067842700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:34.473 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:33.916+0000 7f292459a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T06:43:34.473 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:34.052+0000 7f292459a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T06:43:34.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3894021781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-27T06:43:34.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:43:34.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: Active manager daemon x restarted 2024-03-27T06:43:34.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: Activating manager daemon x 2024-03-27T06:43:34.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: osdmap e89: 12 total, 12 up, 12 in 2024-03-27T06:43:34.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: mgrmap e20: x(active, starting, since 0.0123465s), standbys: z, y 2024-03-27T06:43:34.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:43:34.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T06:43:34.475 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-27T06:43:34.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-27T06:43:34.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-27T06:43:34.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:43:34.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:43:34.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:43:34.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:43:34.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:43:34.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:43:34.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:43:34.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:43:34.476 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: Manager daemon x is now available 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:33 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3894021781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: Active manager daemon x restarted 2024-03-27T06:43:34.477 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: Activating manager daemon x 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: osdmap e89: 12 total, 12 up, 12 in 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: mgrmap e20: x(active, starting, since 0.0123465s), standbys: z, y 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:43:34.478 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-27T06:43:34.479 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: Manager daemon x is now available 2024-03-27T06:43:34.480 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:33 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:43:34.583 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:43:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z[38195]: 2024-03-27T06:43:34.305+0000 7f292459a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T06:43:34.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.798+0000 7fd7b086f640 1 -- 172.21.15.16:0/3156277678 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 21) ==== 138182+0+0 (secure 0 0 0) 0x7fd79400ea40 con 0x7fd7ac101c70 2024-03-27T06:43:34.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.798+0000 7fd7b086f640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd798052550 0x7fd798052930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:34.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.798+0000 7fd7b086f640 1 -- 172.21.15.16:0/3156277678 --> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fd7ac0620f0 con 0x7fd798052550 2024-03-27T06:43:34.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.803+0000 7fd7aaffd640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd798052550 0x7fd798052930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:34.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.805+0000 7fd7aaffd640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd798052550 0x7fd798052930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd79c000c00 tx=0x7fd79c005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:34.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.825+0000 7fd7b086f640 1 -- 172.21.15.16:0/3156277678 <== mgr.15354 v2:172.21.15.16:6800/3715976676 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fd7ac0620f0 con 0x7fd798052550 2024-03-27T06:43:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:{} 2024-03-27T06:43:34.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.827+0000 7fd7b1871640 1 -- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd798052550 msgr2=0x7fd798052930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:34.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.827+0000 7fd7b1871640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd798052550 0x7fd798052930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd79c000c00 tx=0x7fd79c005fd0 comp rx=0 tx=0).stop 2024-03-27T06:43:34.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.827+0000 7fd7b1871640 1 -- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ac101c70 msgr2=0x7fd7ac1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:34.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.827+0000 7fd7b1871640 1 --2- 172.21.15.16:0/3156277678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ac101c70 0x7fd7ac1972f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd79400d950 tx=0x7fd79400de10 comp rx=0 tx=0).stop 2024-03-27T06:43:34.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.827+0000 7fd7ab7fe640 1 -- 172.21.15.16:0/3156277678 reap_dead start 2024-03-27T06:43:34.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.828+0000 7fd7b1871640 1 -- 172.21.15.16:0/3156277678 shutdown_connections 2024-03-27T06:43:34.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.828+0000 7fd7b1871640 1 -- 172.21.15.16:0/3156277678 >> 172.21.15.16:0/3156277678 conn(0x7fd7ac0fc010 msgr2=0x7fd7ac104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:34.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.828+0000 7fd7b1871640 1 -- 172.21.15.16:0/3156277678 shutdown_connections 2024-03-27T06:43:34.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.828+0000 7fd7b1871640 1 -- 172.21.15.16:0/3156277678 wait complete. 2024-03-27T06:43:34.834 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-27T06:43:34.868 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-27T06:43:34.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:43:34.882 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:34.615+0000 7fdea0dbe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-27T06:43:34.883 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:34.759+0000 7fdea0dbe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-27T06:43:34.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.992+0000 7feb2800e640 1 -- 172.21.15.16:0/2238930770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb20101c60 msgr2=0x7feb201020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:34.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.992+0000 7feb2800e640 1 --2- 172.21.15.16:0/2238930770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb20101c60 0x7feb201020e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feb10009900 tx=0x7feb1002bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:34.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.992+0000 7feb2800e640 1 -- 172.21.15.16:0/2238930770 shutdown_connections 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.992+0000 7feb2800e640 1 --2- 172.21.15.16:0/2238930770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb20102620 0x7feb20108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.992+0000 7feb2800e640 1 --2- 172.21.15.16:0/2238930770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb20101c60 0x7feb201020e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.992+0000 7feb2800e640 1 --2- 172.21.15.16:0/2238930770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb20100a70 0x7feb20100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.992+0000 7feb2800e640 1 -- 172.21.15.16:0/2238930770 >> 172.21.15.16:0/2238930770 conn(0x7feb200fbfe0 msgr2=0x7feb200fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.992+0000 7feb2800e640 1 -- 172.21.15.16:0/2238930770 shutdown_connections 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.992+0000 7feb2800e640 1 -- 172.21.15.16:0/2238930770 wait complete. 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.992+0000 7feb2800e640 1 Processor -- start 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.992+0000 7feb2800e640 1 -- start start 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb2800e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb20100a70 0x7feb20198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb2800e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb20101c60 0x7feb20198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb2800e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb20102620 0x7feb2019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb2800e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb2010c7d0 con 0x7feb20101c60 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb2800e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7feb2010c650 con 0x7feb20100a70 2024-03-27T06:43:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb2800e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feb2010c950 con 0x7feb20102620 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb255a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb20101c60 0x7feb20198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb255a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb20101c60 0x7feb20198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35866/0 (socket says 172.21.15.16:35866) 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb255a9640 1 -- 172.21.15.16:0/1178939706 learned_addr learned my addr 172.21.15.16:0/1178939706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb25daa640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb20100a70 0x7feb20198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb265ab640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb20102620 0x7feb2019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb255a9640 1 -- 172.21.15.16:0/1178939706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb20102620 msgr2=0x7feb2019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb255a9640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb20102620 0x7feb2019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb255a9640 1 -- 172.21.15.16:0/1178939706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb20100a70 msgr2=0x7feb20198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb255a9640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb20100a70 0x7feb20198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb255a9640 1 -- 172.21.15.16:0/1178939706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2019fbb0 con 0x7feb20101c60 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb255a9640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb20101c60 0x7feb20198540 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feb1002ca30 tx=0x7feb10004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb16ffd640 1 -- 172.21.15.16:0/1178939706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10043020 con 0x7feb20101c60 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb16ffd640 1 -- 172.21.15.16:0/1178939706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feb10004440 con 0x7feb20101c60 2024-03-27T06:43:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb2800e640 1 -- 172.21.15.16:0/1178939706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2019fe40 con 0x7feb20101c60 2024-03-27T06:43:34.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb16ffd640 1 -- 172.21.15.16:0/1178939706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10003d30 con 0x7feb20101c60 2024-03-27T06:43:34.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.993+0000 7feb2800e640 1 -- 172.21.15.16:0/1178939706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb201a0300 con 0x7feb20101c60 2024-03-27T06:43:34.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.995+0000 7feb2800e640 1 -- 172.21.15.16:0/1178939706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb201a04d0 con 0x7feb20101c60 2024-03-27T06:43:34.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.996+0000 7feb16ffd640 1 -- 172.21.15.16:0/1178939706 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 138182+0+0 (secure 0 0 0) 0x7feb10046020 con 0x7feb20101c60 2024-03-27T06:43:34.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.996+0000 7feb16ffd640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feb0c0a3410 0x7feb0c0a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:34.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.996+0000 7feb25daa640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feb0c0a3410 0x7feb0c0a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:34.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.997+0000 7feb16ffd640 1 -- 172.21.15.16:0/1178939706 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7feb100f1bd0 con 0x7feb20101c60 2024-03-27T06:43:34.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:34.997+0000 7feb25daa640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feb0c0a3410 0x7feb0c0a58d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb20101ac0 tx=0x7feb1c00b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:34.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.001+0000 7feb16ffd640 1 -- 172.21.15.16:0/1178939706 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7feb1004d020 con 0x7feb20101c60 2024-03-27T06:43:35.079 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-27T06:43:35.079 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-27T06:43:35.079 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-mon[31908]: Standby manager daemon z restarted 2024-03-27T06:43:35.079 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-mon[31908]: Standby manager daemon z started 2024-03-27T06:43:35.079 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-27T06:43:35.079 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:43:35.079 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-27T06:43:35.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:43:35.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-mon[31908]: mgrmap e21: x(active, since 1.0191s), standbys: y, z 2024-03-27T06:43:35.080 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.812+0000 7f7062838700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:35.080 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.812+0000 7f7062838700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:35.080 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.812+0000 7f7062838700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:35.080 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.812+0000 7f7062838700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:35.080 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:43:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:43:34.812+0000 7f7062838700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:43:35.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.193+0000 7feb2800e640 1 -- 172.21.15.16:0/1178939706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb201a0860 con 0x7feb20101c60 2024-03-27T06:43:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.195+0000 7feb16ffd640 1 -- 172.21.15.16:0/1178939706 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22982 (secure 0 0 0) 0x7feb100be9a0 con 0x7feb20101c60 2024-03-27T06:43:35.191 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:35.191 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:43:33.780855+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3655179049":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3519122217":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6800/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/212881673":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/361067788":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:43:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 -- 172.21.15.16:0/1178939706 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feb0c0a3410 msgr2=0x7feb0c0a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feb0c0a3410 0x7feb0c0a58d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb20101ac0 tx=0x7feb1c00b390 comp rx=0 tx=0).stop 2024-03-27T06:43:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 -- 172.21.15.16:0/1178939706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb20101c60 msgr2=0x7feb20198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb20101c60 0x7feb20198540 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feb1002ca30 tx=0x7feb10004040 comp rx=0 tx=0).stop 2024-03-27T06:43:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 -- 172.21.15.16:0/1178939706 shutdown_connections 2024-03-27T06:43:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feb0c0a3410 0x7feb0c0a58d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb20102620 0x7feb2019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb20101c60 0x7feb20198540 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 --2- 172.21.15.16:0/1178939706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb20100a70 0x7feb20198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 -- 172.21.15.16:0/1178939706 >> 172.21.15.16:0/1178939706 conn(0x7feb200fbfe0 msgr2=0x7feb200fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 -- 172.21.15.16:0/1178939706 shutdown_connections 2024-03-27T06:43:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.197+0000 7feb2800e640 1 -- 172.21.15.16:0/1178939706 wait complete. 2024-03-27T06:43:35.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T06:41:26.393978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T06:41:27.707469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:43:35.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T06:43:35.221 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:34 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-27T06:43:35.222 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:34 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-27T06:43:35.222 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:34 smithi170 ceph-mon[36438]: Standby manager daemon z restarted 2024-03-27T06:43:35.222 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:34 smithi170 ceph-mon[36438]: Standby manager daemon z started 2024-03-27T06:43:35.222 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:34 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-27T06:43:35.222 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:34 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:43:35.222 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:34 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-27T06:43:35.222 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:34 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:43:35.222 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:34 smithi170 ceph-mon[36438]: mgrmap e21: x(active, since 1.0191s), standbys: y, z 2024-03-27T06:43:35.223 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y[37645]: 2024-03-27T06:43:35.053+0000 7fdea0dbe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-27T06:43:35.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:34 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-27T06:43:35.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:34 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-27T06:43:35.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:34 smithi156 ceph-mon[36261]: Standby manager daemon z restarted 2024-03-27T06:43:35.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:34 smithi156 ceph-mon[36261]: Standby manager daemon z started 2024-03-27T06:43:35.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:34 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-27T06:43:35.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:34 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:43:35.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:34 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-27T06:43:35.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:34 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.170:0/3206366596' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:43:35.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:34 smithi156 ceph-mon[36261]: mgrmap e21: x(active, since 1.0191s), standbys: y, z 2024-03-27T06:43:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.319+0000 7f7543d72640 1 -- 172.21.15.16:0/1635002214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f753c102600 msgr2=0x7f753c108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.319+0000 7f7543d72640 1 --2- 172.21.15.16:0/1635002214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f753c102600 0x7f753c108e90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7534009ca0 tx=0x7f753402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.319+0000 7f7543d72640 1 -- 172.21.15.16:0/1635002214 shutdown_connections 2024-03-27T06:43:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.319+0000 7f7543d72640 1 --2- 172.21.15.16:0/1635002214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f753c102600 0x7f753c108e90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.319+0000 7f7543d72640 1 --2- 172.21.15.16:0/1635002214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c101c40 0x7f753c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.319+0000 7f7543d72640 1 --2- 172.21.15.16:0/1635002214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c100a50 0x7f753c100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.319+0000 7f7543d72640 1 -- 172.21.15.16:0/1635002214 >> 172.21.15.16:0/1635002214 conn(0x7f753c0fbfe0 msgr2=0x7f753c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.319+0000 7f7543d72640 1 -- 172.21.15.16:0/1635002214 shutdown_connections 2024-03-27T06:43:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.319+0000 7f7543d72640 1 -- 172.21.15.16:0/1635002214 wait complete. 2024-03-27T06:43:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7543d72640 1 Processor -- start 2024-03-27T06:43:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7543d72640 1 -- start start 2024-03-27T06:43:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7543d72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f753c100a50 0x7f753c069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7543d72640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c101c40 0x7f753c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7543d72640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c102600 0x7f753c19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7543d72640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10c800 con 0x7f753c100a50 2024-03-27T06:43:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7543d72640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10c680 con 0x7f753c101c40 2024-03-27T06:43:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7543d72640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10c980 con 0x7f753c102600 2024-03-27T06:43:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7541b0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f753c100a50 0x7f753c069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7541b0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f753c100a50 0x7f753c069840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35870/0 (socket says 172.21.15.16:35870) 2024-03-27T06:43:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7541b0e640 1 -- 172.21.15.16:0/2406444746 learned_addr learned my addr 172.21.15.16:0/2406444746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f754130d640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c101c40 0x7f753c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f754230f640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c102600 0x7f753c19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7541b0e640 1 -- 172.21.15.16:0/2406444746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c102600 msgr2=0x7f753c19f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7541b0e640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c102600 0x7f753c19f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7541b0e640 1 -- 172.21.15.16:0/2406444746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c101c40 msgr2=0x7f753c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7541b0e640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c101c40 0x7f753c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7541b0e640 1 -- 172.21.15.16:0/2406444746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c19fb40 con 0x7f753c100a50 2024-03-27T06:43:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f754230f640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c102600 0x7f753c19f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:43:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.320+0000 7f7541b0e640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f753c100a50 0x7f753c069840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f752c00e9c0 tx=0x7f752c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.321+0000 7f7532ffd640 1 -- 172.21.15.16:0/2406444746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c00cd80 con 0x7f753c100a50 2024-03-27T06:43:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.321+0000 7f7532ffd640 1 -- 172.21.15.16:0/2406444746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f752c0035f0 con 0x7f753c100a50 2024-03-27T06:43:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.321+0000 7f7543d72640 1 -- 172.21.15.16:0/2406444746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c19fe30 con 0x7f753c100a50 2024-03-27T06:43:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.321+0000 7f7532ffd640 1 -- 172.21.15.16:0/2406444746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c010690 con 0x7f753c100a50 2024-03-27T06:43:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.321+0000 7f7543d72640 1 -- 172.21.15.16:0/2406444746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c1a02f0 con 0x7f753c100a50 2024-03-27T06:43:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.322+0000 7f7543d72640 1 -- 172.21.15.16:0/2406444746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c107d10 con 0x7f753c100a50 2024-03-27T06:43:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.323+0000 7f7532ffd640 1 -- 172.21.15.16:0/2406444746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 138182+0+0 (secure 0 0 0) 0x7f752c0037a0 con 0x7f753c100a50 2024-03-27T06:43:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.324+0000 7f7532ffd640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75280a3410 0x7f75280a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.324+0000 7f754130d640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75280a3410 0x7f75280a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.324+0000 7f7532ffd640 1 -- 172.21.15.16:0/2406444746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f752c0cff40 con 0x7f753c100a50 2024-03-27T06:43:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.325+0000 7f754130d640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75280a3410 0x7f75280a58d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f753c0695b0 tx=0x7f7538005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.328+0000 7f7532ffd640 1 -- 172.21.15.16:0/2406444746 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f752c09cda0 con 0x7f753c100a50 2024-03-27T06:43:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.515+0000 7f7543d72640 1 -- 172.21.15.16:0/2406444746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f753c02d5e0 con 0x7f753c100a50 2024-03-27T06:43:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.516+0000 7f7532ffd640 1 -- 172.21.15.16:0/2406444746 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f752c09c6a0 con 0x7f753c100a50 2024-03-27T06:43:35.511 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:43:35.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 -- 172.21.15.16:0/2406444746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75280a3410 msgr2=0x7f75280a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75280a3410 0x7f75280a58d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f753c0695b0 tx=0x7f7538005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:35.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 -- 172.21.15.16:0/2406444746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f753c100a50 msgr2=0x7f753c069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f753c100a50 0x7f753c069840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f752c00e9c0 tx=0x7f752c00ee80 comp rx=0 tx=0).stop 2024-03-27T06:43:35.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 -- 172.21.15.16:0/2406444746 shutdown_connections 2024-03-27T06:43:35.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75280a3410 0x7f75280a58d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c102600 0x7f753c19f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753c101c40 0x7f753c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 --2- 172.21.15.16:0/2406444746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f753c100a50 0x7f753c069840 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 -- 172.21.15.16:0/2406444746 >> 172.21.15.16:0/2406444746 conn(0x7f753c0fbfe0 msgr2=0x7f753c104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:35.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 -- 172.21.15.16:0/2406444746 shutdown_connections 2024-03-27T06:43:35.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.518+0000 7f7543d72640 1 -- 172.21.15.16:0/2406444746 wait complete. 2024-03-27T06:43:35.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T06:43:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.650+0000 7f0c527d0640 1 -- 172.21.15.16:0/217705878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c103710 msgr2=0x7f0c4c109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.650+0000 7f0c527d0640 1 --2- 172.21.15.16:0/217705878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c103710 0x7f0c4c109fa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009ca0 tx=0x7f0c4402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.650+0000 7f0c527d0640 1 -- 172.21.15.16:0/217705878 shutdown_connections 2024-03-27T06:43:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.650+0000 7f0c527d0640 1 --2- 172.21.15.16:0/217705878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c103710 0x7f0c4c109fa0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.650+0000 7f0c527d0640 1 --2- 172.21.15.16:0/217705878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c102d50 0x7f0c4c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.650+0000 7f0c527d0640 1 --2- 172.21.15.16:0/217705878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c4c101b60 0x7f0c4c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.650+0000 7f0c527d0640 1 -- 172.21.15.16:0/217705878 >> 172.21.15.16:0/217705878 conn(0x7f0c4c0fd0f0 msgr2=0x7f0c4c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.650+0000 7f0c527d0640 1 -- 172.21.15.16:0/217705878 shutdown_connections 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.650+0000 7f0c527d0640 1 -- 172.21.15.16:0/217705878 wait complete. 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.650+0000 7f0c527d0640 1 Processor -- start 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.650+0000 7f0c527d0640 1 -- start start 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c527d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c101b60 0x7f0c4c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c527d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c4c102d50 0x7f0c4c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c527d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c103710 0x7f0c4c19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c527d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c10d910 con 0x7f0c4c101b60 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c527d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c10d790 con 0x7f0c4c102d50 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c527d0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c10da90 con 0x7f0c4c103710 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c101b60 0x7f0c4c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c101b60 0x7f0c4c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35888/0 (socket says 172.21.15.16:35888) 2024-03-27T06:43:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c4bfff640 1 -- 172.21.15.16:0/1194668052 learned_addr learned my addr 172.21.15.16:0/1194668052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c50d6d640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c103710 0x7f0c4c19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c4b7fe640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c4c102d50 0x7f0c4c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c4bfff640 1 -- 172.21.15.16:0/1194668052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c103710 msgr2=0x7f0c4c19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c4bfff640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c103710 0x7f0c4c19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c4bfff640 1 -- 172.21.15.16:0/1194668052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c4c102d50 msgr2=0x7f0c4c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c4bfff640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c4c102d50 0x7f0c4c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c4bfff640 1 -- 172.21.15.16:0/1194668052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c19fb10 con 0x7f0c4c101b60 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c50d6d640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c103710 0x7f0c4c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.651+0000 7f0c4bfff640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c101b60 0x7f0c4c197ef0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00e9c0 tx=0x7f0c3c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.652+0000 7f0c497fa640 1 -- 172.21.15.16:0/1194668052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c00cd80 con 0x7f0c4c101b60 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.652+0000 7f0c527d0640 1 -- 172.21.15.16:0/1194668052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4c19fe00 con 0x7f0c4c101b60 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.652+0000 7f0c497fa640 1 -- 172.21.15.16:0/1194668052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0c3c0035f0 con 0x7f0c4c101b60 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.652+0000 7f0c497fa640 1 -- 172.21.15.16:0/1194668052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c010690 con 0x7f0c4c101b60 2024-03-27T06:43:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.652+0000 7f0c527d0640 1 -- 172.21.15.16:0/1194668052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4c1a02c0 con 0x7f0c4c101b60 2024-03-27T06:43:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.653+0000 7f0c527d0640 1 -- 172.21.15.16:0/1194668052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4c108e20 con 0x7f0c4c101b60 2024-03-27T06:43:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.654+0000 7f0c497fa640 1 -- 172.21.15.16:0/1194668052 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 138182+0+0 (secure 0 0 0) 0x7f0c3c0037a0 con 0x7f0c4c101b60 2024-03-27T06:43:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.655+0000 7f0c497fa640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0c340a3410 0x7f0c340a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.655+0000 7f0c497fa640 1 -- 172.21.15.16:0/1194668052 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0c3c0cfe80 con 0x7f0c4c101b60 2024-03-27T06:43:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.655+0000 7f0c4b7fe640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0c340a3410 0x7f0c340a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.656+0000 7f0c4b7fe640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0c340a3410 0x7f0c340a58d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c199400 tx=0x7f0c40005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.659+0000 7f0c497fa640 1 -- 172.21.15.16:0/1194668052 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f0c3c09cce0 con 0x7f0c4c101b60 2024-03-27T06:43:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.877+0000 7f0c527d0640 1 -- 172.21.15.16:0/1194668052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0c4c02d370 con 0x7f0c4c101b60 2024-03-27T06:43:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.878+0000 7f0c497fa640 1 -- 172.21.15.16:0/1194668052 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f0c3c09c5e0 con 0x7f0c4c101b60 2024-03-27T06:43:35.873 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-03-27T06:43:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 -- 172.21.15.16:0/1194668052 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0c340a3410 msgr2=0x7f0c340a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0c340a3410 0x7f0c340a58d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c199400 tx=0x7f0c40005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 -- 172.21.15.16:0/1194668052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c101b60 msgr2=0x7f0c4c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c101b60 0x7f0c4c197ef0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00e9c0 tx=0x7f0c3c00ee80 comp rx=0 tx=0).stop 2024-03-27T06:43:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 -- 172.21.15.16:0/1194668052 shutdown_connections 2024-03-27T06:43:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0c340a3410 0x7f0c340a58d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c103710 0x7f0c4c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c4c102d50 0x7f0c4c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 --2- 172.21.15.16:0/1194668052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c4c101b60 0x7f0c4c197ef0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 -- 172.21.15.16:0/1194668052 >> 172.21.15.16:0/1194668052 conn(0x7f0c4c0fd0f0 msgr2=0x7f0c4c105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 -- 172.21.15.16:0/1194668052 shutdown_connections 2024-03-27T06:43:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:35.880+0000 7f0c527d0640 1 -- 172.21.15.16:0/1194668052 wait complete. 2024-03-27T06:43:35.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T06:43:36.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.017+0000 7f7b0cedf640 1 -- 172.21.15.16:0/1221244696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b081025d0 msgr2=0x7f7b081049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.017+0000 7f7b0cedf640 1 --2- 172.21.15.16:0/1221244696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b081025d0 0x7f7b081049c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009900 tx=0x7f7af802bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:36.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.017+0000 7f7b0cedf640 1 -- 172.21.15.16:0/1221244696 shutdown_connections 2024-03-27T06:43:36.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.017+0000 7f7b0cedf640 1 --2- 172.21.15.16:0/1221244696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08107830 0x7f7b08109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.017+0000 7f7b0cedf640 1 --2- 172.21.15.16:0/1221244696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b08104f00 0x7f7b081072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.017+0000 7f7b0cedf640 1 --2- 172.21.15.16:0/1221244696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b081025d0 0x7f7b081049c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.017+0000 7f7b0cedf640 1 -- 172.21.15.16:0/1221244696 >> 172.21.15.16:0/1221244696 conn(0x7f7b080fbfe0 msgr2=0x7f7b080fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:36.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.017+0000 7f7b0cedf640 1 -- 172.21.15.16:0/1221244696 shutdown_connections 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.017+0000 7f7b0cedf640 1 -- 172.21.15.16:0/1221244696 wait complete. 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b0cedf640 1 Processor -- start 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b0cedf640 1 -- start start 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b0cedf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b081025d0 0x7f7b08197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b0cedf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08104f00 0x7f7b08198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b0cedf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b08107830 0x7f7b0819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b0cedf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0810d8d0 con 0x7f7b081025d0 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b0cedf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0810d750 con 0x7f7b08104f00 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b0cedf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0810da50 con 0x7f7b08107830 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b0659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b081025d0 0x7f7b08197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b05d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08104f00 0x7f7b08198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b0659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b081025d0 0x7f7b08197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51016/0 (socket says 172.21.15.16:51016) 2024-03-27T06:43:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b0659c640 1 -- 172.21.15.16:0/428653869 learned_addr learned my addr 172.21.15.16:0/428653869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.018+0000 7f7b06d9d640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b08107830 0x7f7b0819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7b0659c640 1 -- 172.21.15.16:0/428653869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b08107830 msgr2=0x7f7b0819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7b0659c640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b08107830 0x7f7b0819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7b0659c640 1 -- 172.21.15.16:0/428653869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08104f00 msgr2=0x7f7b08198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7b0659c640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08104f00 0x7f7b08198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7b0659c640 1 -- 172.21.15.16:0/428653869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0819fbf0 con 0x7f7b081025d0 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7b06d9d640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b08107830 0x7f7b0819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7b05d9b640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08104f00 0x7f7b08198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7b0659c640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b081025d0 0x7f7b08197fd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009230 tx=0x7f7af8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7af77fe640 1 -- 172.21.15.16:0/428653869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8043020 con 0x7f7b081025d0 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7af77fe640 1 -- 172.21.15.16:0/428653869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7af8004250 con 0x7f7b081025d0 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7b0cedf640 1 -- 172.21.15.16:0/428653869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0819fe80 con 0x7f7b081025d0 2024-03-27T06:43:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7af77fe640 1 -- 172.21.15.16:0/428653869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8003b50 con 0x7f7b081025d0 2024-03-27T06:43:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.019+0000 7f7b0cedf640 1 -- 172.21.15.16:0/428653869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b081a02e0 con 0x7f7b081025d0 2024-03-27T06:43:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.020+0000 7f7b0cedf640 1 -- 172.21.15.16:0/428653869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b080fdad0 con 0x7f7b081025d0 2024-03-27T06:43:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.021+0000 7f7af77fe640 1 -- 172.21.15.16:0/428653869 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 138541+0+0 (secure 0 0 0) 0x7f7af8046020 con 0x7f7b081025d0 2024-03-27T06:43:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.022+0000 7f7af77fe640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7af00a3810 0x7f7af00a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.022+0000 7f7af77fe640 1 -- 172.21.15.16:0/428653869 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7af80f2640 con 0x7f7b081025d0 2024-03-27T06:43:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.022+0000 7f7b05d9b640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7af00a3810 0x7f7af00a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.023+0000 7f7b05d9b640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7af00a3810 0x7f7af00a5cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b081994e0 tx=0x7f7afc005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.029+0000 7f7af77fe640 1 -- 172.21.15.16:0/428653869 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7af804d020 con 0x7f7b081025d0 2024-03-27T06:43:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.226+0000 7f7b0cedf640 1 -- 172.21.15.16:0/428653869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7b0802d5c0 con 0x7f7b081025d0 2024-03-27T06:43:36.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.227+0000 7f7af77fe640 1 -- 172.21.15.16:0/428653869 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f7af80beea0 con 0x7f7b081025d0 2024-03-27T06:43:36.222 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-03-27T06:43:36.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-mon[36261]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:36.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-mon[36261]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:36.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-mon[36261]: Standby manager daemon y restarted 2024-03-27T06:43:36.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-mon[36261]: Standby manager daemon y started 2024-03-27T06:43:36.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-27T06:43:36.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:43:36.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-27T06:43:36.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-mon[36261]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:43:36.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1178939706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:36.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2406444746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:36.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:35 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1194668052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 -- 172.21.15.16:0/428653869 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7af00a3810 msgr2=0x7f7af00a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7af00a3810 0x7f7af00a5cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b081994e0 tx=0x7f7afc005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 -- 172.21.15.16:0/428653869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b081025d0 msgr2=0x7f7b08197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b081025d0 0x7f7b08197fd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009230 tx=0x7f7af8004ab0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 -- 172.21.15.16:0/428653869 shutdown_connections 2024-03-27T06:43:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7af00a3810 0x7f7af00a5cd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b08107830 0x7f7b0819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b08104f00 0x7f7b08198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 --2- 172.21.15.16:0/428653869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b081025d0 0x7f7b08197fd0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 -- 172.21.15.16:0/428653869 >> 172.21.15.16:0/428653869 conn(0x7f7b080fbfe0 msgr2=0x7f7b08103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 -- 172.21.15.16:0/428653869 shutdown_connections 2024-03-27T06:43:36.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.229+0000 7f7b0cedf640 1 -- 172.21.15.16:0/428653869 wait complete. 2024-03-27T06:43:36.238 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:35 smithi016 ceph-mon[31908]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:36.238 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:35 smithi016 ceph-mon[31908]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:36.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:35 smithi016 ceph-mon[31908]: Standby manager daemon y restarted 2024-03-27T06:43:36.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:35 smithi016 ceph-mon[31908]: Standby manager daemon y started 2024-03-27T06:43:36.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:35 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-27T06:43:36.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:35 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:43:36.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:35 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-27T06:43:36.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:35 smithi016 ceph-mon[31908]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:43:36.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:35 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1178939706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:36.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:35 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2406444746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:36.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:35 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1194668052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:36.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:43:36.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:35 smithi170 ceph-mon[36438]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:36.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:35 smithi170 ceph-mon[36438]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:36.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:35 smithi170 ceph-mon[36438]: Standby manager daemon y restarted 2024-03-27T06:43:36.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:35 smithi170 ceph-mon[36438]: Standby manager daemon y started 2024-03-27T06:43:36.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:35 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-27T06:43:36.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:35 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-27T06:43:36.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:35 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-27T06:43:36.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:35 smithi170 ceph-mon[36438]: from='mgr.? 172.21.15.156:0/2032580951' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-27T06:43:36.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:35 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1178939706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:36.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:35 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2406444746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:36.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:35 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1194668052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.352+0000 7fcf239bf640 1 -- 172.21.15.16:0/1343822933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c1025f0 msgr2=0x7fcf1c1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.352+0000 7fcf239bf640 1 --2- 172.21.15.16:0/1343822933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c1025f0 0x7fcf1c1049e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009900 tx=0x7fcf1802bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.352+0000 7fcf239bf640 1 -- 172.21.15.16:0/1343822933 shutdown_connections 2024-03-27T06:43:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.352+0000 7fcf239bf640 1 --2- 172.21.15.16:0/1343822933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf1c107850 0x7fcf1c109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.352+0000 7fcf239bf640 1 --2- 172.21.15.16:0/1343822933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1c104f20 0x7fcf1c107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.352+0000 7fcf239bf640 1 --2- 172.21.15.16:0/1343822933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c1025f0 0x7fcf1c1049e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.352+0000 7fcf239bf640 1 -- 172.21.15.16:0/1343822933 >> 172.21.15.16:0/1343822933 conn(0x7fcf1c0fbfe0 msgr2=0x7fcf1c0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.352+0000 7fcf239bf640 1 -- 172.21.15.16:0/1343822933 shutdown_connections 2024-03-27T06:43:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.352+0000 7fcf239bf640 1 -- 172.21.15.16:0/1343822933 wait complete. 2024-03-27T06:43:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf239bf640 1 Processor -- start 2024-03-27T06:43:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf239bf640 1 -- start start 2024-03-27T06:43:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf239bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c1025f0 0x7fcf1c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf239bf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1c104f20 0x7fcf1c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf239bf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf1c107850 0x7fcf1c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf239bf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c10d8f0 con 0x7fcf1c1025f0 2024-03-27T06:43:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf239bf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c10d770 con 0x7fcf1c107850 2024-03-27T06:43:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf239bf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c10da70 con 0x7fcf1c104f20 2024-03-27T06:43:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf2175b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c1025f0 0x7fcf1c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf2175b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c1025f0 0x7fcf1c197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51042/0 (socket says 172.21.15.16:51042) 2024-03-27T06:43:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf2175b640 1 -- 172.21.15.16:0/992334262 learned_addr learned my addr 172.21.15.16:0/992334262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf20f5a640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1c104f20 0x7fcf1c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf21f5c640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf1c107850 0x7fcf1c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf2175b640 1 -- 172.21.15.16:0/992334262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1c104f20 msgr2=0x7fcf1c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf2175b640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1c104f20 0x7fcf1c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf2175b640 1 -- 172.21.15.16:0/992334262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf1c107850 msgr2=0x7fcf1c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf2175b640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf1c107850 0x7fcf1c19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf2175b640 1 -- 172.21.15.16:0/992334262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c19fb90 con 0x7fcf1c1025f0 2024-03-27T06:43:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.353+0000 7fcf21f5c640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf1c107850 0x7fcf1c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:43:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.354+0000 7fcf2175b640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c1025f0 0x7fcf1c197f70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009230 tx=0x7fcf18004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.354+0000 7fcf127fc640 1 -- 172.21.15.16:0/992334262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18043020 con 0x7fcf1c1025f0 2024-03-27T06:43:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.354+0000 7fcf239bf640 1 -- 172.21.15.16:0/992334262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1c19fe20 con 0x7fcf1c1025f0 2024-03-27T06:43:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.354+0000 7fcf127fc640 1 -- 172.21.15.16:0/992334262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcf18004250 con 0x7fcf1c1025f0 2024-03-27T06:43:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.354+0000 7fcf127fc640 1 -- 172.21.15.16:0/992334262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18003b50 con 0x7fcf1c1025f0 2024-03-27T06:43:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.354+0000 7fcf239bf640 1 -- 172.21.15.16:0/992334262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1c1a02e0 con 0x7fcf1c1025f0 2024-03-27T06:43:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.355+0000 7fcf239bf640 1 -- 172.21.15.16:0/992334262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1c0fdad0 con 0x7fcf1c1025f0 2024-03-27T06:43:36.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.356+0000 7fcf127fc640 1 -- 172.21.15.16:0/992334262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 138541+0+0 (secure 0 0 0) 0x7fcf18046020 con 0x7fcf1c1025f0 2024-03-27T06:43:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.356+0000 7fcf127fc640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf080a3810 0x7fcf080a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.356+0000 7fcf127fc640 1 -- 172.21.15.16:0/992334262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fcf180f23a0 con 0x7fcf1c1025f0 2024-03-27T06:43:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.357+0000 7fcf20f5a640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf080a3810 0x7fcf080a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.357+0000 7fcf20f5a640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf080a3810 0x7fcf080a5cd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c199480 tx=0x7fcf0c005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:36.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.361+0000 7fcf127fc640 1 -- 172.21.15.16:0/992334262 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fcf1804d020 con 0x7fcf1c1025f0 2024-03-27T06:43:36.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.590+0000 7fcf239bf640 1 -- 172.21.15.16:0/992334262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf1c02d520 con 0x7fcf1c1025f0 2024-03-27T06:43:36.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.592+0000 7fcf127fc640 1 -- 172.21.15.16:0/992334262 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22982 (secure 0 0 0) 0x7fcf180beba0 con 0x7fcf1c1025f0 2024-03-27T06:43:36.587 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:36.587 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:43:33.780855+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3655179049":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3519122217":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6800/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/212881673":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/361067788":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:43:36.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 -- 172.21.15.16:0/992334262 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf080a3810 msgr2=0x7fcf080a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf080a3810 0x7fcf080a5cd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c199480 tx=0x7fcf0c005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:36.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 -- 172.21.15.16:0/992334262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c1025f0 msgr2=0x7fcf1c197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c1025f0 0x7fcf1c197f70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009230 tx=0x7fcf18004ab0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 -- 172.21.15.16:0/992334262 shutdown_connections 2024-03-27T06:43:36.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf1c107850 0x7fcf1c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf080a3810 0x7fcf080a5cd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1c104f20 0x7fcf1c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 --2- 172.21.15.16:0/992334262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c1025f0 0x7fcf1c197f70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 -- 172.21.15.16:0/992334262 >> 172.21.15.16:0/992334262 conn(0x7fcf1c0fbfe0 msgr2=0x7fcf1c103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:36.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 -- 172.21.15.16:0/992334262 shutdown_connections 2024-03-27T06:43:36.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.594+0000 7fcf239bf640 1 -- 172.21.15.16:0/992334262 wait complete. 2024-03-27T06:43:36.600 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T06:41:26.393978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T06:41:27.707469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:43:36.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T06:43:36.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.717+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/219332357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fb8107870 msgr2=0x7f5fb8109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.717+0000 7f5fbf0eb640 1 --2- 172.21.15.16:0/219332357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fb8107870 0x7f5fb8109cc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009900 tx=0x7f5fb002bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:36.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.717+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/219332357 shutdown_connections 2024-03-27T06:43:36.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.717+0000 7f5fbf0eb640 1 --2- 172.21.15.16:0/219332357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fb8107870 0x7f5fb8109cc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.717+0000 7f5fbf0eb640 1 --2- 172.21.15.16:0/219332357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb8104f40 0x7f5fb8107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.717+0000 7f5fbf0eb640 1 --2- 172.21.15.16:0/219332357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fb8102610 0x7f5fb8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.717+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/219332357 >> 172.21.15.16:0/219332357 conn(0x7f5fb80fc000 msgr2=0x7f5fb80fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:36.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.718+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/219332357 shutdown_connections 2024-03-27T06:43:36.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.718+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/219332357 wait complete. 2024-03-27T06:43:36.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.718+0000 7f5fbf0eb640 1 Processor -- start 2024-03-27T06:43:36.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.718+0000 7f5fbf0eb640 1 -- start start 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbf0eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fb8102610 0x7f5fb8101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbf0eb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb8104f40 0x7f5fb81005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbf0eb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fb8107870 0x7f5fb8100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbf0eb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb810d910 con 0x7f5fb8102610 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbf0eb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb810d790 con 0x7f5fb8104f40 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbf0eb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb810da90 con 0x7f5fb8107870 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbce87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fb8102610 0x7f5fb8101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbce87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fb8102610 0x7f5fb8101f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51060/0 (socket says 172.21.15.16:51060) 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbce87640 1 -- 172.21.15.16:0/1600195556 learned_addr learned my addr 172.21.15.16:0/1600195556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fb7fff640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb8104f40 0x7f5fb81005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbd688640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fb8107870 0x7f5fb8100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbce87640 1 -- 172.21.15.16:0/1600195556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fb8107870 msgr2=0x7f5fb8100b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbce87640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fb8107870 0x7f5fb8100b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbce87640 1 -- 172.21.15.16:0/1600195556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb8104f40 msgr2=0x7f5fb81005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbce87640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb8104f40 0x7f5fb81005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbce87640 1 -- 172.21.15.16:0/1600195556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb8101390 con 0x7f5fb8102610 2024-03-27T06:43:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbce87640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fb8102610 0x7f5fb8101f80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800eb90 tx=0x7f5fa800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fb5ffb640 1 -- 172.21.15.16:0/1600195556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8004d20 con 0x7f5fb8102610 2024-03-27T06:43:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/1600195556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb81a20a0 con 0x7f5fb8102610 2024-03-27T06:43:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fb5ffb640 1 -- 172.21.15.16:0/1600195556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5fa800ee90 con 0x7f5fb8102610 2024-03-27T06:43:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.719+0000 7f5fb5ffb640 1 -- 172.21.15.16:0/1600195556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80108b0 con 0x7f5fb8102610 2024-03-27T06:43:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.720+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/1600195556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb81a25a0 con 0x7f5fb8102610 2024-03-27T06:43:36.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.721+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/1600195556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb80fdaf0 con 0x7f5fb8102610 2024-03-27T06:43:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.722+0000 7f5fb5ffb640 1 -- 172.21.15.16:0/1600195556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 138541+0+0 (secure 0 0 0) 0x7f5fa8010a50 con 0x7f5fb8102610 2024-03-27T06:43:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.723+0000 7f5fb5ffb640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5fa40a3810 0x7f5fa40a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.723+0000 7f5fb7fff640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5fa40a3810 0x7f5fa40a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.723+0000 7f5fb5ffb640 1 -- 172.21.15.16:0/1600195556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5fa80d0b00 con 0x7f5fb8102610 2024-03-27T06:43:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.723+0000 7f5fb7fff640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5fa40a3810 0x7f5fa40a5cd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8101cf0 tx=0x7f5fa0005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:36.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.726+0000 7f5fb5ffb640 1 -- 172.21.15.16:0/1600195556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5fa8053cb0 con 0x7f5fb8102610 2024-03-27T06:43:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.973+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/1600195556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5fb8062750 con 0x7f5fb8102610 2024-03-27T06:43:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.974+0000 7f5fb5ffb640 1 -- 172.21.15.16:0/1600195556 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f5fa809d250 con 0x7f5fb8102610 2024-03-27T06:43:36.969 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:43:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/1600195556 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5fa40a3810 msgr2=0x7f5fa40a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5fa40a3810 0x7f5fa40a5cd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8101cf0 tx=0x7f5fa0005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/1600195556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fb8102610 msgr2=0x7f5fb8101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fb8102610 0x7f5fb8101f80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800eb90 tx=0x7f5fa800c6a0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/1600195556 shutdown_connections 2024-03-27T06:43:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5fa40a3810 0x7f5fa40a5cd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fb8107870 0x7f5fb8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fb8104f40 0x7f5fb81005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 --2- 172.21.15.16:0/1600195556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fb8102610 0x7f5fb8101f80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/1600195556 >> 172.21.15.16:0/1600195556 conn(0x7f5fb80fc000 msgr2=0x7f5fb8102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:36.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/1600195556 shutdown_connections 2024-03-27T06:43:36.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:36.976+0000 7f5fbf0eb640 1 -- 172.21.15.16:0/1600195556 wait complete. 2024-03-27T06:43:36.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T06:43:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.095+0000 7efe6d683640 1 -- 172.21.15.16:0/3541008898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68102630 msgr2=0x7efe68104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.095+0000 7efe6d683640 1 --2- 172.21.15.16:0/3541008898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68102630 0x7efe68104a20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efe58009900 tx=0x7efe5802bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.095+0000 7efe6d683640 1 -- 172.21.15.16:0/3541008898 shutdown_connections 2024-03-27T06:43:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.095+0000 7efe6d683640 1 --2- 172.21.15.16:0/3541008898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe68107890 0x7efe68109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.095+0000 7efe6d683640 1 --2- 172.21.15.16:0/3541008898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe68104f60 0x7efe68107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.095+0000 7efe6d683640 1 --2- 172.21.15.16:0/3541008898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68102630 0x7efe68104a20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.095+0000 7efe6d683640 1 -- 172.21.15.16:0/3541008898 >> 172.21.15.16:0/3541008898 conn(0x7efe680fbfe0 msgr2=0x7efe680fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.095+0000 7efe6d683640 1 -- 172.21.15.16:0/3541008898 shutdown_connections 2024-03-27T06:43:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.095+0000 7efe6d683640 1 -- 172.21.15.16:0/3541008898 wait complete. 2024-03-27T06:43:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.095+0000 7efe6d683640 1 Processor -- start 2024-03-27T06:43:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe6d683640 1 -- start start 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe6d683640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe68102630 0x7efe6819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe6d683640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68104f60 0x7efe6819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe6d683640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe68107890 0x7efe681a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe6d683640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810d8b0 con 0x7efe68104f60 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe6d683640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810d730 con 0x7efe68102630 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe6d683640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810da30 con 0x7efe68107890 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68104f60 0x7efe6819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68104f60 0x7efe6819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51078/0 (socket says 172.21.15.16:51078) 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe667fc640 1 -- 172.21.15.16:0/2628363166 learned_addr learned my addr 172.21.15.16:0/2628363166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe66ffd640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe68102630 0x7efe6819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe677fe640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe68107890 0x7efe681a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe667fc640 1 -- 172.21.15.16:0/2628363166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe68107890 msgr2=0x7efe681a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe667fc640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe68107890 0x7efe681a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe667fc640 1 -- 172.21.15.16:0/2628363166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe68102630 msgr2=0x7efe6819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe667fc640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe68102630 0x7efe6819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe667fc640 1 -- 172.21.15.16:0/2628363166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe681a1dd0 con 0x7efe68104f60 2024-03-27T06:43:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.096+0000 7efe667fc640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68104f60 0x7efe6819a6f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005fd0 tx=0x7efe5c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.097+0000 7efe4ffff640 1 -- 172.21.15.16:0/2628363166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c017070 con 0x7efe68104f60 2024-03-27T06:43:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.097+0000 7efe6d683640 1 -- 172.21.15.16:0/2628363166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe681a20c0 con 0x7efe68104f60 2024-03-27T06:43:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.097+0000 7efe4ffff640 1 -- 172.21.15.16:0/2628363166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efe5c009c30 con 0x7efe68104f60 2024-03-27T06:43:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.097+0000 7efe4ffff640 1 -- 172.21.15.16:0/2628363166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c010460 con 0x7efe68104f60 2024-03-27T06:43:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.097+0000 7efe6d683640 1 -- 172.21.15.16:0/2628363166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe681a2660 con 0x7efe68104f60 2024-03-27T06:43:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.098+0000 7efe6d683640 1 -- 172.21.15.16:0/2628363166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe68109380 con 0x7efe68104f60 2024-03-27T06:43:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.100+0000 7efe4ffff640 1 -- 172.21.15.16:0/2628363166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 138541+0+0 (secure 0 0 0) 0x7efe5c003eb0 con 0x7efe68104f60 2024-03-27T06:43:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.100+0000 7efe4ffff640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7efe500a3810 0x7efe500a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.100+0000 7efe66ffd640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7efe500a3810 0x7efe500a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.100+0000 7efe4ffff640 1 -- 172.21.15.16:0/2628363166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7efe5c0cfee0 con 0x7efe68104f60 2024-03-27T06:43:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.101+0000 7efe66ffd640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7efe500a3810 0x7efe500a5cd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efe58000c00 tx=0x7efe58005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.109+0000 7efe4ffff640 1 -- 172.21.15.16:0/2628363166 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7efe5c020030 con 0x7efe68104f60 2024-03-27T06:43:37.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:36 smithi156 ceph-mon[36261]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:37.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:36 smithi156 ceph-mon[36261]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-27T06:43:37.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:36 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:36 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:36 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:36 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:36 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:36 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:36 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/428653869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:37.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:36 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/992334262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:37.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.296+0000 7efe6d683640 1 -- 172.21.15.16:0/2628363166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efe6802d570 con 0x7efe68104f60 2024-03-27T06:43:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.296+0000 7efe4ffff640 1 -- 172.21.15.16:0/2628363166 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7efe5c09c630 con 0x7efe68104f60 2024-03-27T06:43:37.291 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-03-27T06:43:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 -- 172.21.15.16:0/2628363166 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7efe500a3810 msgr2=0x7efe500a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7efe500a3810 0x7efe500a5cd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efe58000c00 tx=0x7efe58005fd0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 -- 172.21.15.16:0/2628363166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68104f60 msgr2=0x7efe6819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68104f60 0x7efe6819a6f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005fd0 tx=0x7efe5c00dde0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 -- 172.21.15.16:0/2628363166 shutdown_connections 2024-03-27T06:43:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7efe500a3810 0x7efe500a5cd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe68107890 0x7efe681a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe68104f60 0x7efe6819a6f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 --2- 172.21.15.16:0/2628363166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe68102630 0x7efe6819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 -- 172.21.15.16:0/2628363166 >> 172.21.15.16:0/2628363166 conn(0x7efe680fbfe0 msgr2=0x7efe68101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:37.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 -- 172.21.15.16:0/2628363166 shutdown_connections 2024-03-27T06:43:37.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.299+0000 7efe6d683640 1 -- 172.21.15.16:0/2628363166 wait complete. 2024-03-27T06:43:37.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:36 smithi016 ceph-mon[31908]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:37.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:36 smithi016 ceph-mon[31908]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-27T06:43:37.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:36 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:36 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:36 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:36 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:36 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:36 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:36 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/428653869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:37.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:36 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/992334262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:37.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T06:43:37.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:36 smithi170 ceph-mon[36438]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:37.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:36 smithi170 ceph-mon[36438]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-27T06:43:37.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:36 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:36 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:36 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:36 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:36 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:36 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:37.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:36 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/428653869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:37.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:36 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/992334262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.430+0000 7f4180d8a640 1 -- 172.21.15.16:0/595903542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c102600 msgr2=0x7f417c108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.430+0000 7f4180d8a640 1 --2- 172.21.15.16:0/595903542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c102600 0x7f417c108e90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4174009900 tx=0x7f417402bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.430+0000 7f4180d8a640 1 -- 172.21.15.16:0/595903542 shutdown_connections 2024-03-27T06:43:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.430+0000 7f4180d8a640 1 --2- 172.21.15.16:0/595903542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c102600 0x7f417c108e90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.430+0000 7f4180d8a640 1 --2- 172.21.15.16:0/595903542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c101c40 0x7f417c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.430+0000 7f4180d8a640 1 --2- 172.21.15.16:0/595903542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f417c100a50 0x7f417c100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.430+0000 7f4180d8a640 1 -- 172.21.15.16:0/595903542 >> 172.21.15.16:0/595903542 conn(0x7f417c0fbfe0 msgr2=0x7f417c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.430+0000 7f4180d8a640 1 -- 172.21.15.16:0/595903542 shutdown_connections 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.430+0000 7f4180d8a640 1 -- 172.21.15.16:0/595903542 wait complete. 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f4180d8a640 1 Processor -- start 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f4180d8a640 1 -- start start 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f4180d8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c100a50 0x7f417c069880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f4180d8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f417c101c40 0x7f417c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f4180d8a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c102600 0x7f417c19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f4180d8a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10c7b0 con 0x7f417c100a50 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f4180d8a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10c630 con 0x7f417c102600 2024-03-27T06:43:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f4180d8a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10c930 con 0x7f417c101c40 2024-03-27T06:43:37.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f417a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c100a50 0x7f417c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f417a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c100a50 0x7f417c069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51110/0 (socket says 172.21.15.16:51110) 2024-03-27T06:43:37.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f417a59c640 1 -- 172.21.15.16:0/2302198908 learned_addr learned my addr 172.21.15.16:0/2302198908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:37.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f4179d9b640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f417c101c40 0x7f417c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f417ad9d640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c102600 0x7f417c19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f417a59c640 1 -- 172.21.15.16:0/2302198908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f417c101c40 msgr2=0x7f417c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f417a59c640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f417c101c40 0x7f417c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f417a59c640 1 -- 172.21.15.16:0/2302198908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c102600 msgr2=0x7f417c19f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f417a59c640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c102600 0x7f417c19f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.431+0000 7f417a59c640 1 -- 172.21.15.16:0/2302198908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417c19fb40 con 0x7f417c100a50 2024-03-27T06:43:37.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.432+0000 7f417a59c640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c100a50 0x7f417c069880 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f416c00eb70 tx=0x7f416c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.432+0000 7f416b7fe640 1 -- 172.21.15.16:0/2302198908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c003b70 con 0x7f417c100a50 2024-03-27T06:43:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.432+0000 7f4180d8a640 1 -- 172.21.15.16:0/2302198908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417c19fe30 con 0x7f417c100a50 2024-03-27T06:43:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.432+0000 7f416b7fe640 1 -- 172.21.15.16:0/2302198908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f416c00ee70 con 0x7f417c100a50 2024-03-27T06:43:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.432+0000 7f416b7fe640 1 -- 172.21.15.16:0/2302198908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c0108d0 con 0x7f417c100a50 2024-03-27T06:43:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.432+0000 7f4180d8a640 1 -- 172.21.15.16:0/2302198908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417c1a0240 con 0x7f417c100a50 2024-03-27T06:43:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.433+0000 7f4180d8a640 1 -- 172.21.15.16:0/2302198908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f417c107d70 con 0x7f417c100a50 2024-03-27T06:43:37.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.434+0000 7f416b7fe640 1 -- 172.21.15.16:0/2302198908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 138541+0+0 (secure 0 0 0) 0x7f416c003f30 con 0x7f417c100a50 2024-03-27T06:43:37.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.434+0000 7f416b7fe640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f41640a3810 0x7f41640a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.435+0000 7f4179d9b640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f41640a3810 0x7f41640a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.435+0000 7f416b7fe640 1 -- 172.21.15.16:0/2302198908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f416c0d0890 con 0x7f417c100a50 2024-03-27T06:43:37.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.435+0000 7f4179d9b640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f41640a3810 0x7f41640a5cd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f417c0695f0 tx=0x7f4170005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:37.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.439+0000 7f416b7fe640 1 -- 172.21.15.16:0/2302198908 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f416c09cfe0 con 0x7f417c100a50 2024-03-27T06:43:37.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.632+0000 7f4180d8a640 1 -- 172.21.15.16:0/2302198908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f417c02d5e0 con 0x7f417c100a50 2024-03-27T06:43:37.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.632+0000 7f416b7fe640 1 -- 172.21.15.16:0/2302198908 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f416c085e20 con 0x7f417c100a50 2024-03-27T06:43:37.627 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-03-27T06:43:37.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 -- 172.21.15.16:0/2302198908 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f41640a3810 msgr2=0x7f41640a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f41640a3810 0x7f41640a5cd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f417c0695f0 tx=0x7f4170005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 -- 172.21.15.16:0/2302198908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c100a50 msgr2=0x7f417c069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c100a50 0x7f417c069880 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f416c00eb70 tx=0x7f416c00c6a0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 -- 172.21.15.16:0/2302198908 shutdown_connections 2024-03-27T06:43:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f417c102600 0x7f417c19f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f41640a3810 0x7f41640a5cd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f417c101c40 0x7f417c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 --2- 172.21.15.16:0/2302198908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417c100a50 0x7f417c069880 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 -- 172.21.15.16:0/2302198908 >> 172.21.15.16:0/2302198908 conn(0x7f417c0fbfe0 msgr2=0x7f417c104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:37.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 -- 172.21.15.16:0/2302198908 shutdown_connections 2024-03-27T06:43:37.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.635+0000 7f4180d8a640 1 -- 172.21.15.16:0/2302198908 wait complete. 2024-03-27T06:43:37.642 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-27T06:43:37.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:43:37.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.753+0000 7f944d581640 1 -- 172.21.15.16:0/3002476560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9448102620 msgr2=0x7f9448108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.753+0000 7f944d581640 1 --2- 172.21.15.16:0/3002476560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9448102620 0x7f9448108eb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9440009ca0 tx=0x7f944002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.753+0000 7f944d581640 1 -- 172.21.15.16:0/3002476560 shutdown_connections 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.753+0000 7f944d581640 1 --2- 172.21.15.16:0/3002476560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9448102620 0x7f9448108eb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.753+0000 7f944d581640 1 --2- 172.21.15.16:0/3002476560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448101c60 0x7f94481020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.753+0000 7f944d581640 1 --2- 172.21.15.16:0/3002476560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9448100a70 0x7f9448100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.753+0000 7f944d581640 1 -- 172.21.15.16:0/3002476560 >> 172.21.15.16:0/3002476560 conn(0x7f94480fbfe0 msgr2=0x7f94480fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.753+0000 7f944d581640 1 -- 172.21.15.16:0/3002476560 shutdown_connections 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.753+0000 7f944d581640 1 -- 172.21.15.16:0/3002476560 wait complete. 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.753+0000 7f944d581640 1 Processor -- start 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.753+0000 7f944d581640 1 -- start start 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f944d581640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448100a70 0x7f944810b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f944d581640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9448101c60 0x7f9448109a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f944d581640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9448102620 0x7f9448109f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f944d581640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f944810d930 con 0x7f9448102620 2024-03-27T06:43:37.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f944d581640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f944810d7b0 con 0x7f9448101c60 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f944d581640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f944810dab0 con 0x7f9448100a70 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f94477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9448102620 0x7f9448109f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f94477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9448102620 0x7f9448109f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51130/0 (socket says 172.21.15.16:51130) 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f94477fe640 1 -- 172.21.15.16:0/504699565 learned_addr learned my addr 172.21.15.16:0/504699565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f9446ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448100a70 0x7f944810b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f94467fc640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9448101c60 0x7f9448109a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f94477fe640 1 -- 172.21.15.16:0/504699565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448100a70 msgr2=0x7f944810b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f94477fe640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448100a70 0x7f944810b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f94477fe640 1 -- 172.21.15.16:0/504699565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9448101c60 msgr2=0x7f9448109a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f94477fe640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9448101c60 0x7f9448109a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f94477fe640 1 -- 172.21.15.16:0/504699565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944810a800 con 0x7f9448102620 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.754+0000 7f94477fe640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9448102620 0x7f9448109f70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f944002c630 tx=0x7f9440034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.755+0000 7f942ffff640 1 -- 172.21.15.16:0/504699565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440039040 con 0x7f9448102620 2024-03-27T06:43:37.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.755+0000 7f944d581640 1 -- 172.21.15.16:0/504699565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94481a2040 con 0x7f9448102620 2024-03-27T06:43:37.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.755+0000 7f942ffff640 1 -- 172.21.15.16:0/504699565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9440044070 con 0x7f9448102620 2024-03-27T06:43:37.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.755+0000 7f942ffff640 1 -- 172.21.15.16:0/504699565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440004420 con 0x7f9448102620 2024-03-27T06:43:37.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.755+0000 7f944d581640 1 -- 172.21.15.16:0/504699565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94481a2590 con 0x7f9448102620 2024-03-27T06:43:37.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.756+0000 7f944d581640 1 -- 172.21.15.16:0/504699565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9448107cc0 con 0x7f9448102620 2024-03-27T06:43:37.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.758+0000 7f942ffff640 1 -- 172.21.15.16:0/504699565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 138541+0+0 (secure 0 0 0) 0x7f94400045c0 con 0x7f9448102620 2024-03-27T06:43:37.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.758+0000 7f942ffff640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f94300a3810 0x7f94300a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:37.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.758+0000 7f9446ffd640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f94300a3810 0x7f94300a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:37.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.758+0000 7f942ffff640 1 -- 172.21.15.16:0/504699565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f94400f2e40 con 0x7f9448102620 2024-03-27T06:43:37.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.759+0000 7f9446ffd640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f94300a3810 0x7f94300a5cd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9448101ac0 tx=0x7f9438005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:37.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.761+0000 7f942ffff640 1 -- 172.21.15.16:0/504699565 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f94400bf6a0 con 0x7f9448102620 2024-03-27T06:43:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.992+0000 7f944d581640 1 -- 172.21.15.16:0/504699565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f944802d490 con 0x7f9448102620 2024-03-27T06:43:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.993+0000 7f942ffff640 1 -- 172.21.15.16:0/504699565 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11932 (secure 0 0 0) 0x7f94400befa0 con 0x7f9448102620 2024-03-27T06:43:37.988 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:37.988 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:43:32.824696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24467,24527,24500,24472,24493]},"id":1}]} 2024-03-27T06:43:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-03-27T06:43:37.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 -- 172.21.15.16:0/504699565 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f94300a3810 msgr2=0x7f94300a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f94300a3810 0x7f94300a5cd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9448101ac0 tx=0x7f9438005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:37.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 -- 172.21.15.16:0/504699565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9448102620 msgr2=0x7f9448109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:37.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9448102620 0x7f9448109f70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f944002c630 tx=0x7f9440034670 comp rx=0 tx=0).stop 2024-03-27T06:43:37.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 -- 172.21.15.16:0/504699565 shutdown_connections 2024-03-27T06:43:37.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9448102620 0x7f9448109f70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9448101c60 0x7f9448109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f94300a3810 0x7f94300a5cd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 --2- 172.21.15.16:0/504699565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448100a70 0x7f944810b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:37.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 -- 172.21.15.16:0/504699565 >> 172.21.15.16:0/504699565 conn(0x7f94480fbfe0 msgr2=0x7f9448104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:37.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 -- 172.21.15.16:0/504699565 shutdown_connections 2024-03-27T06:43:37.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:37.995+0000 7f944d581640 1 -- 172.21.15.16:0/504699565 wait complete. 2024-03-27T06:43:38.001 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-27T06:43:38.001 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-27T06:43:38.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T06:43:38.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.112+0000 7f404e8ed640 1 -- 172.21.15.16:0/1899450304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4048102620 msgr2=0x7f4048108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.112+0000 7f404e8ed640 1 --2- 172.21.15.16:0/1899450304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4048102620 0x7f4048108eb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4040009ca0 tx=0x7f404002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.112+0000 7f404e8ed640 1 -- 172.21.15.16:0/1899450304 shutdown_connections 2024-03-27T06:43:38.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.112+0000 7f404e8ed640 1 --2- 172.21.15.16:0/1899450304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4048102620 0x7f4048108eb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.112+0000 7f404e8ed640 1 --2- 172.21.15.16:0/1899450304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4048101c60 0x7f40481020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.112+0000 7f404e8ed640 1 --2- 172.21.15.16:0/1899450304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4048100a70 0x7f4048100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.112+0000 7f404e8ed640 1 -- 172.21.15.16:0/1899450304 >> 172.21.15.16:0/1899450304 conn(0x7f40480fc000 msgr2=0x7f40480fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.112+0000 7f404e8ed640 1 -- 172.21.15.16:0/1899450304 shutdown_connections 2024-03-27T06:43:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.112+0000 7f404e8ed640 1 -- 172.21.15.16:0/1899450304 wait complete. 2024-03-27T06:43:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.112+0000 7f404e8ed640 1 Processor -- start 2024-03-27T06:43:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f404e8ed640 1 -- start start 2024-03-27T06:43:38.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f404e8ed640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4048100a70 0x7f4048196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f404e8ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4048101c60 0x7f4048197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f404e8ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4048102620 0x7f404819e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f404e8ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f404810c820 con 0x7f4048101c60 2024-03-27T06:43:38.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f404e8ed640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f404810c6a0 con 0x7f4048100a70 2024-03-27T06:43:38.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f404e8ed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f404810c9a0 con 0x7f4048102620 2024-03-27T06:43:38.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f40477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4048101c60 0x7f4048197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f40477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4048101c60 0x7f4048197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51154/0 (socket says 172.21.15.16:51154) 2024-03-27T06:43:38.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f40477fe640 1 -- 172.21.15.16:0/840354574 learned_addr learned my addr 172.21.15.16:0/840354574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f4047fff640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4048100a70 0x7f4048196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f404ce8a640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4048102620 0x7f404819e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f40477fe640 1 -- 172.21.15.16:0/840354574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4048102620 msgr2=0x7f404819e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f40477fe640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4048102620 0x7f404819e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f40477fe640 1 -- 172.21.15.16:0/840354574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4048100a70 msgr2=0x7f4048196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.113+0000 7f40477fe640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4048100a70 0x7f4048196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.114+0000 7f40477fe640 1 -- 172.21.15.16:0/840354574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404819ea00 con 0x7f4048101c60 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.114+0000 7f4047fff640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4048100a70 0x7f4048196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.114+0000 7f404ce8a640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4048102620 0x7f404819e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.114+0000 7f40477fe640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4048101c60 0x7f4048197320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f403c00eb00 tx=0x7f403c009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.114+0000 7f40457fa640 1 -- 172.21.15.16:0/840354574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c01c070 con 0x7f4048101c60 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.114+0000 7f40457fa640 1 -- 172.21.15.16:0/840354574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f403c00bc30 con 0x7f4048101c60 2024-03-27T06:43:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.114+0000 7f40457fa640 1 -- 172.21.15.16:0/840354574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c016730 con 0x7f4048101c60 2024-03-27T06:43:38.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.114+0000 7f404e8ed640 1 -- 172.21.15.16:0/840354574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404806a180 con 0x7f4048101c60 2024-03-27T06:43:38.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.114+0000 7f404e8ed640 1 -- 172.21.15.16:0/840354574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404806a6d0 con 0x7f4048101c60 2024-03-27T06:43:38.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.115+0000 7f404e8ed640 1 -- 172.21.15.16:0/840354574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4048068d90 con 0x7f4048101c60 2024-03-27T06:43:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.116+0000 7f40457fa640 1 -- 172.21.15.16:0/840354574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 138541+0+0 (secure 0 0 0) 0x7f403c01f070 con 0x7f4048101c60 2024-03-27T06:43:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.117+0000 7f40457fa640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f40300a3810 0x7f40300a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.117+0000 7f4047fff640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f40300a3810 0x7f40300a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.117+0000 7f40457fa640 1 -- 172.21.15.16:0/840354574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f403c0d17f0 con 0x7f4048101c60 2024-03-27T06:43:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.117+0000 7f4047fff640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f40300a3810 0x7f40300a5cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4048101ac0 tx=0x7f4038005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.122+0000 7f40457fa640 1 -- 172.21.15.16:0/840354574 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f403c00bde0 con 0x7f4048101c60 2024-03-27T06:43:38.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1600195556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:38.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: [27/Mar/2024:06:43:37] ENGINE Bus STARTING 2024-03-27T06:43:38.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: [27/Mar/2024:06:43:37] ENGINE Serving on http://172.21.15.16:8765 2024-03-27T06:43:38.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2628363166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:38.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.225 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi170 to 174.1M 2024-03-27T06:43:38.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: Unable to set osd_memory_target on smithi170 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-03-27T06:43:38.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-03-27T06:43:38.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: Unable to set osd_memory_target on smithi016 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-03-27T06:43:38.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: [27/Mar/2024:06:43:37] ENGINE Serving on https://172.21.15.16:7150 2024-03-27T06:43:38.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: [27/Mar/2024:06:43:37] ENGINE Bus STARTED 2024-03-27T06:43:38.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.229 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.229 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.229 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.229 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-27T06:43:38.229 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-27T06:43:38.229 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:43:38.229 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:43:38.230 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-27T06:43:38.230 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-27T06:43:38.230 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-27T06:43:38.230 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:37 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2302198908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.306+0000 7f404e8ed640 1 -- 172.21.15.16:0/840354574 --> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f404806a990 con 0x7f40300a3810 2024-03-27T06:43:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.317+0000 7f40457fa640 1 -- 172.21.15.16:0/840354574 <== mgr.15354 v2:172.21.15.16:6800/3715976676 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258017 (secure 0 0 0) 0x7f404806a990 con 0x7f40300a3810 2024-03-27T06:43:38.315 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:38.316 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-03-27T06:43:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.321+0000 7f404e8ed640 1 -- 172.21.15.16:0/840354574 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f40300a3810 msgr2=0x7f40300a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.321+0000 7f404e8ed640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f40300a3810 0x7f40300a5cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4048101ac0 tx=0x7f4038005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.321+0000 7f404e8ed640 1 -- 172.21.15.16:0/840354574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4048101c60 msgr2=0x7f4048197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.321+0000 7f404e8ed640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4048101c60 0x7f4048197320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f403c00eb00 tx=0x7f403c009f20 comp rx=0 tx=0).stop 2024-03-27T06:43:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.322+0000 7f404e8ed640 1 -- 172.21.15.16:0/840354574 shutdown_connections 2024-03-27T06:43:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.322+0000 7f404e8ed640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f40300a3810 0x7f40300a5cd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.322+0000 7f404e8ed640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4048102620 0x7f404819e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.322+0000 7f404e8ed640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4048101c60 0x7f4048197320 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.322+0000 7f404e8ed640 1 --2- 172.21.15.16:0/840354574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4048100a70 0x7f4048196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.322+0000 7f404e8ed640 1 -- 172.21.15.16:0/840354574 >> 172.21.15.16:0/840354574 conn(0x7f40480fc000 msgr2=0x7f4048104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:38.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.322+0000 7f404e8ed640 1 -- 172.21.15.16:0/840354574 shutdown_connections 2024-03-27T06:43:38.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.322+0000 7f404e8ed640 1 -- 172.21.15.16:0/840354574 wait complete. 2024-03-27T06:43:38.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1600195556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:38.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: [27/Mar/2024:06:43:37] ENGINE Bus STARTING 2024-03-27T06:43:38.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: [27/Mar/2024:06:43:37] ENGINE Serving on http://172.21.15.16:8765 2024-03-27T06:43:38.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2628363166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:38.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi170 to 174.1M 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: Unable to set osd_memory_target on smithi170 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: Unable to set osd_memory_target on smithi016 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: [27/Mar/2024:06:43:37] ENGINE Serving on https://172.21.15.16:7150 2024-03-27T06:43:38.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: [27/Mar/2024:06:43:37] ENGINE Bus STARTED 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-27T06:43:38.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:37 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2302198908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:38.334 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-03-27T06:43:37.799887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191594,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1471,"num_read_kb":902,"num_write":557,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1180960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":942,"ondisk_log_size":942,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369548,"kb_used_data":12052,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572276,"statfs":{"total":1151940427776,"available":1151562010624,"internally_reserved":0,"allocated":12341248,"data_stored":9141984,"data_compressed":205493,"data_compressed_allocated":266240,"data_compressed_original":3383296,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.986546"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044192+0000","last_change":"2024-03-27T06:41:29.408953+0000","last_active":"2024-03-27T06:43:34.044192+0000","last_peered":"2024-03-27T06:43:34.044192+0000","last_clean":"2024-03-27T06:43:34.044192+0000","last_became_active":"2024-03-27T06:41:29.408143+0000","last_became_peered":"2024-03-27T06:41:29.408143+0000","last_unstale":"2024-03-27T06:43:34.044192+0000","last_undegraded":"2024-03-27T06:43:34.044192+0000","last_fullsized":"2024-03-27T06:43:34.044192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:22.486951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044362+0000","last_change":"2024-03-27T06:41:29.405080+0000","last_active":"2024-03-27T06:43:34.044362+0000","last_peered":"2024-03-27T06:43:34.044362+0000","last_clean":"2024-03-27T06:43:34.044362+0000","last_became_active":"2024-03-27T06:41:29.404743+0000","last_became_peered":"2024-03-27T06:41:29.404743+0000","last_unstale":"2024-03-27T06:43:34.044362+0000","last_undegraded":"2024-03-27T06:43:34.044362+0000","last_fullsized":"2024-03-27T06:43:34.044362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:24:30.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044526+0000","last_change":"2024-03-27T06:41:29.409346+0000","last_active":"2024-03-27T06:43:34.044526+0000","last_peered":"2024-03-27T06:43:34.044526+0000","last_clean":"2024-03-27T06:43:34.044526+0000","last_became_active":"2024-03-27T06:41:29.408837+0000","last_became_peered":"2024-03-27T06:41:29.408837+0000","last_unstale":"2024-03-27T06:43:34.044526+0000","last_undegraded":"2024-03-27T06:43:34.044526+0000","last_fullsized":"2024-03-27T06:43:34.044526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:11:12.274869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052302+0000","last_change":"2024-03-27T06:41:29.412115+0000","last_active":"2024-03-27T06:43:34.052302+0000","last_peered":"2024-03-27T06:43:34.052302+0000","last_clean":"2024-03-27T06:43:34.052302+0000","last_became_active":"2024-03-27T06:41:29.411702+0000","last_became_peered":"2024-03-27T06:41:29.411702+0000","last_unstale":"2024-03-27T06:43:34.052302+0000","last_undegraded":"2024-03-27T06:43:34.052302+0000","last_fullsized":"2024-03-27T06:43:34.052302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:04:42.347229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890812+0000","last_change":"2024-03-27T06:41:29.409888+0000","last_active":"2024-03-27T06:43:33.890812+0000","last_peered":"2024-03-27T06:43:33.890812+0000","last_clean":"2024-03-27T06:43:33.890812+0000","last_became_active":"2024-03-27T06:41:29.409558+0000","last_became_peered":"2024-03-27T06:41:29.409558+0000","last_unstale":"2024-03-27T06:43:33.890812+0000","last_undegraded":"2024-03-27T06:43:33.890812+0000","last_fullsized":"2024-03-27T06:43:33.890812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:38:51.452793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902755+0000","last_change":"2024-03-27T06:41:29.404088+0000","last_active":"2024-03-27T06:43:33.902755+0000","last_peered":"2024-03-27T06:43:33.902755+0000","last_clean":"2024-03-27T06:43:33.902755+0000","last_became_active":"2024-03-27T06:41:29.403698+0000","last_became_peered":"2024-03-27T06:41:29.403698+0000","last_unstale":"2024-03-27T06:43:33.902755+0000","last_undegraded":"2024-03-27T06:43:33.902755+0000","last_fullsized":"2024-03-27T06:43:33.902755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:14:31.296523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788203+0000","last_change":"2024-03-27T06:41:29.413157+0000","last_active":"2024-03-27T06:43:33.788203+0000","last_peered":"2024-03-27T06:43:33.788203+0000","last_clean":"2024-03-27T06:43:33.788203+0000","last_became_active":"2024-03-27T06:41:29.412669+0000","last_became_peered":"2024-03-27T06:41:29.412669+0000","last_unstale":"2024-03-27T06:43:33.788203+0000","last_undegraded":"2024-03-27T06:43:33.788203+0000","last_fullsized":"2024-03-27T06:43:33.788203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:28:20.823770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890507+0000","last_change":"2024-03-27T06:41:29.417207+0000","last_active":"2024-03-27T06:43:33.890507+0000","last_peered":"2024-03-27T06:43:33.890507+0000","last_clean":"2024-03-27T06:43:33.890507+0000","last_became_active":"2024-03-27T06:41:29.415556+0000","last_became_peered":"2024-03-27T06:41:29.415556+0000","last_unstale":"2024-03-27T06:43:33.890507+0000","last_undegraded":"2024-03-27T06:43:33.890507+0000","last_fullsized":"2024-03-27T06:43:33.890507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:26:04.040296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.053665+0000","last_change":"2024-03-27T06:41:29.400032+0000","last_active":"2024-03-27T06:43:34.053665+0000","last_peered":"2024-03-27T06:43:34.053665+0000","last_clean":"2024-03-27T06:43:34.053665+0000","last_became_active":"2024-03-27T06:41:29.399274+0000","last_became_peered":"2024-03-27T06:41:29.399274+0000","last_unstale":"2024-03-27T06:43:34.053665+0000","last_undegraded":"2024-03-27T06:43:34.053665+0000","last_fullsized":"2024-03-27T06:43:34.053665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:55.306309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.903052+0000","last_change":"2024-03-27T06:41:29.412471+0000","last_active":"2024-03-27T06:43:33.903052+0000","last_peered":"2024-03-27T06:43:33.903052+0000","last_clean":"2024-03-27T06:43:33.903052+0000","last_became_active":"2024-03-27T06:41:29.412134+0000","last_became_peered":"2024-03-27T06:41:29.412134+0000","last_unstale":"2024-03-27T06:43:33.903052+0000","last_undegraded":"2024-03-27T06:43:33.903052+0000","last_fullsized":"2024-03-27T06:43:33.903052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:35:57.146944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788956+0000","last_change":"2024-03-27T06:41:29.407915+0000","last_active":"2024-03-27T06:43:33.788956+0000","last_peered":"2024-03-27T06:43:33.788956+0000","last_clean":"2024-03-27T06:43:33.788956+0000","last_became_active":"2024-03-27T06:41:29.407563+0000","last_became_peered":"2024-03-27T06:41:29.407563+0000","last_unstale":"2024-03-27T06:43:33.788956+0000","last_undegraded":"2024-03-27T06:43:33.788956+0000","last_fullsized":"2024-03-27T06:43:33.788956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:09:05.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052028+0000","last_change":"2024-03-27T06:41:29.413374+0000","last_active":"2024-03-27T06:43:34.052028+0000","last_peered":"2024-03-27T06:43:34.052028+0000","last_clean":"2024-03-27T06:43:34.052028+0000","last_became_active":"2024-03-27T06:41:29.413010+0000","last_became_peered":"2024-03-27T06:41:29.413010+0000","last_unstale":"2024-03-27T06:43:34.052028+0000","last_undegraded":"2024-03-27T06:43:34.052028+0000","last_fullsized":"2024-03-27T06:43:34.052028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:48:52.329611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050925+0000","last_change":"2024-03-27T06:41:29.408640+0000","last_active":"2024-03-27T06:43:34.050925+0000","last_peered":"2024-03-27T06:43:34.050925+0000","last_clean":"2024-03-27T06:43:34.050925+0000","last_became_active":"2024-03-27T06:41:29.408268+0000","last_became_peered":"2024-03-27T06:41:29.408268+0000","last_unstale":"2024-03-27T06:43:34.050925+0000","last_undegraded":"2024-03-27T06:43:34.050925+0000","last_fullsized":"2024-03-27T06:43:34.050925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:51:59.803596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043668+0000","last_change":"2024-03-27T06:41:29.420436+0000","last_active":"2024-03-27T06:43:34.043668+0000","last_peered":"2024-03-27T06:43:34.043668+0000","last_clean":"2024-03-27T06:43:34.043668+0000","last_became_active":"2024-03-27T06:41:29.419426+0000","last_became_peered":"2024-03-27T06:41:29.419426+0000","last_unstale":"2024-03-27T06:43:34.043668+0000","last_undegraded":"2024-03-27T06:43:34.043668+0000","last_fullsized":"2024-03-27T06:43:34.043668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:38:15.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043640+0000","last_change":"2024-03-27T06:41:29.415673+0000","last_active":"2024-03-27T06:43:34.043640+0000","last_peered":"2024-03-27T06:43:34.043640+0000","last_clean":"2024-03-27T06:43:34.043640+0000","last_became_active":"2024-03-27T06:41:29.414741+0000","last_became_peered":"2024-03-27T06:41:29.414741+0000","last_unstale":"2024-03-27T06:43:34.043640+0000","last_undegraded":"2024-03-27T06:43:34.043640+0000","last_fullsized":"2024-03-27T06:43:34.043640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:38:43.183517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051498+0000","last_change":"2024-03-27T06:41:29.408609+0000","last_active":"2024-03-27T06:43:34.051498+0000","last_peered":"2024-03-27T06:43:34.051498+0000","last_clean":"2024-03-27T06:43:34.051498+0000","last_became_active":"2024-03-27T06:41:29.408252+0000","last_became_peered":"2024-03-27T06:41:29.408252+0000","last_unstale":"2024-03-27T06:43:34.051498+0000","last_undegraded":"2024-03-27T06:43:34.051498+0000","last_fullsized":"2024-03-27T06:43:34.051498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:14:13.609113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044330+0000","last_change":"2024-03-27T06:41:29.409221+0000","last_active":"2024-03-27T06:43:34.044330+0000","last_peered":"2024-03-27T06:43:34.044330+0000","last_clean":"2024-03-27T06:43:34.044330+0000","last_became_active":"2024-03-27T06:41:29.408598+0000","last_became_peered":"2024-03-27T06:41:29.408598+0000","last_unstale":"2024-03-27T06:43:34.044330+0000","last_undegraded":"2024-03-27T06:43:34.044330+0000","last_fullsized":"2024-03-27T06:43:34.044330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:42:16.251149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.786349+0000","last_change":"2024-03-27T06:41:29.406143+0000","last_active":"2024-03-27T06:43:33.786349+0000","last_peered":"2024-03-27T06:43:33.786349+0000","last_clean":"2024-03-27T06:43:33.786349+0000","last_became_active":"2024-03-27T06:41:29.405754+0000","last_became_peered":"2024-03-27T06:41:29.405754+0000","last_unstale":"2024-03-27T06:43:33.786349+0000","last_undegraded":"2024-03-27T06:43:33.786349+0000","last_fullsized":"2024-03-27T06:43:33.786349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:42:47.787099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788822+0000","last_change":"2024-03-27T06:41:29.402471+0000","last_active":"2024-03-27T06:43:33.788822+0000","last_peered":"2024-03-27T06:43:33.788822+0000","last_clean":"2024-03-27T06:43:33.788822+0000","last_became_active":"2024-03-27T06:41:29.402113+0000","last_became_peered":"2024-03-27T06:41:29.402113+0000","last_unstale":"2024-03-27T06:43:33.788822+0000","last_undegraded":"2024-03-27T06:43:33.788822+0000","last_fullsized":"2024-03-27T06:43:33.788822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:34:16.408058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'14","reported_seq":157,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607533+0000","last_change":"2024-03-27T06:41:28.397815+0000","last_active":"2024-03-27T06:43:34.607533+0000","last_peered":"2024-03-27T06:43:34.607533+0000","last_clean":"2024-03-27T06:43:34.607533+0000","last_became_active":"2024-03-27T06:41:28.397503+0000","last_became_peered":"2024-03-27T06:41:28.397503+0000","last_unstale":"2024-03-27T06:43:34.607533+0000","last_undegraded":"2024-03-27T06:43:34.607533+0000","last_fullsized":"2024-03-27T06:43:34.607533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:15:53.751809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890734+0000","last_change":"2024-03-27T06:41:29.419528+0000","last_active":"2024-03-27T06:43:33.890734+0000","last_peered":"2024-03-27T06:43:33.890734+0000","last_clean":"2024-03-27T06:43:33.890734+0000","last_became_active":"2024-03-27T06:41:29.417091+0000","last_became_peered":"2024-03-27T06:41:29.417091+0000","last_unstale":"2024-03-27T06:43:33.890734+0000","last_undegraded":"2024-03-27T06:43:33.890734+0000","last_fullsized":"2024-03-27T06:43:33.890734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:13:14.629211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891107+0000","last_change":"2024-03-27T06:41:28.400468+0000","last_active":"2024-03-27T06:43:33.891107+0000","last_peered":"2024-03-27T06:43:33.891107+0000","last_clean":"2024-03-27T06:43:33.891107+0000","last_became_active":"2024-03-27T06:41:28.400190+0000","last_became_peered":"2024-03-27T06:41:28.400190+0000","last_unstale":"2024-03-27T06:43:33.891107+0000","last_undegraded":"2024-03-27T06:43:33.891107+0000","last_fullsized":"2024-03-27T06:43:33.891107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:24:17.059874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.045055+0000","last_change":"2024-03-27T06:41:29.409364+0000","last_active":"2024-03-27T06:43:34.045055+0000","last_peered":"2024-03-27T06:43:34.045055+0000","last_clean":"2024-03-27T06:43:34.045055+0000","last_became_active":"2024-03-27T06:41:29.408946+0000","last_became_peered":"2024-03-27T06:41:29.408946+0000","last_unstale":"2024-03-27T06:43:34.045055+0000","last_undegraded":"2024-03-27T06:43:34.045055+0000","last_fullsized":"2024-03-27T06:43:34.045055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:46:17.132071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890918+0000","last_change":"2024-03-27T06:41:28.401327+0000","last_active":"2024-03-27T06:43:33.890918+0000","last_peered":"2024-03-27T06:43:33.890918+0000","last_clean":"2024-03-27T06:43:33.890918+0000","last_became_active":"2024-03-27T06:41:28.400524+0000","last_became_peered":"2024-03-27T06:41:28.400524+0000","last_unstale":"2024-03-27T06:43:33.890918+0000","last_undegraded":"2024-03-27T06:43:33.890918+0000","last_fullsized":"2024-03-27T06:43:33.890918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:11:52.608690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051934+0000","last_change":"2024-03-27T06:41:29.403042+0000","last_active":"2024-03-27T06:43:34.051934+0000","last_peered":"2024-03-27T06:43:34.051934+0000","last_clean":"2024-03-27T06:43:34.051934+0000","last_became_active":"2024-03-27T06:41:29.402516+0000","last_became_peered":"2024-03-27T06:41:29.402516+0000","last_unstale":"2024-03-27T06:43:34.051934+0000","last_undegraded":"2024-03-27T06:43:34.051934+0000","last_fullsized":"2024-03-27T06:43:34.051934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:20:55.777498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891130+0000","last_change":"2024-03-27T06:41:28.395155+0000","last_active":"2024-03-27T06:43:33.891130+0000","last_peered":"2024-03-27T06:43:33.891130+0000","last_clean":"2024-03-27T06:43:33.891130+0000","last_became_active":"2024-03-27T06:41:28.394760+0000","last_became_peered":"2024-03-27T06:41:28.394760+0000","last_unstale":"2024-03-27T06:43:33.891130+0000","last_undegraded":"2024-03-27T06:43:33.891130+0000","last_fullsized":"2024-03-27T06:43:33.891130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:15:28.182581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899706+0000","last_change":"2024-03-27T06:41:29.409747+0000","last_active":"2024-03-27T06:43:33.899706+0000","last_peered":"2024-03-27T06:43:33.899706+0000","last_clean":"2024-03-27T06:43:33.899706+0000","last_became_active":"2024-03-27T06:41:29.409425+0000","last_became_peered":"2024-03-27T06:41:29.409425+0000","last_unstale":"2024-03-27T06:43:33.899706+0000","last_undegraded":"2024-03-27T06:43:33.899706+0000","last_fullsized":"2024-03-27T06:43:33.899706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:36.181279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.605066+0000","last_change":"2024-03-27T06:41:28.394726+0000","last_active":"2024-03-27T06:43:34.605066+0000","last_peered":"2024-03-27T06:43:34.605066+0000","last_clean":"2024-03-27T06:43:34.605066+0000","last_became_active":"2024-03-27T06:41:28.394378+0000","last_became_peered":"2024-03-27T06:41:28.394378+0000","last_unstale":"2024-03-27T06:43:34.605066+0000","last_undegraded":"2024-03-27T06:43:34.605066+0000","last_fullsized":"2024-03-27T06:43:34.605066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:30:22.194211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890573+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:43:33.890573+0000","last_peered":"2024-03-27T06:43:33.890573+0000","last_clean":"2024-03-27T06:43:33.890573+0000","last_became_active":"2024-03-27T06:41:29.408826+0000","last_became_peered":"2024-03-27T06:41:29.408826+0000","last_unstale":"2024-03-27T06:43:33.890573+0000","last_undegraded":"2024-03-27T06:43:33.890573+0000","last_fullsized":"2024-03-27T06:43:33.890573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:50:53.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902801+0000","last_change":"2024-03-27T06:41:28.393655+0000","last_active":"2024-03-27T06:43:33.902801+0000","last_peered":"2024-03-27T06:43:33.902801+0000","last_clean":"2024-03-27T06:43:33.902801+0000","last_became_active":"2024-03-27T06:41:28.393245+0000","last_became_peered":"2024-03-27T06:41:28.393245+0000","last_unstale":"2024-03-27T06:43:33.902801+0000","last_undegraded":"2024-03-27T06:43:33.902801+0000","last_fullsized":"2024-03-27T06:43:33.902801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:58:21.169444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899501+0000","last_change":"2024-03-27T06:41:29.417730+0000","last_active":"2024-03-27T06:43:33.899501+0000","last_peered":"2024-03-27T06:43:33.899501+0000","last_clean":"2024-03-27T06:43:33.899501+0000","last_became_active":"2024-03-27T06:41:29.417319+0000","last_became_peered":"2024-03-27T06:41:29.417319+0000","last_unstale":"2024-03-27T06:43:33.899501+0000","last_undegraded":"2024-03-27T06:43:33.899501+0000","last_fullsized":"2024-03-27T06:43:33.899501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:25.224220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050475+0000","last_change":"2024-03-27T06:41:28.402259+0000","last_active":"2024-03-27T06:43:34.050475+0000","last_peered":"2024-03-27T06:43:34.050475+0000","last_clean":"2024-03-27T06:43:34.050475+0000","last_became_active":"2024-03-27T06:41:28.401922+0000","last_became_peered":"2024-03-27T06:41:28.401922+0000","last_unstale":"2024-03-27T06:43:34.050475+0000","last_undegraded":"2024-03-27T06:43:34.050475+0000","last_fullsized":"2024-03-27T06:43:34.050475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:57:33.245183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890562+0000","last_change":"2024-03-27T06:41:29.409034+0000","last_active":"2024-03-27T06:43:33.890562+0000","last_peered":"2024-03-27T06:43:33.890562+0000","last_clean":"2024-03-27T06:43:33.890562+0000","last_became_active":"2024-03-27T06:41:29.408666+0000","last_became_peered":"2024-03-27T06:41:29.408666+0000","last_unstale":"2024-03-27T06:43:33.890562+0000","last_undegraded":"2024-03-27T06:43:33.890562+0000","last_fullsized":"2024-03-27T06:43:33.890562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:09:55.532059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899143+0000","last_change":"2024-03-27T06:41:28.393802+0000","last_active":"2024-03-27T06:43:33.899143+0000","last_peered":"2024-03-27T06:43:33.899143+0000","last_clean":"2024-03-27T06:43:33.899143+0000","last_became_active":"2024-03-27T06:41:28.393503+0000","last_became_peered":"2024-03-27T06:41:28.393503+0000","last_unstale":"2024-03-27T06:43:33.899143+0000","last_undegraded":"2024-03-27T06:43:33.899143+0000","last_fullsized":"2024-03-27T06:43:33.899143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:17.765709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.786232+0000","last_change":"2024-03-27T06:41:29.416557+0000","last_active":"2024-03-27T06:43:33.786232+0000","last_peered":"2024-03-27T06:43:33.786232+0000","last_clean":"2024-03-27T06:43:33.786232+0000","last_became_active":"2024-03-27T06:41:29.416223+0000","last_became_peered":"2024-03-27T06:41:29.416223+0000","last_unstale":"2024-03-27T06:43:33.786232+0000","last_undegraded":"2024-03-27T06:43:33.786232+0000","last_fullsized":"2024-03-27T06:43:33.786232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:54:20.929325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":268,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.604510+0000","last_change":"2024-03-27T06:41:28.396014+0000","last_active":"2024-03-27T06:43:34.604510+0000","last_peered":"2024-03-27T06:43:34.604510+0000","last_clean":"2024-03-27T06:43:34.604510+0000","last_became_active":"2024-03-27T06:41:28.395699+0000","last_became_peered":"2024-03-27T06:41:28.395699+0000","last_unstale":"2024-03-27T06:43:34.604510+0000","last_undegraded":"2024-03-27T06:43:34.604510+0000","last_fullsized":"2024-03-27T06:43:34.604510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:00:59.679843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.053523+0000","last_change":"2024-03-27T06:41:29.401668+0000","last_active":"2024-03-27T06:43:34.053523+0000","last_peered":"2024-03-27T06:43:34.053523+0000","last_clean":"2024-03-27T06:43:34.053523+0000","last_became_active":"2024-03-27T06:41:29.401328+0000","last_became_peered":"2024-03-27T06:41:29.401328+0000","last_unstale":"2024-03-27T06:43:34.053523+0000","last_undegraded":"2024-03-27T06:43:34.053523+0000","last_fullsized":"2024-03-27T06:43:34.053523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:32:54.273194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789142+0000","last_change":"2024-03-27T06:42:21.011596+0000","last_active":"2024-03-27T06:43:33.789142+0000","last_peered":"2024-03-27T06:43:33.789142+0000","last_clean":"2024-03-27T06:43:33.789142+0000","last_became_active":"2024-03-27T06:42:21.005035+0000","last_became_peered":"2024-03-27T06:42:21.005035+0000","last_unstale":"2024-03-27T06:43:33.789142+0000","last_undegraded":"2024-03-27T06:43:33.789142+0000","last_fullsized":"2024-03-27T06:43:33.789142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:51:35.652266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044846+0000","last_change":"2024-03-27T06:41:29.413919+0000","last_active":"2024-03-27T06:43:34.044846+0000","last_peered":"2024-03-27T06:43:34.044846+0000","last_clean":"2024-03-27T06:43:34.044846+0000","last_became_active":"2024-03-27T06:41:29.413566+0000","last_became_peered":"2024-03-27T06:41:29.413566+0000","last_unstale":"2024-03-27T06:43:34.044846+0000","last_undegraded":"2024-03-27T06:43:34.044846+0000","last_fullsized":"2024-03-27T06:43:34.044846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:45:03.705372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899758+0000","last_change":"2024-03-27T06:41:28.391661+0000","last_active":"2024-03-27T06:43:33.899758+0000","last_peered":"2024-03-27T06:43:33.899758+0000","last_clean":"2024-03-27T06:43:33.899758+0000","last_became_active":"2024-03-27T06:41:28.391230+0000","last_became_peered":"2024-03-27T06:41:28.391230+0000","last_unstale":"2024-03-27T06:43:33.899758+0000","last_undegraded":"2024-03-27T06:43:33.899758+0000","last_fullsized":"2024-03-27T06:43:33.899758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:41:22.789267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052236+0000","last_change":"2024-03-27T06:41:29.405727+0000","last_active":"2024-03-27T06:43:34.052236+0000","last_peered":"2024-03-27T06:43:34.052236+0000","last_clean":"2024-03-27T06:43:34.052236+0000","last_became_active":"2024-03-27T06:41:29.405413+0000","last_became_peered":"2024-03-27T06:41:29.405413+0000","last_unstale":"2024-03-27T06:43:34.052236+0000","last_undegraded":"2024-03-27T06:43:34.052236+0000","last_fullsized":"2024-03-27T06:43:34.052236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:35:21.575729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":308,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.605048+0000","last_change":"2024-03-27T06:41:28.391155+0000","last_active":"2024-03-27T06:43:34.605048+0000","last_peered":"2024-03-27T06:43:34.605048+0000","last_clean":"2024-03-27T06:43:34.605048+0000","last_became_active":"2024-03-27T06:41:28.390828+0000","last_became_peered":"2024-03-27T06:41:28.390828+0000","last_unstale":"2024-03-27T06:43:34.605048+0000","last_undegraded":"2024-03-27T06:43:34.605048+0000","last_fullsized":"2024-03-27T06:43:34.605048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:40.192560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66735,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":190,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890788+0000","last_change":"2024-03-27T06:41:29.413283+0000","last_active":"2024-03-27T06:43:33.890788+0000","last_peered":"2024-03-27T06:43:33.890788+0000","last_clean":"2024-03-27T06:43:33.890788+0000","last_became_active":"2024-03-27T06:41:29.412247+0000","last_became_peered":"2024-03-27T06:41:29.412247+0000","last_unstale":"2024-03-27T06:43:33.890788+0000","last_undegraded":"2024-03-27T06:43:33.890788+0000","last_fullsized":"2024-03-27T06:43:33.890788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:50:15.336003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788566+0000","last_change":"2024-03-27T06:41:28.395089+0000","last_active":"2024-03-27T06:43:33.788566+0000","last_peered":"2024-03-27T06:43:33.788566+0000","last_clean":"2024-03-27T06:43:33.788566+0000","last_became_active":"2024-03-27T06:41:28.394145+0000","last_became_peered":"2024-03-27T06:41:28.394145+0000","last_unstale":"2024-03-27T06:43:33.788566+0000","last_undegraded":"2024-03-27T06:43:33.788566+0000","last_fullsized":"2024-03-27T06:43:33.788566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:40.858404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050686+0000","last_change":"2024-03-27T06:41:29.416955+0000","last_active":"2024-03-27T06:43:34.050686+0000","last_peered":"2024-03-27T06:43:34.050686+0000","last_clean":"2024-03-27T06:43:34.050686+0000","last_became_active":"2024-03-27T06:41:29.416616+0000","last_became_peered":"2024-03-27T06:41:29.416616+0000","last_unstale":"2024-03-27T06:43:34.050686+0000","last_undegraded":"2024-03-27T06:43:34.050686+0000","last_fullsized":"2024-03-27T06:43:34.050686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:38:18.791602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899329+0000","last_change":"2024-03-27T06:41:28.393876+0000","last_active":"2024-03-27T06:43:33.899329+0000","last_peered":"2024-03-27T06:43:33.899329+0000","last_clean":"2024-03-27T06:43:33.899329+0000","last_became_active":"2024-03-27T06:41:28.393557+0000","last_became_peered":"2024-03-27T06:41:28.393557+0000","last_unstale":"2024-03-27T06:43:33.899329+0000","last_undegraded":"2024-03-27T06:43:33.899329+0000","last_fullsized":"2024-03-27T06:43:33.899329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:30.831039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788517+0000","last_change":"2024-03-27T06:42:20.660026+0000","last_active":"2024-03-27T06:43:33.788517+0000","last_peered":"2024-03-27T06:43:33.788517+0000","last_clean":"2024-03-27T06:43:33.788517+0000","last_became_active":"2024-03-27T06:42:20.659628+0000","last_became_peered":"2024-03-27T06:42:20.659628+0000","last_unstale":"2024-03-27T06:43:33.788517+0000","last_undegraded":"2024-03-27T06:43:33.788517+0000","last_fullsized":"2024-03-27T06:43:33.788517+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:18:59.396472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607543+0000","last_change":"2024-03-27T06:41:28.393849+0000","last_active":"2024-03-27T06:43:34.607543+0000","last_peered":"2024-03-27T06:43:34.607543+0000","last_clean":"2024-03-27T06:43:34.607543+0000","last_became_active":"2024-03-27T06:41:28.393529+0000","last_became_peered":"2024-03-27T06:41:28.393529+0000","last_unstale":"2024-03-27T06:43:34.607543+0000","last_undegraded":"2024-03-27T06:43:34.607543+0000","last_fullsized":"2024-03-27T06:43:34.607543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:24:23.251723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.787942+0000","last_change":"2024-03-27T06:41:29.411273+0000","last_active":"2024-03-27T06:43:33.787942+0000","last_peered":"2024-03-27T06:43:33.787942+0000","last_clean":"2024-03-27T06:43:33.787942+0000","last_became_active":"2024-03-27T06:41:29.410936+0000","last_became_peered":"2024-03-27T06:41:29.410936+0000","last_unstale":"2024-03-27T06:43:33.787942+0000","last_undegraded":"2024-03-27T06:43:33.787942+0000","last_fullsized":"2024-03-27T06:43:33.787942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:41.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043527+0000","last_change":"2024-03-27T06:42:20.673735+0000","last_active":"2024-03-27T06:43:34.043527+0000","last_peered":"2024-03-27T06:43:34.043527+0000","last_clean":"2024-03-27T06:43:34.043527+0000","last_became_active":"2024-03-27T06:42:20.655144+0000","last_became_peered":"2024-03-27T06:42:20.655144+0000","last_unstale":"2024-03-27T06:43:34.043527+0000","last_undegraded":"2024-03-27T06:43:34.043527+0000","last_fullsized":"2024-03-27T06:43:34.043527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:05:39.059633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899135+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:43:33.899135+0000","last_peered":"2024-03-27T06:43:33.899135+0000","last_clean":"2024-03-27T06:43:33.899135+0000","last_became_active":"2024-03-27T06:41:29.408815+0000","last_became_peered":"2024-03-27T06:41:29.408815+0000","last_unstale":"2024-03-27T06:43:33.899135+0000","last_undegraded":"2024-03-27T06:43:33.899135+0000","last_fullsized":"2024-03-27T06:43:33.899135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:45.547747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891036+0000","last_change":"2024-03-27T06:41:28.398249+0000","last_active":"2024-03-27T06:43:33.891036+0000","last_peered":"2024-03-27T06:43:33.891036+0000","last_clean":"2024-03-27T06:43:33.891036+0000","last_became_active":"2024-03-27T06:41:28.397176+0000","last_became_peered":"2024-03-27T06:41:28.397176+0000","last_unstale":"2024-03-27T06:43:33.891036+0000","last_undegraded":"2024-03-27T06:43:33.891036+0000","last_fullsized":"2024-03-27T06:43:33.891036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:32.614558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044019+0000","last_change":"2024-03-27T06:41:29.412928+0000","last_active":"2024-03-27T06:43:34.044019+0000","last_peered":"2024-03-27T06:43:34.044019+0000","last_clean":"2024-03-27T06:43:34.044019+0000","last_became_active":"2024-03-27T06:41:29.412576+0000","last_became_peered":"2024-03-27T06:41:29.412576+0000","last_unstale":"2024-03-27T06:43:34.044019+0000","last_undegraded":"2024-03-27T06:43:34.044019+0000","last_fullsized":"2024-03-27T06:43:34.044019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:01:22.389424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.786409+0000","last_change":"2024-03-27T06:41:28.397135+0000","last_active":"2024-03-27T06:43:33.786409+0000","last_peered":"2024-03-27T06:43:33.786409+0000","last_clean":"2024-03-27T06:43:33.786409+0000","last_became_active":"2024-03-27T06:41:28.396483+0000","last_became_peered":"2024-03-27T06:41:28.396483+0000","last_unstale":"2024-03-27T06:43:33.786409+0000","last_undegraded":"2024-03-27T06:43:33.786409+0000","last_fullsized":"2024-03-27T06:43:33.786409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:05:02.038024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902974+0000","last_change":"2024-03-27T06:41:29.418629+0000","last_active":"2024-03-27T06:43:33.902974+0000","last_peered":"2024-03-27T06:43:33.902974+0000","last_clean":"2024-03-27T06:43:33.902974+0000","last_became_active":"2024-03-27T06:41:29.418278+0000","last_became_peered":"2024-03-27T06:41:29.418278+0000","last_unstale":"2024-03-27T06:43:33.902974+0000","last_undegraded":"2024-03-27T06:43:33.902974+0000","last_fullsized":"2024-03-27T06:43:33.902974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:33:54.461349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.608536+0000","last_change":"2024-03-27T06:41:28.400611+0000","last_active":"2024-03-27T06:43:34.608536+0000","last_peered":"2024-03-27T06:43:34.608536+0000","last_clean":"2024-03-27T06:43:34.608536+0000","last_became_active":"2024-03-27T06:41:28.400211+0000","last_became_peered":"2024-03-27T06:41:28.400211+0000","last_unstale":"2024-03-27T06:43:34.608536+0000","last_undegraded":"2024-03-27T06:43:34.608536+0000","last_fullsized":"2024-03-27T06:43:34.608536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:46:35.491071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902979+0000","last_change":"2024-03-27T06:41:29.413246+0000","last_active":"2024-03-27T06:43:33.902979+0000","last_peered":"2024-03-27T06:43:33.902979+0000","last_clean":"2024-03-27T06:43:33.902979+0000","last_became_active":"2024-03-27T06:41:29.412946+0000","last_became_peered":"2024-03-27T06:41:29.412946+0000","last_unstale":"2024-03-27T06:43:33.902979+0000","last_undegraded":"2024-03-27T06:43:33.902979+0000","last_fullsized":"2024-03-27T06:43:33.902979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:41:51.441233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899393+0000","last_change":"2024-03-27T06:41:29.402160+0000","last_active":"2024-03-27T06:43:33.899393+0000","last_peered":"2024-03-27T06:43:33.899393+0000","last_clean":"2024-03-27T06:43:33.899393+0000","last_became_active":"2024-03-27T06:41:29.401789+0000","last_became_peered":"2024-03-27T06:41:29.401789+0000","last_unstale":"2024-03-27T06:43:33.899393+0000","last_undegraded":"2024-03-27T06:43:33.899393+0000","last_fullsized":"2024-03-27T06:43:33.899393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:31:27.821966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607060+0000","last_change":"2024-03-27T06:41:28.395978+0000","last_active":"2024-03-27T06:43:34.607060+0000","last_peered":"2024-03-27T06:43:34.607060+0000","last_clean":"2024-03-27T06:43:34.607060+0000","last_became_active":"2024-03-27T06:41:28.395650+0000","last_became_peered":"2024-03-27T06:41:28.395650+0000","last_unstale":"2024-03-27T06:43:34.607060+0000","last_undegraded":"2024-03-27T06:43:34.607060+0000","last_fullsized":"2024-03-27T06:43:34.607060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:01:15.841344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890934+0000","last_change":"2024-03-27T06:41:29.423303+0000","last_active":"2024-03-27T06:43:33.890934+0000","last_peered":"2024-03-27T06:43:33.890934+0000","last_clean":"2024-03-27T06:43:33.890934+0000","last_became_active":"2024-03-27T06:41:29.422977+0000","last_became_peered":"2024-03-27T06:41:29.422977+0000","last_unstale":"2024-03-27T06:43:33.890934+0000","last_undegraded":"2024-03-27T06:43:33.890934+0000","last_fullsized":"2024-03-27T06:43:33.890934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:14:01.877801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051856+0000","last_change":"2024-03-27T06:41:28.395141+0000","last_active":"2024-03-27T06:43:34.051856+0000","last_peered":"2024-03-27T06:43:34.051856+0000","last_clean":"2024-03-27T06:43:34.051856+0000","last_became_active":"2024-03-27T06:41:28.394826+0000","last_became_peered":"2024-03-27T06:41:28.394826+0000","last_unstale":"2024-03-27T06:43:34.051856+0000","last_undegraded":"2024-03-27T06:43:34.051856+0000","last_fullsized":"2024-03-27T06:43:34.051856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:29:49.997090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788039+0000","last_change":"2024-03-27T06:40:16.460939+0000","last_active":"2024-03-27T06:43:33.788039+0000","last_peered":"2024-03-27T06:43:33.788039+0000","last_clean":"2024-03-27T06:43:33.788039+0000","last_became_active":"2024-03-27T06:40:16.460453+0000","last_became_peered":"2024-03-27T06:40:16.460453+0000","last_unstale":"2024-03-27T06:43:33.788039+0000","last_undegraded":"2024-03-27T06:43:33.788039+0000","last_fullsized":"2024-03-27T06:43:33.788039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_clean_scrub_stamp":"2024-03-27T06:36:48.325777+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:28.241688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050380+0000","last_change":"2024-03-27T06:42:20.658690+0000","last_active":"2024-03-27T06:43:34.050380+0000","last_peered":"2024-03-27T06:43:34.050380+0000","last_clean":"2024-03-27T06:43:34.050380+0000","last_became_active":"2024-03-27T06:42:20.658348+0000","last_became_peered":"2024-03-27T06:42:20.658348+0000","last_unstale":"2024-03-27T06:43:34.050380+0000","last_undegraded":"2024-03-27T06:43:34.050380+0000","last_fullsized":"2024-03-27T06:43:34.050380+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:12:29.985173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788194+0000","last_change":"2024-03-27T06:41:28.399205+0000","last_active":"2024-03-27T06:43:33.788194+0000","last_peered":"2024-03-27T06:43:33.788194+0000","last_clean":"2024-03-27T06:43:33.788194+0000","last_became_active":"2024-03-27T06:41:28.398889+0000","last_became_peered":"2024-03-27T06:41:28.398889+0000","last_unstale":"2024-03-27T06:43:33.788194+0000","last_undegraded":"2024-03-27T06:43:33.788194+0000","last_fullsized":"2024-03-27T06:43:33.788194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:28:08.769187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043816+0000","last_change":"2024-03-27T06:42:20.663370+0000","last_active":"2024-03-27T06:43:34.043816+0000","last_peered":"2024-03-27T06:43:34.043816+0000","last_clean":"2024-03-27T06:43:34.043816+0000","last_became_active":"2024-03-27T06:42:20.654243+0000","last_became_peered":"2024-03-27T06:42:20.654243+0000","last_unstale":"2024-03-27T06:43:34.043816+0000","last_undegraded":"2024-03-27T06:43:34.043816+0000","last_fullsized":"2024-03-27T06:43:34.043816+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:43:46.566952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051012+0000","last_change":"2024-03-27T06:41:29.395466+0000","last_active":"2024-03-27T06:43:34.051012+0000","last_peered":"2024-03-27T06:43:34.051012+0000","last_clean":"2024-03-27T06:43:34.051012+0000","last_became_active":"2024-03-27T06:41:29.395094+0000","last_became_peered":"2024-03-27T06:41:29.395094+0000","last_unstale":"2024-03-27T06:43:34.051012+0000","last_undegraded":"2024-03-27T06:43:34.051012+0000","last_fullsized":"2024-03-27T06:43:34.051012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:29:01.746809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'48","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.011094+0000","last_change":"2024-03-27T06:41:28.400420+0000","last_active":"2024-03-27T06:43:34.011094+0000","last_peered":"2024-03-27T06:43:34.011094+0000","last_clean":"2024-03-27T06:43:34.011094+0000","last_became_active":"2024-03-27T06:41:28.400115+0000","last_became_peered":"2024-03-27T06:41:28.400115+0000","last_unstale":"2024-03-27T06:43:34.011094+0000","last_undegraded":"2024-03-27T06:43:34.011094+0000","last_fullsized":"2024-03-27T06:43:34.011094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:37:40.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891028+0000","last_change":"2024-03-27T06:41:29.411428+0000","last_active":"2024-03-27T06:43:33.891028+0000","last_peered":"2024-03-27T06:43:33.891028+0000","last_clean":"2024-03-27T06:43:33.891028+0000","last_became_active":"2024-03-27T06:41:29.411075+0000","last_became_peered":"2024-03-27T06:41:29.411075+0000","last_unstale":"2024-03-27T06:43:33.891028+0000","last_undegraded":"2024-03-27T06:43:33.891028+0000","last_fullsized":"2024-03-27T06:43:33.891028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:55.764459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052106+0000","last_change":"2024-03-27T06:41:28.399799+0000","last_active":"2024-03-27T06:43:34.052106+0000","last_peered":"2024-03-27T06:43:34.052106+0000","last_clean":"2024-03-27T06:43:34.052106+0000","last_became_active":"2024-03-27T06:41:28.399385+0000","last_became_peered":"2024-03-27T06:41:28.399385+0000","last_unstale":"2024-03-27T06:43:34.052106+0000","last_undegraded":"2024-03-27T06:43:34.052106+0000","last_fullsized":"2024-03-27T06:43:34.052106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:16:18.005519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044636+0000","last_change":"2024-03-27T06:41:29.418991+0000","last_active":"2024-03-27T06:43:34.044636+0000","last_peered":"2024-03-27T06:43:34.044636+0000","last_clean":"2024-03-27T06:43:34.044636+0000","last_became_active":"2024-03-27T06:41:29.418527+0000","last_became_peered":"2024-03-27T06:41:29.418527+0000","last_unstale":"2024-03-27T06:43:34.044636+0000","last_undegraded":"2024-03-27T06:43:34.044636+0000","last_fullsized":"2024-03-27T06:43:34.044636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:49:55.913276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051676+0000","last_change":"2024-03-27T06:41:28.397771+0000","last_active":"2024-03-27T06:43:34.051676+0000","last_peered":"2024-03-27T06:43:34.051676+0000","last_clean":"2024-03-27T06:43:34.051676+0000","last_became_active":"2024-03-27T06:41:28.397470+0000","last_became_peered":"2024-03-27T06:41:28.397470+0000","last_unstale":"2024-03-27T06:43:34.051676+0000","last_undegraded":"2024-03-27T06:43:34.051676+0000","last_fullsized":"2024-03-27T06:43:34.051676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:37:21.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890518+0000","last_change":"2024-03-27T06:41:29.412280+0000","last_active":"2024-03-27T06:43:33.890518+0000","last_peered":"2024-03-27T06:43:33.890518+0000","last_clean":"2024-03-27T06:43:33.890518+0000","last_became_active":"2024-03-27T06:41:29.411336+0000","last_became_peered":"2024-03-27T06:41:29.411336+0000","last_unstale":"2024-03-27T06:43:33.890518+0000","last_undegraded":"2024-03-27T06:43:33.890518+0000","last_fullsized":"2024-03-27T06:43:33.890518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:13:03.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890676+0000","last_change":"2024-03-27T06:41:28.403529+0000","last_active":"2024-03-27T06:43:33.890676+0000","last_peered":"2024-03-27T06:43:33.890676+0000","last_clean":"2024-03-27T06:43:33.890676+0000","last_became_active":"2024-03-27T06:41:28.403155+0000","last_became_peered":"2024-03-27T06:41:28.403155+0000","last_unstale":"2024-03-27T06:43:33.890676+0000","last_undegraded":"2024-03-27T06:43:33.890676+0000","last_fullsized":"2024-03-27T06:43:33.890676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:23:53.971374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043609+0000","last_change":"2024-03-27T06:41:29.404929+0000","last_active":"2024-03-27T06:43:34.043609+0000","last_peered":"2024-03-27T06:43:34.043609+0000","last_clean":"2024-03-27T06:43:34.043609+0000","last_became_active":"2024-03-27T06:41:29.404602+0000","last_became_peered":"2024-03-27T06:41:29.404602+0000","last_unstale":"2024-03-27T06:43:34.043609+0000","last_undegraded":"2024-03-27T06:43:34.043609+0000","last_fullsized":"2024-03-27T06:43:34.043609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:19:07.379046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788341+0000","last_change":"2024-03-27T06:41:28.398602+0000","last_active":"2024-03-27T06:43:33.788341+0000","last_peered":"2024-03-27T06:43:33.788341+0000","last_clean":"2024-03-27T06:43:33.788341+0000","last_became_active":"2024-03-27T06:41:28.398068+0000","last_became_peered":"2024-03-27T06:41:28.398068+0000","last_unstale":"2024-03-27T06:43:33.788341+0000","last_undegraded":"2024-03-27T06:43:33.788341+0000","last_fullsized":"2024-03-27T06:43:33.788341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:10:53.312044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891033+0000","last_change":"2024-03-27T06:41:29.420245+0000","last_active":"2024-03-27T06:43:33.891033+0000","last_peered":"2024-03-27T06:43:33.891033+0000","last_clean":"2024-03-27T06:43:33.891033+0000","last_became_active":"2024-03-27T06:41:29.419912+0000","last_became_peered":"2024-03-27T06:41:29.419912+0000","last_unstale":"2024-03-27T06:43:33.891033+0000","last_undegraded":"2024-03-27T06:43:33.891033+0000","last_fullsized":"2024-03-27T06:43:33.891033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:46:58.203083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890685+0000","last_change":"2024-03-27T06:41:28.398595+0000","last_active":"2024-03-27T06:43:33.890685+0000","last_peered":"2024-03-27T06:43:33.890685+0000","last_clean":"2024-03-27T06:43:33.890685+0000","last_became_active":"2024-03-27T06:41:28.398036+0000","last_became_peered":"2024-03-27T06:41:28.398036+0000","last_unstale":"2024-03-27T06:43:33.890685+0000","last_undegraded":"2024-03-27T06:43:33.890685+0000","last_fullsized":"2024-03-27T06:43:33.890685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:47:15.769760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788967+0000","last_change":"2024-03-27T06:41:29.406576+0000","last_active":"2024-03-27T06:43:33.788967+0000","last_peered":"2024-03-27T06:43:33.788967+0000","last_clean":"2024-03-27T06:43:33.788967+0000","last_became_active":"2024-03-27T06:41:29.406222+0000","last_became_peered":"2024-03-27T06:41:29.406222+0000","last_unstale":"2024-03-27T06:43:33.788967+0000","last_undegraded":"2024-03-27T06:43:33.788967+0000","last_fullsized":"2024-03-27T06:43:33.788967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:05:12.446154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607123+0000","last_change":"2024-03-27T06:41:28.399361+0000","last_active":"2024-03-27T06:43:34.607123+0000","last_peered":"2024-03-27T06:43:34.607123+0000","last_clean":"2024-03-27T06:43:34.607123+0000","last_became_active":"2024-03-27T06:41:28.398021+0000","last_became_peered":"2024-03-27T06:41:28.398021+0000","last_unstale":"2024-03-27T06:43:34.607123+0000","last_undegraded":"2024-03-27T06:43:34.607123+0000","last_fullsized":"2024-03-27T06:43:34.607123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:51:34.106340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789247+0000","last_change":"2024-03-27T06:41:29.405696+0000","last_active":"2024-03-27T06:43:33.789247+0000","last_peered":"2024-03-27T06:43:33.789247+0000","last_clean":"2024-03-27T06:43:33.789247+0000","last_became_active":"2024-03-27T06:41:29.405273+0000","last_became_peered":"2024-03-27T06:41:29.405273+0000","last_unstale":"2024-03-27T06:43:33.789247+0000","last_undegraded":"2024-03-27T06:43:33.789247+0000","last_fullsized":"2024-03-27T06:43:33.789247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:02:58.778464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'41","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.006432+0000","last_change":"2024-03-27T06:41:28.391406+0000","last_active":"2024-03-27T06:43:34.006432+0000","last_peered":"2024-03-27T06:43:34.006432+0000","last_clean":"2024-03-27T06:43:34.006432+0000","last_became_active":"2024-03-27T06:41:28.390950+0000","last_became_peered":"2024-03-27T06:41:28.390950+0000","last_unstale":"2024-03-27T06:43:34.006432+0000","last_undegraded":"2024-03-27T06:43:34.006432+0000","last_fullsized":"2024-03-27T06:43:34.006432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:10:48.347139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788757+0000","last_change":"2024-03-27T06:41:29.410422+0000","last_active":"2024-03-27T06:43:33.788757+0000","last_peered":"2024-03-27T06:43:33.788757+0000","last_clean":"2024-03-27T06:43:33.788757+0000","last_became_active":"2024-03-27T06:41:29.409738+0000","last_became_peered":"2024-03-27T06:41:29.409738+0000","last_unstale":"2024-03-27T06:43:33.788757+0000","last_undegraded":"2024-03-27T06:43:33.788757+0000","last_fullsized":"2024-03-27T06:43:33.788757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:38:17.546504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051045+0000","last_change":"2024-03-27T06:42:21.008586+0000","last_active":"2024-03-27T06:43:34.051045+0000","last_peered":"2024-03-27T06:43:34.051045+0000","last_clean":"2024-03-27T06:43:34.051045+0000","last_became_active":"2024-03-27T06:42:21.004964+0000","last_became_peered":"2024-03-27T06:42:21.004964+0000","last_unstale":"2024-03-27T06:43:34.051045+0000","last_undegraded":"2024-03-27T06:43:34.051045+0000","last_fullsized":"2024-03-27T06:43:34.051045+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:11:13.246994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789419+0000","last_change":"2024-03-27T06:41:29.408809+0000","last_active":"2024-03-27T06:43:33.789419+0000","last_peered":"2024-03-27T06:43:33.789419+0000","last_clean":"2024-03-27T06:43:33.789419+0000","last_became_active":"2024-03-27T06:41:29.408447+0000","last_became_peered":"2024-03-27T06:41:29.408447+0000","last_unstale":"2024-03-27T06:43:33.789419+0000","last_undegraded":"2024-03-27T06:43:33.789419+0000","last_fullsized":"2024-03-27T06:43:33.789419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:04:55.269490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043772+0000","last_change":"2024-03-27T06:42:20.655859+0000","last_active":"2024-03-27T06:43:34.043772+0000","last_peered":"2024-03-27T06:43:34.043772+0000","last_clean":"2024-03-27T06:43:34.043772+0000","last_became_active":"2024-03-27T06:42:20.655446+0000","last_became_peered":"2024-03-27T06:42:20.655446+0000","last_unstale":"2024-03-27T06:43:34.043772+0000","last_undegraded":"2024-03-27T06:43:34.043772+0000","last_fullsized":"2024-03-27T06:43:34.043772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:43:07.920040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.787928+0000","last_change":"2024-03-27T06:41:29.409896+0000","last_active":"2024-03-27T06:43:33.787928+0000","last_peered":"2024-03-27T06:43:33.787928+0000","last_clean":"2024-03-27T06:43:33.787928+0000","last_became_active":"2024-03-27T06:41:29.408279+0000","last_became_peered":"2024-03-27T06:41:29.408279+0000","last_unstale":"2024-03-27T06:43:33.787928+0000","last_undegraded":"2024-03-27T06:43:33.787928+0000","last_fullsized":"2024-03-27T06:43:33.787928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:49:03.257543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788142+0000","last_change":"2024-03-27T06:41:29.412526+0000","last_active":"2024-03-27T06:43:33.788142+0000","last_peered":"2024-03-27T06:43:33.788142+0000","last_clean":"2024-03-27T06:43:33.788142+0000","last_became_active":"2024-03-27T06:41:29.412180+0000","last_became_peered":"2024-03-27T06:41:29.412180+0000","last_unstale":"2024-03-27T06:43:33.788142+0000","last_undegraded":"2024-03-27T06:43:33.788142+0000","last_fullsized":"2024-03-27T06:43:33.788142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:47.461457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044158+0000","last_change":"2024-03-27T06:42:20.657735+0000","last_active":"2024-03-27T06:43:34.044158+0000","last_peered":"2024-03-27T06:43:34.044158+0000","last_clean":"2024-03-27T06:43:34.044158+0000","last_became_active":"2024-03-27T06:42:20.657378+0000","last_became_peered":"2024-03-27T06:42:20.657378+0000","last_unstale":"2024-03-27T06:43:34.044158+0000","last_undegraded":"2024-03-27T06:43:34.044158+0000","last_fullsized":"2024-03-27T06:43:34.044158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:13:56.368229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051027+0000","last_change":"2024-03-27T06:41:29.402442+0000","last_active":"2024-03-27T06:43:34.051027+0000","last_peered":"2024-03-27T06:43:34.051027+0000","last_clean":"2024-03-27T06:43:34.051027+0000","last_became_active":"2024-03-27T06:41:29.402078+0000","last_became_peered":"2024-03-27T06:41:29.402078+0000","last_unstale":"2024-03-27T06:43:34.051027+0000","last_undegraded":"2024-03-27T06:43:34.051027+0000","last_fullsized":"2024-03-27T06:43:34.051027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:07:35.344881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044286+0000","last_change":"2024-03-27T06:41:29.418563+0000","last_active":"2024-03-27T06:43:34.044286+0000","last_peered":"2024-03-27T06:43:34.044286+0000","last_clean":"2024-03-27T06:43:34.044286+0000","last_became_active":"2024-03-27T06:41:29.418209+0000","last_became_peered":"2024-03-27T06:41:29.418209+0000","last_unstale":"2024-03-27T06:43:34.044286+0000","last_undegraded":"2024-03-27T06:43:34.044286+0000","last_fullsized":"2024-03-27T06:43:34.044286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:53:58.536148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789184+0000","last_change":"2024-03-27T06:41:29.400414+0000","last_active":"2024-03-27T06:43:33.789184+0000","last_peered":"2024-03-27T06:43:33.789184+0000","last_clean":"2024-03-27T06:43:33.789184+0000","last_became_active":"2024-03-27T06:41:29.397865+0000","last_became_peered":"2024-03-27T06:41:29.397865+0000","last_unstale":"2024-03-27T06:43:33.789184+0000","last_undegraded":"2024-03-27T06:43:33.789184+0000","last_fullsized":"2024-03-27T06:43:33.789184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:35:28.405347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890973+0000","last_change":"2024-03-27T06:41:29.416740+0000","last_active":"2024-03-27T06:43:33.890973+0000","last_peered":"2024-03-27T06:43:33.890973+0000","last_clean":"2024-03-27T06:43:33.890973+0000","last_became_active":"2024-03-27T06:41:29.416417+0000","last_became_peered":"2024-03-27T06:41:29.416417+0000","last_unstale":"2024-03-27T06:43:33.890973+0000","last_undegraded":"2024-03-27T06:43:33.890973+0000","last_fullsized":"2024-03-27T06:43:33.890973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:08:34.186527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899169+0000","last_change":"2024-03-27T06:41:29.418797+0000","last_active":"2024-03-27T06:43:33.899169+0000","last_peered":"2024-03-27T06:43:33.899169+0000","last_clean":"2024-03-27T06:43:33.899169+0000","last_became_active":"2024-03-27T06:41:29.418472+0000","last_became_peered":"2024-03-27T06:41:29.418472+0000","last_unstale":"2024-03-27T06:43:33.899169+0000","last_undegraded":"2024-03-27T06:43:33.899169+0000","last_fullsized":"2024-03-27T06:43:33.899169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:30:55.540353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890611+0000","last_change":"2024-03-27T06:41:29.416124+0000","last_active":"2024-03-27T06:43:33.890611+0000","last_peered":"2024-03-27T06:43:33.890611+0000","last_clean":"2024-03-27T06:43:33.890611+0000","last_became_active":"2024-03-27T06:41:29.415665+0000","last_became_peered":"2024-03-27T06:41:29.415665+0000","last_unstale":"2024-03-27T06:43:33.890611+0000","last_undegraded":"2024-03-27T06:43:33.890611+0000","last_fullsized":"2024-03-27T06:43:33.890611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:46:04.160705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.053062+0000","last_change":"2024-03-27T06:41:29.417165+0000","last_active":"2024-03-27T06:43:34.053062+0000","last_peered":"2024-03-27T06:43:34.053062+0000","last_clean":"2024-03-27T06:43:34.053062+0000","last_became_active":"2024-03-27T06:41:29.416856+0000","last_became_peered":"2024-03-27T06:41:29.416856+0000","last_unstale":"2024-03-27T06:43:34.053062+0000","last_undegraded":"2024-03-27T06:43:34.053062+0000","last_fullsized":"2024-03-27T06:43:34.053062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:55:20.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050772+0000","last_change":"2024-03-27T06:41:29.412377+0000","last_active":"2024-03-27T06:43:34.050772+0000","last_peered":"2024-03-27T06:43:34.050772+0000","last_clean":"2024-03-27T06:43:34.050772+0000","last_became_active":"2024-03-27T06:41:29.411761+0000","last_became_peered":"2024-03-27T06:41:29.411761+0000","last_unstale":"2024-03-27T06:43:34.050772+0000","last_undegraded":"2024-03-27T06:43:34.050772+0000","last_fullsized":"2024-03-27T06:43:34.050772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:16:56.844342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043855+0000","last_change":"2024-03-27T06:41:29.413660+0000","last_active":"2024-03-27T06:43:34.043855+0000","last_peered":"2024-03-27T06:43:34.043855+0000","last_clean":"2024-03-27T06:43:34.043855+0000","last_became_active":"2024-03-27T06:41:29.413201+0000","last_became_peered":"2024-03-27T06:41:29.413201+0000","last_unstale":"2024-03-27T06:43:34.043855+0000","last_undegraded":"2024-03-27T06:43:34.043855+0000","last_fullsized":"2024-03-27T06:43:34.043855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:14:46.446595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106419,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600992,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1335,"num_read_kb":783,"num_write":374,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1802976,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1155072,"data_stored":1259064,"data_compressed":44525,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":861923,"data_compressed":22042,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037813,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":862077,"data_compressed":22042,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":1063449,"data_compressed":38369,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":520990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":470545,"data_compressed":2896,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":537997,"data_compressed":2896,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":993065,"data_compressed":28198,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":459890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1193413,"data_compressed":44525,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135343,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":394492,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200994,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T06:43:38.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T06:43:38.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1600195556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:38.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: [27/Mar/2024:06:43:37] ENGINE Bus STARTING 2024-03-27T06:43:38.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: [27/Mar/2024:06:43:37] ENGINE Serving on http://172.21.15.16:8765 2024-03-27T06:43:38.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2628363166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:38.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi170 to 174.1M 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: Unable to set osd_memory_target on smithi170 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: Unable to set osd_memory_target on smithi016 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: [27/Mar/2024:06:43:37] ENGINE Serving on https://172.21.15.16:7150 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: [27/Mar/2024:06:43:37] ENGINE Bus STARTED 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-27T06:43:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:43:38.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:43:38.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-27T06:43:38.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: Updating smithi156:/etc/ceph/ceph.conf 2024-03-27T06:43:38.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-27T06:43:38.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:37 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2302198908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.494+0000 7f7dcbac0640 1 -- 172.21.15.16:0/1917380616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc4100a50 msgr2=0x7f7dc4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.494+0000 7f7dcbac0640 1 --2- 172.21.15.16:0/1917380616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc4100a50 0x7f7dc4100ed0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009930 tx=0x7f7db402bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.494+0000 7f7dcbac0640 1 -- 172.21.15.16:0/1917380616 shutdown_connections 2024-03-27T06:43:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.494+0000 7f7dcbac0640 1 --2- 172.21.15.16:0/1917380616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dc4101410 0x7f7dc410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.494+0000 7f7dcbac0640 1 --2- 172.21.15.16:0/1917380616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc4100a50 0x7f7dc4100ed0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.494+0000 7f7dcbac0640 1 --2- 172.21.15.16:0/1917380616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc4106a60 0x7f7dc4106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.494+0000 7f7dcbac0640 1 -- 172.21.15.16:0/1917380616 >> 172.21.15.16:0/1917380616 conn(0x7f7dc4077b50 msgr2=0x7f7dc4077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.494+0000 7f7dcbac0640 1 -- 172.21.15.16:0/1917380616 shutdown_connections 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.494+0000 7f7dcbac0640 1 -- 172.21.15.16:0/1917380616 wait complete. 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dcbac0640 1 Processor -- start 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dcbac0640 1 -- start start 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dcbac0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dc4100a50 0x7f7dc41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dcbac0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc4101410 0x7f7dc4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dcbac0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc4106a60 0x7f7dc419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dcbac0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc4110f40 con 0x7f7dc4101410 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dcbac0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc4110dc0 con 0x7f7dc4106a60 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dcbac0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc41110c0 con 0x7f7dc4100a50 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dc905b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc4101410 0x7f7dc4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dc985c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dc4100a50 0x7f7dc41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dc905b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc4101410 0x7f7dc4198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51174/0 (socket says 172.21.15.16:51174) 2024-03-27T06:43:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dca05d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc4106a60 0x7f7dc419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dc905b640 1 -- 172.21.15.16:0/4154408697 learned_addr learned my addr 172.21.15.16:0/4154408697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dc985c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dc4100a50 0x7f7dc41985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:46512/0 (socket says 172.21.15.16:46512) 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dc905b640 1 -- 172.21.15.16:0/4154408697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dc4100a50 msgr2=0x7f7dc41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dc905b640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dc4100a50 0x7f7dc41985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dc905b640 1 -- 172.21.15.16:0/4154408697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc4106a60 msgr2=0x7f7dc419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dc905b640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc4106a60 0x7f7dc419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dc905b640 1 -- 172.21.15.16:0/4154408697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc419d590 con 0x7f7dc4101410 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.495+0000 7f7dc905b640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc4101410 0x7f7dc4198b30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009230 tx=0x7f7db402dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.496+0000 7f7dbaffd640 1 -- 172.21.15.16:0/4154408697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4043020 con 0x7f7dc4101410 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.496+0000 7f7dcbac0640 1 -- 172.21.15.16:0/4154408697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc419d820 con 0x7f7dc4101410 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.496+0000 7f7dbaffd640 1 -- 172.21.15.16:0/4154408697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7db4004250 con 0x7f7dc4101410 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.496+0000 7f7dbaffd640 1 -- 172.21.15.16:0/4154408697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4003970 con 0x7f7dc4101410 2024-03-27T06:43:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.496+0000 7f7dcbac0640 1 -- 172.21.15.16:0/4154408697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc41a5110 con 0x7f7dc4101410 2024-03-27T06:43:38.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.497+0000 7f7dcbac0640 1 -- 172.21.15.16:0/4154408697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc4068940 con 0x7f7dc4101410 2024-03-27T06:43:38.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.498+0000 7f7dbaffd640 1 -- 172.21.15.16:0/4154408697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7f7db4046020 con 0x7f7dc4101410 2024-03-27T06:43:38.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.498+0000 7f7dbaffd640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7db00a38a0 0x7f7db00a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.499+0000 7f7dbaffd640 1 -- 172.21.15.16:0/4154408697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7db40f2240 con 0x7f7dc4101410 2024-03-27T06:43:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.499+0000 7f7dc985c640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7db00a38a0 0x7f7db00a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.499+0000 7f7dc985c640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7db00a38a0 0x7f7db00a5d60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009b50 tx=0x7f7dc0005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.503+0000 7f7dbaffd640 1 -- 172.21.15.16:0/4154408697 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7db404d020 con 0x7f7dc4101410 2024-03-27T06:43:38.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.696+0000 7f7dcbac0640 1 -- 172.21.15.16:0/4154408697 --> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7dc4061f90 con 0x7f7db00a38a0 2024-03-27T06:43:38.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.704+0000 7f7dbaffd640 1 -- 172.21.15.16:0/4154408697 <== mgr.15354 v2:172.21.15.16:6800/3715976676 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258017 (secure 0 0 0) 0x7f7dc4061f90 con 0x7f7db00a38a0 2024-03-27T06:43:38.701 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:38.703 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-03-27T06:43:38.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.706+0000 7f7dcbac0640 1 -- 172.21.15.16:0/4154408697 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7db00a38a0 msgr2=0x7f7db00a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.706+0000 7f7dcbac0640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7db00a38a0 0x7f7db00a5d60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009b50 tx=0x7f7dc0005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:38.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.706+0000 7f7dcbac0640 1 -- 172.21.15.16:0/4154408697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc4101410 msgr2=0x7f7dc4198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.706+0000 7f7dcbac0640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc4101410 0x7f7dc4198b30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009230 tx=0x7f7db402dc90 comp rx=0 tx=0).stop 2024-03-27T06:43:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.707+0000 7f7dcbac0640 1 -- 172.21.15.16:0/4154408697 shutdown_connections 2024-03-27T06:43:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.707+0000 7f7dcbac0640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc4106a60 0x7f7dc419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.707+0000 7f7dcbac0640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc4101410 0x7f7dc4198b30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.707+0000 7f7dcbac0640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7db00a38a0 0x7f7db00a5d60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.707+0000 7f7dcbac0640 1 --2- 172.21.15.16:0/4154408697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dc4100a50 0x7f7dc41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.707+0000 7f7dcbac0640 1 -- 172.21.15.16:0/4154408697 >> 172.21.15.16:0/4154408697 conn(0x7f7dc4077b50 msgr2=0x7f7dc4069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.707+0000 7f7dcbac0640 1 -- 172.21.15.16:0/4154408697 shutdown_connections 2024-03-27T06:43:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.707+0000 7f7dcbac0640 1 -- 172.21.15.16:0/4154408697 wait complete. 2024-03-27T06:43:38.713 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-03-27T06:43:37.799887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191594,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1471,"num_read_kb":902,"num_write":557,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1180960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":942,"ondisk_log_size":942,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369548,"kb_used_data":12052,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572276,"statfs":{"total":1151940427776,"available":1151562010624,"internally_reserved":0,"allocated":12341248,"data_stored":9141984,"data_compressed":205493,"data_compressed_allocated":266240,"data_compressed_original":3383296,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.986546"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044192+0000","last_change":"2024-03-27T06:41:29.408953+0000","last_active":"2024-03-27T06:43:34.044192+0000","last_peered":"2024-03-27T06:43:34.044192+0000","last_clean":"2024-03-27T06:43:34.044192+0000","last_became_active":"2024-03-27T06:41:29.408143+0000","last_became_peered":"2024-03-27T06:41:29.408143+0000","last_unstale":"2024-03-27T06:43:34.044192+0000","last_undegraded":"2024-03-27T06:43:34.044192+0000","last_fullsized":"2024-03-27T06:43:34.044192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:22.486951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044362+0000","last_change":"2024-03-27T06:41:29.405080+0000","last_active":"2024-03-27T06:43:34.044362+0000","last_peered":"2024-03-27T06:43:34.044362+0000","last_clean":"2024-03-27T06:43:34.044362+0000","last_became_active":"2024-03-27T06:41:29.404743+0000","last_became_peered":"2024-03-27T06:41:29.404743+0000","last_unstale":"2024-03-27T06:43:34.044362+0000","last_undegraded":"2024-03-27T06:43:34.044362+0000","last_fullsized":"2024-03-27T06:43:34.044362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:24:30.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044526+0000","last_change":"2024-03-27T06:41:29.409346+0000","last_active":"2024-03-27T06:43:34.044526+0000","last_peered":"2024-03-27T06:43:34.044526+0000","last_clean":"2024-03-27T06:43:34.044526+0000","last_became_active":"2024-03-27T06:41:29.408837+0000","last_became_peered":"2024-03-27T06:41:29.408837+0000","last_unstale":"2024-03-27T06:43:34.044526+0000","last_undegraded":"2024-03-27T06:43:34.044526+0000","last_fullsized":"2024-03-27T06:43:34.044526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:11:12.274869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052302+0000","last_change":"2024-03-27T06:41:29.412115+0000","last_active":"2024-03-27T06:43:34.052302+0000","last_peered":"2024-03-27T06:43:34.052302+0000","last_clean":"2024-03-27T06:43:34.052302+0000","last_became_active":"2024-03-27T06:41:29.411702+0000","last_became_peered":"2024-03-27T06:41:29.411702+0000","last_unstale":"2024-03-27T06:43:34.052302+0000","last_undegraded":"2024-03-27T06:43:34.052302+0000","last_fullsized":"2024-03-27T06:43:34.052302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:04:42.347229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890812+0000","last_change":"2024-03-27T06:41:29.409888+0000","last_active":"2024-03-27T06:43:33.890812+0000","last_peered":"2024-03-27T06:43:33.890812+0000","last_clean":"2024-03-27T06:43:33.890812+0000","last_became_active":"2024-03-27T06:41:29.409558+0000","last_became_peered":"2024-03-27T06:41:29.409558+0000","last_unstale":"2024-03-27T06:43:33.890812+0000","last_undegraded":"2024-03-27T06:43:33.890812+0000","last_fullsized":"2024-03-27T06:43:33.890812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:38:51.452793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902755+0000","last_change":"2024-03-27T06:41:29.404088+0000","last_active":"2024-03-27T06:43:33.902755+0000","last_peered":"2024-03-27T06:43:33.902755+0000","last_clean":"2024-03-27T06:43:33.902755+0000","last_became_active":"2024-03-27T06:41:29.403698+0000","last_became_peered":"2024-03-27T06:41:29.403698+0000","last_unstale":"2024-03-27T06:43:33.902755+0000","last_undegraded":"2024-03-27T06:43:33.902755+0000","last_fullsized":"2024-03-27T06:43:33.902755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:14:31.296523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788203+0000","last_change":"2024-03-27T06:41:29.413157+0000","last_active":"2024-03-27T06:43:33.788203+0000","last_peered":"2024-03-27T06:43:33.788203+0000","last_clean":"2024-03-27T06:43:33.788203+0000","last_became_active":"2024-03-27T06:41:29.412669+0000","last_became_peered":"2024-03-27T06:41:29.412669+0000","last_unstale":"2024-03-27T06:43:33.788203+0000","last_undegraded":"2024-03-27T06:43:33.788203+0000","last_fullsized":"2024-03-27T06:43:33.788203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:28:20.823770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890507+0000","last_change":"2024-03-27T06:41:29.417207+0000","last_active":"2024-03-27T06:43:33.890507+0000","last_peered":"2024-03-27T06:43:33.890507+0000","last_clean":"2024-03-27T06:43:33.890507+0000","last_became_active":"2024-03-27T06:41:29.415556+0000","last_became_peered":"2024-03-27T06:41:29.415556+0000","last_unstale":"2024-03-27T06:43:33.890507+0000","last_undegraded":"2024-03-27T06:43:33.890507+0000","last_fullsized":"2024-03-27T06:43:33.890507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:26:04.040296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.053665+0000","last_change":"2024-03-27T06:41:29.400032+0000","last_active":"2024-03-27T06:43:34.053665+0000","last_peered":"2024-03-27T06:43:34.053665+0000","last_clean":"2024-03-27T06:43:34.053665+0000","last_became_active":"2024-03-27T06:41:29.399274+0000","last_became_peered":"2024-03-27T06:41:29.399274+0000","last_unstale":"2024-03-27T06:43:34.053665+0000","last_undegraded":"2024-03-27T06:43:34.053665+0000","last_fullsized":"2024-03-27T06:43:34.053665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:55.306309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.903052+0000","last_change":"2024-03-27T06:41:29.412471+0000","last_active":"2024-03-27T06:43:33.903052+0000","last_peered":"2024-03-27T06:43:33.903052+0000","last_clean":"2024-03-27T06:43:33.903052+0000","last_became_active":"2024-03-27T06:41:29.412134+0000","last_became_peered":"2024-03-27T06:41:29.412134+0000","last_unstale":"2024-03-27T06:43:33.903052+0000","last_undegraded":"2024-03-27T06:43:33.903052+0000","last_fullsized":"2024-03-27T06:43:33.903052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:35:57.146944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788956+0000","last_change":"2024-03-27T06:41:29.407915+0000","last_active":"2024-03-27T06:43:33.788956+0000","last_peered":"2024-03-27T06:43:33.788956+0000","last_clean":"2024-03-27T06:43:33.788956+0000","last_became_active":"2024-03-27T06:41:29.407563+0000","last_became_peered":"2024-03-27T06:41:29.407563+0000","last_unstale":"2024-03-27T06:43:33.788956+0000","last_undegraded":"2024-03-27T06:43:33.788956+0000","last_fullsized":"2024-03-27T06:43:33.788956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:09:05.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052028+0000","last_change":"2024-03-27T06:41:29.413374+0000","last_active":"2024-03-27T06:43:34.052028+0000","last_peered":"2024-03-27T06:43:34.052028+0000","last_clean":"2024-03-27T06:43:34.052028+0000","last_became_active":"2024-03-27T06:41:29.413010+0000","last_became_peered":"2024-03-27T06:41:29.413010+0000","last_unstale":"2024-03-27T06:43:34.052028+0000","last_undegraded":"2024-03-27T06:43:34.052028+0000","last_fullsized":"2024-03-27T06:43:34.052028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:48:52.329611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050925+0000","last_change":"2024-03-27T06:41:29.408640+0000","last_active":"2024-03-27T06:43:34.050925+0000","last_peered":"2024-03-27T06:43:34.050925+0000","last_clean":"2024-03-27T06:43:34.050925+0000","last_became_active":"2024-03-27T06:41:29.408268+0000","last_became_peered":"2024-03-27T06:41:29.408268+0000","last_unstale":"2024-03-27T06:43:34.050925+0000","last_undegraded":"2024-03-27T06:43:34.050925+0000","last_fullsized":"2024-03-27T06:43:34.050925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:51:59.803596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043668+0000","last_change":"2024-03-27T06:41:29.420436+0000","last_active":"2024-03-27T06:43:34.043668+0000","last_peered":"2024-03-27T06:43:34.043668+0000","last_clean":"2024-03-27T06:43:34.043668+0000","last_became_active":"2024-03-27T06:41:29.419426+0000","last_became_peered":"2024-03-27T06:41:29.419426+0000","last_unstale":"2024-03-27T06:43:34.043668+0000","last_undegraded":"2024-03-27T06:43:34.043668+0000","last_fullsized":"2024-03-27T06:43:34.043668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:38:15.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043640+0000","last_change":"2024-03-27T06:41:29.415673+0000","last_active":"2024-03-27T06:43:34.043640+0000","last_peered":"2024-03-27T06:43:34.043640+0000","last_clean":"2024-03-27T06:43:34.043640+0000","last_became_active":"2024-03-27T06:41:29.414741+0000","last_became_peered":"2024-03-27T06:41:29.414741+0000","last_unstale":"2024-03-27T06:43:34.043640+0000","last_undegraded":"2024-03-27T06:43:34.043640+0000","last_fullsized":"2024-03-27T06:43:34.043640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:38:43.183517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051498+0000","last_change":"2024-03-27T06:41:29.408609+0000","last_active":"2024-03-27T06:43:34.051498+0000","last_peered":"2024-03-27T06:43:34.051498+0000","last_clean":"2024-03-27T06:43:34.051498+0000","last_became_active":"2024-03-27T06:41:29.408252+0000","last_became_peered":"2024-03-27T06:41:29.408252+0000","last_unstale":"2024-03-27T06:43:34.051498+0000","last_undegraded":"2024-03-27T06:43:34.051498+0000","last_fullsized":"2024-03-27T06:43:34.051498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:14:13.609113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044330+0000","last_change":"2024-03-27T06:41:29.409221+0000","last_active":"2024-03-27T06:43:34.044330+0000","last_peered":"2024-03-27T06:43:34.044330+0000","last_clean":"2024-03-27T06:43:34.044330+0000","last_became_active":"2024-03-27T06:41:29.408598+0000","last_became_peered":"2024-03-27T06:41:29.408598+0000","last_unstale":"2024-03-27T06:43:34.044330+0000","last_undegraded":"2024-03-27T06:43:34.044330+0000","last_fullsized":"2024-03-27T06:43:34.044330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:42:16.251149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.786349+0000","last_change":"2024-03-27T06:41:29.406143+0000","last_active":"2024-03-27T06:43:33.786349+0000","last_peered":"2024-03-27T06:43:33.786349+0000","last_clean":"2024-03-27T06:43:33.786349+0000","last_became_active":"2024-03-27T06:41:29.405754+0000","last_became_peered":"2024-03-27T06:41:29.405754+0000","last_unstale":"2024-03-27T06:43:33.786349+0000","last_undegraded":"2024-03-27T06:43:33.786349+0000","last_fullsized":"2024-03-27T06:43:33.786349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:42:47.787099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788822+0000","last_change":"2024-03-27T06:41:29.402471+0000","last_active":"2024-03-27T06:43:33.788822+0000","last_peered":"2024-03-27T06:43:33.788822+0000","last_clean":"2024-03-27T06:43:33.788822+0000","last_became_active":"2024-03-27T06:41:29.402113+0000","last_became_peered":"2024-03-27T06:41:29.402113+0000","last_unstale":"2024-03-27T06:43:33.788822+0000","last_undegraded":"2024-03-27T06:43:33.788822+0000","last_fullsized":"2024-03-27T06:43:33.788822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:34:16.408058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'14","reported_seq":157,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607533+0000","last_change":"2024-03-27T06:41:28.397815+0000","last_active":"2024-03-27T06:43:34.607533+0000","last_peered":"2024-03-27T06:43:34.607533+0000","last_clean":"2024-03-27T06:43:34.607533+0000","last_became_active":"2024-03-27T06:41:28.397503+0000","last_became_peered":"2024-03-27T06:41:28.397503+0000","last_unstale":"2024-03-27T06:43:34.607533+0000","last_undegraded":"2024-03-27T06:43:34.607533+0000","last_fullsized":"2024-03-27T06:43:34.607533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:15:53.751809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890734+0000","last_change":"2024-03-27T06:41:29.419528+0000","last_active":"2024-03-27T06:43:33.890734+0000","last_peered":"2024-03-27T06:43:33.890734+0000","last_clean":"2024-03-27T06:43:33.890734+0000","last_became_active":"2024-03-27T06:41:29.417091+0000","last_became_peered":"2024-03-27T06:41:29.417091+0000","last_unstale":"2024-03-27T06:43:33.890734+0000","last_undegraded":"2024-03-27T06:43:33.890734+0000","last_fullsized":"2024-03-27T06:43:33.890734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:13:14.629211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891107+0000","last_change":"2024-03-27T06:41:28.400468+0000","last_active":"2024-03-27T06:43:33.891107+0000","last_peered":"2024-03-27T06:43:33.891107+0000","last_clean":"2024-03-27T06:43:33.891107+0000","last_became_active":"2024-03-27T06:41:28.400190+0000","last_became_peered":"2024-03-27T06:41:28.400190+0000","last_unstale":"2024-03-27T06:43:33.891107+0000","last_undegraded":"2024-03-27T06:43:33.891107+0000","last_fullsized":"2024-03-27T06:43:33.891107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:24:17.059874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.045055+0000","last_change":"2024-03-27T06:41:29.409364+0000","last_active":"2024-03-27T06:43:34.045055+0000","last_peered":"2024-03-27T06:43:34.045055+0000","last_clean":"2024-03-27T06:43:34.045055+0000","last_became_active":"2024-03-27T06:41:29.408946+0000","last_became_peered":"2024-03-27T06:41:29.408946+0000","last_unstale":"2024-03-27T06:43:34.045055+0000","last_undegraded":"2024-03-27T06:43:34.045055+0000","last_fullsized":"2024-03-27T06:43:34.045055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:46:17.132071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890918+0000","last_change":"2024-03-27T06:41:28.401327+0000","last_active":"2024-03-27T06:43:33.890918+0000","last_peered":"2024-03-27T06:43:33.890918+0000","last_clean":"2024-03-27T06:43:33.890918+0000","last_became_active":"2024-03-27T06:41:28.400524+0000","last_became_peered":"2024-03-27T06:41:28.400524+0000","last_unstale":"2024-03-27T06:43:33.890918+0000","last_undegraded":"2024-03-27T06:43:33.890918+0000","last_fullsized":"2024-03-27T06:43:33.890918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:11:52.608690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051934+0000","last_change":"2024-03-27T06:41:29.403042+0000","last_active":"2024-03-27T06:43:34.051934+0000","last_peered":"2024-03-27T06:43:34.051934+0000","last_clean":"2024-03-27T06:43:34.051934+0000","last_became_active":"2024-03-27T06:41:29.402516+0000","last_became_peered":"2024-03-27T06:41:29.402516+0000","last_unstale":"2024-03-27T06:43:34.051934+0000","last_undegraded":"2024-03-27T06:43:34.051934+0000","last_fullsized":"2024-03-27T06:43:34.051934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:20:55.777498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891130+0000","last_change":"2024-03-27T06:41:28.395155+0000","last_active":"2024-03-27T06:43:33.891130+0000","last_peered":"2024-03-27T06:43:33.891130+0000","last_clean":"2024-03-27T06:43:33.891130+0000","last_became_active":"2024-03-27T06:41:28.394760+0000","last_became_peered":"2024-03-27T06:41:28.394760+0000","last_unstale":"2024-03-27T06:43:33.891130+0000","last_undegraded":"2024-03-27T06:43:33.891130+0000","last_fullsized":"2024-03-27T06:43:33.891130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:15:28.182581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899706+0000","last_change":"2024-03-27T06:41:29.409747+0000","last_active":"2024-03-27T06:43:33.899706+0000","last_peered":"2024-03-27T06:43:33.899706+0000","last_clean":"2024-03-27T06:43:33.899706+0000","last_became_active":"2024-03-27T06:41:29.409425+0000","last_became_peered":"2024-03-27T06:41:29.409425+0000","last_unstale":"2024-03-27T06:43:33.899706+0000","last_undegraded":"2024-03-27T06:43:33.899706+0000","last_fullsized":"2024-03-27T06:43:33.899706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:36.181279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.605066+0000","last_change":"2024-03-27T06:41:28.394726+0000","last_active":"2024-03-27T06:43:34.605066+0000","last_peered":"2024-03-27T06:43:34.605066+0000","last_clean":"2024-03-27T06:43:34.605066+0000","last_became_active":"2024-03-27T06:41:28.394378+0000","last_became_peered":"2024-03-27T06:41:28.394378+0000","last_unstale":"2024-03-27T06:43:34.605066+0000","last_undegraded":"2024-03-27T06:43:34.605066+0000","last_fullsized":"2024-03-27T06:43:34.605066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:30:22.194211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890573+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:43:33.890573+0000","last_peered":"2024-03-27T06:43:33.890573+0000","last_clean":"2024-03-27T06:43:33.890573+0000","last_became_active":"2024-03-27T06:41:29.408826+0000","last_became_peered":"2024-03-27T06:41:29.408826+0000","last_unstale":"2024-03-27T06:43:33.890573+0000","last_undegraded":"2024-03-27T06:43:33.890573+0000","last_fullsized":"2024-03-27T06:43:33.890573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:50:53.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902801+0000","last_change":"2024-03-27T06:41:28.393655+0000","last_active":"2024-03-27T06:43:33.902801+0000","last_peered":"2024-03-27T06:43:33.902801+0000","last_clean":"2024-03-27T06:43:33.902801+0000","last_became_active":"2024-03-27T06:41:28.393245+0000","last_became_peered":"2024-03-27T06:41:28.393245+0000","last_unstale":"2024-03-27T06:43:33.902801+0000","last_undegraded":"2024-03-27T06:43:33.902801+0000","last_fullsized":"2024-03-27T06:43:33.902801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:58:21.169444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899501+0000","last_change":"2024-03-27T06:41:29.417730+0000","last_active":"2024-03-27T06:43:33.899501+0000","last_peered":"2024-03-27T06:43:33.899501+0000","last_clean":"2024-03-27T06:43:33.899501+0000","last_became_active":"2024-03-27T06:41:29.417319+0000","last_became_peered":"2024-03-27T06:41:29.417319+0000","last_unstale":"2024-03-27T06:43:33.899501+0000","last_undegraded":"2024-03-27T06:43:33.899501+0000","last_fullsized":"2024-03-27T06:43:33.899501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:25.224220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050475+0000","last_change":"2024-03-27T06:41:28.402259+0000","last_active":"2024-03-27T06:43:34.050475+0000","last_peered":"2024-03-27T06:43:34.050475+0000","last_clean":"2024-03-27T06:43:34.050475+0000","last_became_active":"2024-03-27T06:41:28.401922+0000","last_became_peered":"2024-03-27T06:41:28.401922+0000","last_unstale":"2024-03-27T06:43:34.050475+0000","last_undegraded":"2024-03-27T06:43:34.050475+0000","last_fullsized":"2024-03-27T06:43:34.050475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:57:33.245183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890562+0000","last_change":"2024-03-27T06:41:29.409034+0000","last_active":"2024-03-27T06:43:33.890562+0000","last_peered":"2024-03-27T06:43:33.890562+0000","last_clean":"2024-03-27T06:43:33.890562+0000","last_became_active":"2024-03-27T06:41:29.408666+0000","last_became_peered":"2024-03-27T06:41:29.408666+0000","last_unstale":"2024-03-27T06:43:33.890562+0000","last_undegraded":"2024-03-27T06:43:33.890562+0000","last_fullsized":"2024-03-27T06:43:33.890562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:09:55.532059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899143+0000","last_change":"2024-03-27T06:41:28.393802+0000","last_active":"2024-03-27T06:43:33.899143+0000","last_peered":"2024-03-27T06:43:33.899143+0000","last_clean":"2024-03-27T06:43:33.899143+0000","last_became_active":"2024-03-27T06:41:28.393503+0000","last_became_peered":"2024-03-27T06:41:28.393503+0000","last_unstale":"2024-03-27T06:43:33.899143+0000","last_undegraded":"2024-03-27T06:43:33.899143+0000","last_fullsized":"2024-03-27T06:43:33.899143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:17.765709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.786232+0000","last_change":"2024-03-27T06:41:29.416557+0000","last_active":"2024-03-27T06:43:33.786232+0000","last_peered":"2024-03-27T06:43:33.786232+0000","last_clean":"2024-03-27T06:43:33.786232+0000","last_became_active":"2024-03-27T06:41:29.416223+0000","last_became_peered":"2024-03-27T06:41:29.416223+0000","last_unstale":"2024-03-27T06:43:33.786232+0000","last_undegraded":"2024-03-27T06:43:33.786232+0000","last_fullsized":"2024-03-27T06:43:33.786232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:54:20.929325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":268,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.604510+0000","last_change":"2024-03-27T06:41:28.396014+0000","last_active":"2024-03-27T06:43:34.604510+0000","last_peered":"2024-03-27T06:43:34.604510+0000","last_clean":"2024-03-27T06:43:34.604510+0000","last_became_active":"2024-03-27T06:41:28.395699+0000","last_became_peered":"2024-03-27T06:41:28.395699+0000","last_unstale":"2024-03-27T06:43:34.604510+0000","last_undegraded":"2024-03-27T06:43:34.604510+0000","last_fullsized":"2024-03-27T06:43:34.604510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:00:59.679843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.053523+0000","last_change":"2024-03-27T06:41:29.401668+0000","last_active":"2024-03-27T06:43:34.053523+0000","last_peered":"2024-03-27T06:43:34.053523+0000","last_clean":"2024-03-27T06:43:34.053523+0000","last_became_active":"2024-03-27T06:41:29.401328+0000","last_became_peered":"2024-03-27T06:41:29.401328+0000","last_unstale":"2024-03-27T06:43:34.053523+0000","last_undegraded":"2024-03-27T06:43:34.053523+0000","last_fullsized":"2024-03-27T06:43:34.053523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:32:54.273194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789142+0000","last_change":"2024-03-27T06:42:21.011596+0000","last_active":"2024-03-27T06:43:33.789142+0000","last_peered":"2024-03-27T06:43:33.789142+0000","last_clean":"2024-03-27T06:43:33.789142+0000","last_became_active":"2024-03-27T06:42:21.005035+0000","last_became_peered":"2024-03-27T06:42:21.005035+0000","last_unstale":"2024-03-27T06:43:33.789142+0000","last_undegraded":"2024-03-27T06:43:33.789142+0000","last_fullsized":"2024-03-27T06:43:33.789142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:51:35.652266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044846+0000","last_change":"2024-03-27T06:41:29.413919+0000","last_active":"2024-03-27T06:43:34.044846+0000","last_peered":"2024-03-27T06:43:34.044846+0000","last_clean":"2024-03-27T06:43:34.044846+0000","last_became_active":"2024-03-27T06:41:29.413566+0000","last_became_peered":"2024-03-27T06:41:29.413566+0000","last_unstale":"2024-03-27T06:43:34.044846+0000","last_undegraded":"2024-03-27T06:43:34.044846+0000","last_fullsized":"2024-03-27T06:43:34.044846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:45:03.705372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899758+0000","last_change":"2024-03-27T06:41:28.391661+0000","last_active":"2024-03-27T06:43:33.899758+0000","last_peered":"2024-03-27T06:43:33.899758+0000","last_clean":"2024-03-27T06:43:33.899758+0000","last_became_active":"2024-03-27T06:41:28.391230+0000","last_became_peered":"2024-03-27T06:41:28.391230+0000","last_unstale":"2024-03-27T06:43:33.899758+0000","last_undegraded":"2024-03-27T06:43:33.899758+0000","last_fullsized":"2024-03-27T06:43:33.899758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:41:22.789267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052236+0000","last_change":"2024-03-27T06:41:29.405727+0000","last_active":"2024-03-27T06:43:34.052236+0000","last_peered":"2024-03-27T06:43:34.052236+0000","last_clean":"2024-03-27T06:43:34.052236+0000","last_became_active":"2024-03-27T06:41:29.405413+0000","last_became_peered":"2024-03-27T06:41:29.405413+0000","last_unstale":"2024-03-27T06:43:34.052236+0000","last_undegraded":"2024-03-27T06:43:34.052236+0000","last_fullsized":"2024-03-27T06:43:34.052236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:35:21.575729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":308,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.605048+0000","last_change":"2024-03-27T06:41:28.391155+0000","last_active":"2024-03-27T06:43:34.605048+0000","last_peered":"2024-03-27T06:43:34.605048+0000","last_clean":"2024-03-27T06:43:34.605048+0000","last_became_active":"2024-03-27T06:41:28.390828+0000","last_became_peered":"2024-03-27T06:41:28.390828+0000","last_unstale":"2024-03-27T06:43:34.605048+0000","last_undegraded":"2024-03-27T06:43:34.605048+0000","last_fullsized":"2024-03-27T06:43:34.605048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:40.192560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66735,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":190,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890788+0000","last_change":"2024-03-27T06:41:29.413283+0000","last_active":"2024-03-27T06:43:33.890788+0000","last_peered":"2024-03-27T06:43:33.890788+0000","last_clean":"2024-03-27T06:43:33.890788+0000","last_became_active":"2024-03-27T06:41:29.412247+0000","last_became_peered":"2024-03-27T06:41:29.412247+0000","last_unstale":"2024-03-27T06:43:33.890788+0000","last_undegraded":"2024-03-27T06:43:33.890788+0000","last_fullsized":"2024-03-27T06:43:33.890788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:50:15.336003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788566+0000","last_change":"2024-03-27T06:41:28.395089+0000","last_active":"2024-03-27T06:43:33.788566+0000","last_peered":"2024-03-27T06:43:33.788566+0000","last_clean":"2024-03-27T06:43:33.788566+0000","last_became_active":"2024-03-27T06:41:28.394145+0000","last_became_peered":"2024-03-27T06:41:28.394145+0000","last_unstale":"2024-03-27T06:43:33.788566+0000","last_undegraded":"2024-03-27T06:43:33.788566+0000","last_fullsized":"2024-03-27T06:43:33.788566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:40.858404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050686+0000","last_change":"2024-03-27T06:41:29.416955+0000","last_active":"2024-03-27T06:43:34.050686+0000","last_peered":"2024-03-27T06:43:34.050686+0000","last_clean":"2024-03-27T06:43:34.050686+0000","last_became_active":"2024-03-27T06:41:29.416616+0000","last_became_peered":"2024-03-27T06:41:29.416616+0000","last_unstale":"2024-03-27T06:43:34.050686+0000","last_undegraded":"2024-03-27T06:43:34.050686+0000","last_fullsized":"2024-03-27T06:43:34.050686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:38:18.791602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899329+0000","last_change":"2024-03-27T06:41:28.393876+0000","last_active":"2024-03-27T06:43:33.899329+0000","last_peered":"2024-03-27T06:43:33.899329+0000","last_clean":"2024-03-27T06:43:33.899329+0000","last_became_active":"2024-03-27T06:41:28.393557+0000","last_became_peered":"2024-03-27T06:41:28.393557+0000","last_unstale":"2024-03-27T06:43:33.899329+0000","last_undegraded":"2024-03-27T06:43:33.899329+0000","last_fullsized":"2024-03-27T06:43:33.899329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:30.831039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788517+0000","last_change":"2024-03-27T06:42:20.660026+0000","last_active":"2024-03-27T06:43:33.788517+0000","last_peered":"2024-03-27T06:43:33.788517+0000","last_clean":"2024-03-27T06:43:33.788517+0000","last_became_active":"2024-03-27T06:42:20.659628+0000","last_became_peered":"2024-03-27T06:42:20.659628+0000","last_unstale":"2024-03-27T06:43:33.788517+0000","last_undegraded":"2024-03-27T06:43:33.788517+0000","last_fullsized":"2024-03-27T06:43:33.788517+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:18:59.396472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607543+0000","last_change":"2024-03-27T06:41:28.393849+0000","last_active":"2024-03-27T06:43:34.607543+0000","last_peered":"2024-03-27T06:43:34.607543+0000","last_clean":"2024-03-27T06:43:34.607543+0000","last_became_active":"2024-03-27T06:41:28.393529+0000","last_became_peered":"2024-03-27T06:41:28.393529+0000","last_unstale":"2024-03-27T06:43:34.607543+0000","last_undegraded":"2024-03-27T06:43:34.607543+0000","last_fullsized":"2024-03-27T06:43:34.607543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:24:23.251723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.787942+0000","last_change":"2024-03-27T06:41:29.411273+0000","last_active":"2024-03-27T06:43:33.787942+0000","last_peered":"2024-03-27T06:43:33.787942+0000","last_clean":"2024-03-27T06:43:33.787942+0000","last_became_active":"2024-03-27T06:41:29.410936+0000","last_became_peered":"2024-03-27T06:41:29.410936+0000","last_unstale":"2024-03-27T06:43:33.787942+0000","last_undegraded":"2024-03-27T06:43:33.787942+0000","last_fullsized":"2024-03-27T06:43:33.787942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:41.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043527+0000","last_change":"2024-03-27T06:42:20.673735+0000","last_active":"2024-03-27T06:43:34.043527+0000","last_peered":"2024-03-27T06:43:34.043527+0000","last_clean":"2024-03-27T06:43:34.043527+0000","last_became_active":"2024-03-27T06:42:20.655144+0000","last_became_peered":"2024-03-27T06:42:20.655144+0000","last_unstale":"2024-03-27T06:43:34.043527+0000","last_undegraded":"2024-03-27T06:43:34.043527+0000","last_fullsized":"2024-03-27T06:43:34.043527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:05:39.059633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899135+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:43:33.899135+0000","last_peered":"2024-03-27T06:43:33.899135+0000","last_clean":"2024-03-27T06:43:33.899135+0000","last_became_active":"2024-03-27T06:41:29.408815+0000","last_became_peered":"2024-03-27T06:41:29.408815+0000","last_unstale":"2024-03-27T06:43:33.899135+0000","last_undegraded":"2024-03-27T06:43:33.899135+0000","last_fullsized":"2024-03-27T06:43:33.899135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:45.547747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891036+0000","last_change":"2024-03-27T06:41:28.398249+0000","last_active":"2024-03-27T06:43:33.891036+0000","last_peered":"2024-03-27T06:43:33.891036+0000","last_clean":"2024-03-27T06:43:33.891036+0000","last_became_active":"2024-03-27T06:41:28.397176+0000","last_became_peered":"2024-03-27T06:41:28.397176+0000","last_unstale":"2024-03-27T06:43:33.891036+0000","last_undegraded":"2024-03-27T06:43:33.891036+0000","last_fullsized":"2024-03-27T06:43:33.891036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:32.614558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044019+0000","last_change":"2024-03-27T06:41:29.412928+0000","last_active":"2024-03-27T06:43:34.044019+0000","last_peered":"2024-03-27T06:43:34.044019+0000","last_clean":"2024-03-27T06:43:34.044019+0000","last_became_active":"2024-03-27T06:41:29.412576+0000","last_became_peered":"2024-03-27T06:41:29.412576+0000","last_unstale":"2024-03-27T06:43:34.044019+0000","last_undegraded":"2024-03-27T06:43:34.044019+0000","last_fullsized":"2024-03-27T06:43:34.044019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:01:22.389424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.786409+0000","last_change":"2024-03-27T06:41:28.397135+0000","last_active":"2024-03-27T06:43:33.786409+0000","last_peered":"2024-03-27T06:43:33.786409+0000","last_clean":"2024-03-27T06:43:33.786409+0000","last_became_active":"2024-03-27T06:41:28.396483+0000","last_became_peered":"2024-03-27T06:41:28.396483+0000","last_unstale":"2024-03-27T06:43:33.786409+0000","last_undegraded":"2024-03-27T06:43:33.786409+0000","last_fullsized":"2024-03-27T06:43:33.786409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:05:02.038024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902974+0000","last_change":"2024-03-27T06:41:29.418629+0000","last_active":"2024-03-27T06:43:33.902974+0000","last_peered":"2024-03-27T06:43:33.902974+0000","last_clean":"2024-03-27T06:43:33.902974+0000","last_became_active":"2024-03-27T06:41:29.418278+0000","last_became_peered":"2024-03-27T06:41:29.418278+0000","last_unstale":"2024-03-27T06:43:33.902974+0000","last_undegraded":"2024-03-27T06:43:33.902974+0000","last_fullsized":"2024-03-27T06:43:33.902974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:33:54.461349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.608536+0000","last_change":"2024-03-27T06:41:28.400611+0000","last_active":"2024-03-27T06:43:34.608536+0000","last_peered":"2024-03-27T06:43:34.608536+0000","last_clean":"2024-03-27T06:43:34.608536+0000","last_became_active":"2024-03-27T06:41:28.400211+0000","last_became_peered":"2024-03-27T06:41:28.400211+0000","last_unstale":"2024-03-27T06:43:34.608536+0000","last_undegraded":"2024-03-27T06:43:34.608536+0000","last_fullsized":"2024-03-27T06:43:34.608536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:46:35.491071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902979+0000","last_change":"2024-03-27T06:41:29.413246+0000","last_active":"2024-03-27T06:43:33.902979+0000","last_peered":"2024-03-27T06:43:33.902979+0000","last_clean":"2024-03-27T06:43:33.902979+0000","last_became_active":"2024-03-27T06:41:29.412946+0000","last_became_peered":"2024-03-27T06:41:29.412946+0000","last_unstale":"2024-03-27T06:43:33.902979+0000","last_undegraded":"2024-03-27T06:43:33.902979+0000","last_fullsized":"2024-03-27T06:43:33.902979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:41:51.441233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899393+0000","last_change":"2024-03-27T06:41:29.402160+0000","last_active":"2024-03-27T06:43:33.899393+0000","last_peered":"2024-03-27T06:43:33.899393+0000","last_clean":"2024-03-27T06:43:33.899393+0000","last_became_active":"2024-03-27T06:41:29.401789+0000","last_became_peered":"2024-03-27T06:41:29.401789+0000","last_unstale":"2024-03-27T06:43:33.899393+0000","last_undegraded":"2024-03-27T06:43:33.899393+0000","last_fullsized":"2024-03-27T06:43:33.899393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:31:27.821966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607060+0000","last_change":"2024-03-27T06:41:28.395978+0000","last_active":"2024-03-27T06:43:34.607060+0000","last_peered":"2024-03-27T06:43:34.607060+0000","last_clean":"2024-03-27T06:43:34.607060+0000","last_became_active":"2024-03-27T06:41:28.395650+0000","last_became_peered":"2024-03-27T06:41:28.395650+0000","last_unstale":"2024-03-27T06:43:34.607060+0000","last_undegraded":"2024-03-27T06:43:34.607060+0000","last_fullsized":"2024-03-27T06:43:34.607060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:01:15.841344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890934+0000","last_change":"2024-03-27T06:41:29.423303+0000","last_active":"2024-03-27T06:43:33.890934+0000","last_peered":"2024-03-27T06:43:33.890934+0000","last_clean":"2024-03-27T06:43:33.890934+0000","last_became_active":"2024-03-27T06:41:29.422977+0000","last_became_peered":"2024-03-27T06:41:29.422977+0000","last_unstale":"2024-03-27T06:43:33.890934+0000","last_undegraded":"2024-03-27T06:43:33.890934+0000","last_fullsized":"2024-03-27T06:43:33.890934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:14:01.877801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051856+0000","last_change":"2024-03-27T06:41:28.395141+0000","last_active":"2024-03-27T06:43:34.051856+0000","last_peered":"2024-03-27T06:43:34.051856+0000","last_clean":"2024-03-27T06:43:34.051856+0000","last_became_active":"2024-03-27T06:41:28.394826+0000","last_became_peered":"2024-03-27T06:41:28.394826+0000","last_unstale":"2024-03-27T06:43:34.051856+0000","last_undegraded":"2024-03-27T06:43:34.051856+0000","last_fullsized":"2024-03-27T06:43:34.051856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:29:49.997090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788039+0000","last_change":"2024-03-27T06:40:16.460939+0000","last_active":"2024-03-27T06:43:33.788039+0000","last_peered":"2024-03-27T06:43:33.788039+0000","last_clean":"2024-03-27T06:43:33.788039+0000","last_became_active":"2024-03-27T06:40:16.460453+0000","last_became_peered":"2024-03-27T06:40:16.460453+0000","last_unstale":"2024-03-27T06:43:33.788039+0000","last_undegraded":"2024-03-27T06:43:33.788039+0000","last_fullsized":"2024-03-27T06:43:33.788039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_clean_scrub_stamp":"2024-03-27T06:36:48.325777+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:28.241688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050380+0000","last_change":"2024-03-27T06:42:20.658690+0000","last_active":"2024-03-27T06:43:34.050380+0000","last_peered":"2024-03-27T06:43:34.050380+0000","last_clean":"2024-03-27T06:43:34.050380+0000","last_became_active":"2024-03-27T06:42:20.658348+0000","last_became_peered":"2024-03-27T06:42:20.658348+0000","last_unstale":"2024-03-27T06:43:34.050380+0000","last_undegraded":"2024-03-27T06:43:34.050380+0000","last_fullsized":"2024-03-27T06:43:34.050380+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:12:29.985173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788194+0000","last_change":"2024-03-27T06:41:28.399205+0000","last_active":"2024-03-27T06:43:33.788194+0000","last_peered":"2024-03-27T06:43:33.788194+0000","last_clean":"2024-03-27T06:43:33.788194+0000","last_became_active":"2024-03-27T06:41:28.398889+0000","last_became_peered":"2024-03-27T06:41:28.398889+0000","last_unstale":"2024-03-27T06:43:33.788194+0000","last_undegraded":"2024-03-27T06:43:33.788194+0000","last_fullsized":"2024-03-27T06:43:33.788194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:28:08.769187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043816+0000","last_change":"2024-03-27T06:42:20.663370+0000","last_active":"2024-03-27T06:43:34.043816+0000","last_peered":"2024-03-27T06:43:34.043816+0000","last_clean":"2024-03-27T06:43:34.043816+0000","last_became_active":"2024-03-27T06:42:20.654243+0000","last_became_peered":"2024-03-27T06:42:20.654243+0000","last_unstale":"2024-03-27T06:43:34.043816+0000","last_undegraded":"2024-03-27T06:43:34.043816+0000","last_fullsized":"2024-03-27T06:43:34.043816+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:43:46.566952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051012+0000","last_change":"2024-03-27T06:41:29.395466+0000","last_active":"2024-03-27T06:43:34.051012+0000","last_peered":"2024-03-27T06:43:34.051012+0000","last_clean":"2024-03-27T06:43:34.051012+0000","last_became_active":"2024-03-27T06:41:29.395094+0000","last_became_peered":"2024-03-27T06:41:29.395094+0000","last_unstale":"2024-03-27T06:43:34.051012+0000","last_undegraded":"2024-03-27T06:43:34.051012+0000","last_fullsized":"2024-03-27T06:43:34.051012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:29:01.746809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'48","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.011094+0000","last_change":"2024-03-27T06:41:28.400420+0000","last_active":"2024-03-27T06:43:34.011094+0000","last_peered":"2024-03-27T06:43:34.011094+0000","last_clean":"2024-03-27T06:43:34.011094+0000","last_became_active":"2024-03-27T06:41:28.400115+0000","last_became_peered":"2024-03-27T06:41:28.400115+0000","last_unstale":"2024-03-27T06:43:34.011094+0000","last_undegraded":"2024-03-27T06:43:34.011094+0000","last_fullsized":"2024-03-27T06:43:34.011094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:37:40.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891028+0000","last_change":"2024-03-27T06:41:29.411428+0000","last_active":"2024-03-27T06:43:33.891028+0000","last_peered":"2024-03-27T06:43:33.891028+0000","last_clean":"2024-03-27T06:43:33.891028+0000","last_became_active":"2024-03-27T06:41:29.411075+0000","last_became_peered":"2024-03-27T06:41:29.411075+0000","last_unstale":"2024-03-27T06:43:33.891028+0000","last_undegraded":"2024-03-27T06:43:33.891028+0000","last_fullsized":"2024-03-27T06:43:33.891028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:55.764459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052106+0000","last_change":"2024-03-27T06:41:28.399799+0000","last_active":"2024-03-27T06:43:34.052106+0000","last_peered":"2024-03-27T06:43:34.052106+0000","last_clean":"2024-03-27T06:43:34.052106+0000","last_became_active":"2024-03-27T06:41:28.399385+0000","last_became_peered":"2024-03-27T06:41:28.399385+0000","last_unstale":"2024-03-27T06:43:34.052106+0000","last_undegraded":"2024-03-27T06:43:34.052106+0000","last_fullsized":"2024-03-27T06:43:34.052106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:16:18.005519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044636+0000","last_change":"2024-03-27T06:41:29.418991+0000","last_active":"2024-03-27T06:43:34.044636+0000","last_peered":"2024-03-27T06:43:34.044636+0000","last_clean":"2024-03-27T06:43:34.044636+0000","last_became_active":"2024-03-27T06:41:29.418527+0000","last_became_peered":"2024-03-27T06:41:29.418527+0000","last_unstale":"2024-03-27T06:43:34.044636+0000","last_undegraded":"2024-03-27T06:43:34.044636+0000","last_fullsized":"2024-03-27T06:43:34.044636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:49:55.913276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051676+0000","last_change":"2024-03-27T06:41:28.397771+0000","last_active":"2024-03-27T06:43:34.051676+0000","last_peered":"2024-03-27T06:43:34.051676+0000","last_clean":"2024-03-27T06:43:34.051676+0000","last_became_active":"2024-03-27T06:41:28.397470+0000","last_became_peered":"2024-03-27T06:41:28.397470+0000","last_unstale":"2024-03-27T06:43:34.051676+0000","last_undegraded":"2024-03-27T06:43:34.051676+0000","last_fullsized":"2024-03-27T06:43:34.051676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:37:21.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890518+0000","last_change":"2024-03-27T06:41:29.412280+0000","last_active":"2024-03-27T06:43:33.890518+0000","last_peered":"2024-03-27T06:43:33.890518+0000","last_clean":"2024-03-27T06:43:33.890518+0000","last_became_active":"2024-03-27T06:41:29.411336+0000","last_became_peered":"2024-03-27T06:41:29.411336+0000","last_unstale":"2024-03-27T06:43:33.890518+0000","last_undegraded":"2024-03-27T06:43:33.890518+0000","last_fullsized":"2024-03-27T06:43:33.890518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:13:03.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890676+0000","last_change":"2024-03-27T06:41:28.403529+0000","last_active":"2024-03-27T06:43:33.890676+0000","last_peered":"2024-03-27T06:43:33.890676+0000","last_clean":"2024-03-27T06:43:33.890676+0000","last_became_active":"2024-03-27T06:41:28.403155+0000","last_became_peered":"2024-03-27T06:41:28.403155+0000","last_unstale":"2024-03-27T06:43:33.890676+0000","last_undegraded":"2024-03-27T06:43:33.890676+0000","last_fullsized":"2024-03-27T06:43:33.890676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:23:53.971374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043609+0000","last_change":"2024-03-27T06:41:29.404929+0000","last_active":"2024-03-27T06:43:34.043609+0000","last_peered":"2024-03-27T06:43:34.043609+0000","last_clean":"2024-03-27T06:43:34.043609+0000","last_became_active":"2024-03-27T06:41:29.404602+0000","last_became_peered":"2024-03-27T06:41:29.404602+0000","last_unstale":"2024-03-27T06:43:34.043609+0000","last_undegraded":"2024-03-27T06:43:34.043609+0000","last_fullsized":"2024-03-27T06:43:34.043609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:19:07.379046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788341+0000","last_change":"2024-03-27T06:41:28.398602+0000","last_active":"2024-03-27T06:43:33.788341+0000","last_peered":"2024-03-27T06:43:33.788341+0000","last_clean":"2024-03-27T06:43:33.788341+0000","last_became_active":"2024-03-27T06:41:28.398068+0000","last_became_peered":"2024-03-27T06:41:28.398068+0000","last_unstale":"2024-03-27T06:43:33.788341+0000","last_undegraded":"2024-03-27T06:43:33.788341+0000","last_fullsized":"2024-03-27T06:43:33.788341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:10:53.312044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891033+0000","last_change":"2024-03-27T06:41:29.420245+0000","last_active":"2024-03-27T06:43:33.891033+0000","last_peered":"2024-03-27T06:43:33.891033+0000","last_clean":"2024-03-27T06:43:33.891033+0000","last_became_active":"2024-03-27T06:41:29.419912+0000","last_became_peered":"2024-03-27T06:41:29.419912+0000","last_unstale":"2024-03-27T06:43:33.891033+0000","last_undegraded":"2024-03-27T06:43:33.891033+0000","last_fullsized":"2024-03-27T06:43:33.891033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:46:58.203083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890685+0000","last_change":"2024-03-27T06:41:28.398595+0000","last_active":"2024-03-27T06:43:33.890685+0000","last_peered":"2024-03-27T06:43:33.890685+0000","last_clean":"2024-03-27T06:43:33.890685+0000","last_became_active":"2024-03-27T06:41:28.398036+0000","last_became_peered":"2024-03-27T06:41:28.398036+0000","last_unstale":"2024-03-27T06:43:33.890685+0000","last_undegraded":"2024-03-27T06:43:33.890685+0000","last_fullsized":"2024-03-27T06:43:33.890685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:47:15.769760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788967+0000","last_change":"2024-03-27T06:41:29.406576+0000","last_active":"2024-03-27T06:43:33.788967+0000","last_peered":"2024-03-27T06:43:33.788967+0000","last_clean":"2024-03-27T06:43:33.788967+0000","last_became_active":"2024-03-27T06:41:29.406222+0000","last_became_peered":"2024-03-27T06:41:29.406222+0000","last_unstale":"2024-03-27T06:43:33.788967+0000","last_undegraded":"2024-03-27T06:43:33.788967+0000","last_fullsized":"2024-03-27T06:43:33.788967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:05:12.446154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607123+0000","last_change":"2024-03-27T06:41:28.399361+0000","last_active":"2024-03-27T06:43:34.607123+0000","last_peered":"2024-03-27T06:43:34.607123+0000","last_clean":"2024-03-27T06:43:34.607123+0000","last_became_active":"2024-03-27T06:41:28.398021+0000","last_became_peered":"2024-03-27T06:41:28.398021+0000","last_unstale":"2024-03-27T06:43:34.607123+0000","last_undegraded":"2024-03-27T06:43:34.607123+0000","last_fullsized":"2024-03-27T06:43:34.607123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:51:34.106340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789247+0000","last_change":"2024-03-27T06:41:29.405696+0000","last_active":"2024-03-27T06:43:33.789247+0000","last_peered":"2024-03-27T06:43:33.789247+0000","last_clean":"2024-03-27T06:43:33.789247+0000","last_became_active":"2024-03-27T06:41:29.405273+0000","last_became_peered":"2024-03-27T06:41:29.405273+0000","last_unstale":"2024-03-27T06:43:33.789247+0000","last_undegraded":"2024-03-27T06:43:33.789247+0000","last_fullsized":"2024-03-27T06:43:33.789247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:02:58.778464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'41","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.006432+0000","last_change":"2024-03-27T06:41:28.391406+0000","last_active":"2024-03-27T06:43:34.006432+0000","last_peered":"2024-03-27T06:43:34.006432+0000","last_clean":"2024-03-27T06:43:34.006432+0000","last_became_active":"2024-03-27T06:41:28.390950+0000","last_became_peered":"2024-03-27T06:41:28.390950+0000","last_unstale":"2024-03-27T06:43:34.006432+0000","last_undegraded":"2024-03-27T06:43:34.006432+0000","last_fullsized":"2024-03-27T06:43:34.006432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:10:48.347139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788757+0000","last_change":"2024-03-27T06:41:29.410422+0000","last_active":"2024-03-27T06:43:33.788757+0000","last_peered":"2024-03-27T06:43:33.788757+0000","last_clean":"2024-03-27T06:43:33.788757+0000","last_became_active":"2024-03-27T06:41:29.409738+0000","last_became_peered":"2024-03-27T06:41:29.409738+0000","last_unstale":"2024-03-27T06:43:33.788757+0000","last_undegraded":"2024-03-27T06:43:33.788757+0000","last_fullsized":"2024-03-27T06:43:33.788757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:38:17.546504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051045+0000","last_change":"2024-03-27T06:42:21.008586+0000","last_active":"2024-03-27T06:43:34.051045+0000","last_peered":"2024-03-27T06:43:34.051045+0000","last_clean":"2024-03-27T06:43:34.051045+0000","last_became_active":"2024-03-27T06:42:21.004964+0000","last_became_peered":"2024-03-27T06:42:21.004964+0000","last_unstale":"2024-03-27T06:43:34.051045+0000","last_undegraded":"2024-03-27T06:43:34.051045+0000","last_fullsized":"2024-03-27T06:43:34.051045+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:11:13.246994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789419+0000","last_change":"2024-03-27T06:41:29.408809+0000","last_active":"2024-03-27T06:43:33.789419+0000","last_peered":"2024-03-27T06:43:33.789419+0000","last_clean":"2024-03-27T06:43:33.789419+0000","last_became_active":"2024-03-27T06:41:29.408447+0000","last_became_peered":"2024-03-27T06:41:29.408447+0000","last_unstale":"2024-03-27T06:43:33.789419+0000","last_undegraded":"2024-03-27T06:43:33.789419+0000","last_fullsized":"2024-03-27T06:43:33.789419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:04:55.269490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043772+0000","last_change":"2024-03-27T06:42:20.655859+0000","last_active":"2024-03-27T06:43:34.043772+0000","last_peered":"2024-03-27T06:43:34.043772+0000","last_clean":"2024-03-27T06:43:34.043772+0000","last_became_active":"2024-03-27T06:42:20.655446+0000","last_became_peered":"2024-03-27T06:42:20.655446+0000","last_unstale":"2024-03-27T06:43:34.043772+0000","last_undegraded":"2024-03-27T06:43:34.043772+0000","last_fullsized":"2024-03-27T06:43:34.043772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:43:07.920040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.787928+0000","last_change":"2024-03-27T06:41:29.409896+0000","last_active":"2024-03-27T06:43:33.787928+0000","last_peered":"2024-03-27T06:43:33.787928+0000","last_clean":"2024-03-27T06:43:33.787928+0000","last_became_active":"2024-03-27T06:41:29.408279+0000","last_became_peered":"2024-03-27T06:41:29.408279+0000","last_unstale":"2024-03-27T06:43:33.787928+0000","last_undegraded":"2024-03-27T06:43:33.787928+0000","last_fullsized":"2024-03-27T06:43:33.787928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:49:03.257543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788142+0000","last_change":"2024-03-27T06:41:29.412526+0000","last_active":"2024-03-27T06:43:33.788142+0000","last_peered":"2024-03-27T06:43:33.788142+0000","last_clean":"2024-03-27T06:43:33.788142+0000","last_became_active":"2024-03-27T06:41:29.412180+0000","last_became_peered":"2024-03-27T06:41:29.412180+0000","last_unstale":"2024-03-27T06:43:33.788142+0000","last_undegraded":"2024-03-27T06:43:33.788142+0000","last_fullsized":"2024-03-27T06:43:33.788142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:47.461457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044158+0000","last_change":"2024-03-27T06:42:20.657735+0000","last_active":"2024-03-27T06:43:34.044158+0000","last_peered":"2024-03-27T06:43:34.044158+0000","last_clean":"2024-03-27T06:43:34.044158+0000","last_became_active":"2024-03-27T06:42:20.657378+0000","last_became_peered":"2024-03-27T06:42:20.657378+0000","last_unstale":"2024-03-27T06:43:34.044158+0000","last_undegraded":"2024-03-27T06:43:34.044158+0000","last_fullsized":"2024-03-27T06:43:34.044158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:13:56.368229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051027+0000","last_change":"2024-03-27T06:41:29.402442+0000","last_active":"2024-03-27T06:43:34.051027+0000","last_peered":"2024-03-27T06:43:34.051027+0000","last_clean":"2024-03-27T06:43:34.051027+0000","last_became_active":"2024-03-27T06:41:29.402078+0000","last_became_peered":"2024-03-27T06:41:29.402078+0000","last_unstale":"2024-03-27T06:43:34.051027+0000","last_undegraded":"2024-03-27T06:43:34.051027+0000","last_fullsized":"2024-03-27T06:43:34.051027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:07:35.344881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044286+0000","last_change":"2024-03-27T06:41:29.418563+0000","last_active":"2024-03-27T06:43:34.044286+0000","last_peered":"2024-03-27T06:43:34.044286+0000","last_clean":"2024-03-27T06:43:34.044286+0000","last_became_active":"2024-03-27T06:41:29.418209+0000","last_became_peered":"2024-03-27T06:41:29.418209+0000","last_unstale":"2024-03-27T06:43:34.044286+0000","last_undegraded":"2024-03-27T06:43:34.044286+0000","last_fullsized":"2024-03-27T06:43:34.044286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:53:58.536148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789184+0000","last_change":"2024-03-27T06:41:29.400414+0000","last_active":"2024-03-27T06:43:33.789184+0000","last_peered":"2024-03-27T06:43:33.789184+0000","last_clean":"2024-03-27T06:43:33.789184+0000","last_became_active":"2024-03-27T06:41:29.397865+0000","last_became_peered":"2024-03-27T06:41:29.397865+0000","last_unstale":"2024-03-27T06:43:33.789184+0000","last_undegraded":"2024-03-27T06:43:33.789184+0000","last_fullsized":"2024-03-27T06:43:33.789184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:35:28.405347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890973+0000","last_change":"2024-03-27T06:41:29.416740+0000","last_active":"2024-03-27T06:43:33.890973+0000","last_peered":"2024-03-27T06:43:33.890973+0000","last_clean":"2024-03-27T06:43:33.890973+0000","last_became_active":"2024-03-27T06:41:29.416417+0000","last_became_peered":"2024-03-27T06:41:29.416417+0000","last_unstale":"2024-03-27T06:43:33.890973+0000","last_undegraded":"2024-03-27T06:43:33.890973+0000","last_fullsized":"2024-03-27T06:43:33.890973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:08:34.186527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899169+0000","last_change":"2024-03-27T06:41:29.418797+0000","last_active":"2024-03-27T06:43:33.899169+0000","last_peered":"2024-03-27T06:43:33.899169+0000","last_clean":"2024-03-27T06:43:33.899169+0000","last_became_active":"2024-03-27T06:41:29.418472+0000","last_became_peered":"2024-03-27T06:41:29.418472+0000","last_unstale":"2024-03-27T06:43:33.899169+0000","last_undegraded":"2024-03-27T06:43:33.899169+0000","last_fullsized":"2024-03-27T06:43:33.899169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:30:55.540353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890611+0000","last_change":"2024-03-27T06:41:29.416124+0000","last_active":"2024-03-27T06:43:33.890611+0000","last_peered":"2024-03-27T06:43:33.890611+0000","last_clean":"2024-03-27T06:43:33.890611+0000","last_became_active":"2024-03-27T06:41:29.415665+0000","last_became_peered":"2024-03-27T06:41:29.415665+0000","last_unstale":"2024-03-27T06:43:33.890611+0000","last_undegraded":"2024-03-27T06:43:33.890611+0000","last_fullsized":"2024-03-27T06:43:33.890611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:46:04.160705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.053062+0000","last_change":"2024-03-27T06:41:29.417165+0000","last_active":"2024-03-27T06:43:34.053062+0000","last_peered":"2024-03-27T06:43:34.053062+0000","last_clean":"2024-03-27T06:43:34.053062+0000","last_became_active":"2024-03-27T06:41:29.416856+0000","last_became_peered":"2024-03-27T06:41:29.416856+0000","last_unstale":"2024-03-27T06:43:34.053062+0000","last_undegraded":"2024-03-27T06:43:34.053062+0000","last_fullsized":"2024-03-27T06:43:34.053062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:55:20.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050772+0000","last_change":"2024-03-27T06:41:29.412377+0000","last_active":"2024-03-27T06:43:34.050772+0000","last_peered":"2024-03-27T06:43:34.050772+0000","last_clean":"2024-03-27T06:43:34.050772+0000","last_became_active":"2024-03-27T06:41:29.411761+0000","last_became_peered":"2024-03-27T06:41:29.411761+0000","last_unstale":"2024-03-27T06:43:34.050772+0000","last_undegraded":"2024-03-27T06:43:34.050772+0000","last_fullsized":"2024-03-27T06:43:34.050772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:16:56.844342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043855+0000","last_change":"2024-03-27T06:41:29.413660+0000","last_active":"2024-03-27T06:43:34.043855+0000","last_peered":"2024-03-27T06:43:34.043855+0000","last_clean":"2024-03-27T06:43:34.043855+0000","last_became_active":"2024-03-27T06:41:29.413201+0000","last_became_peered":"2024-03-27T06:41:29.413201+0000","last_unstale":"2024-03-27T06:43:34.043855+0000","last_undegraded":"2024-03-27T06:43:34.043855+0000","last_fullsized":"2024-03-27T06:43:34.043855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:14:46.446595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106419,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600992,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1335,"num_read_kb":783,"num_write":374,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1802976,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1155072,"data_stored":1259064,"data_compressed":44525,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":861923,"data_compressed":22042,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037813,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":862077,"data_compressed":22042,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":1063449,"data_compressed":38369,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":520990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":470545,"data_compressed":2896,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":537997,"data_compressed":2896,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":993065,"data_compressed":28198,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":459890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1193413,"data_compressed":44525,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135343,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":394492,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200994,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T06:43:38.717 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-27T06:43:38.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-27T06:43:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.865+0000 7f16ea883640 1 -- 172.21.15.16:0/3551813295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16e41013d0 msgr2=0x7f16e410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.865+0000 7f16ea883640 1 --2- 172.21.15.16:0/3551813295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16e41013d0 0x7f16e410d8e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009cd0 tx=0x7f16dc02c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.865+0000 7f16ea883640 1 -- 172.21.15.16:0/3551813295 shutdown_connections 2024-03-27T06:43:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.865+0000 7f16ea883640 1 --2- 172.21.15.16:0/3551813295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16e41013d0 0x7f16e410d8e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.865+0000 7f16ea883640 1 --2- 172.21.15.16:0/3551813295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16e4100a10 0x7f16e4100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.865+0000 7f16ea883640 1 --2- 172.21.15.16:0/3551813295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16e4106a20 0x7f16e4106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.865+0000 7f16ea883640 1 -- 172.21.15.16:0/3551813295 >> 172.21.15.16:0/3551813295 conn(0x7f16e4077af0 msgr2=0x7f16e4077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:38.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.865+0000 7f16ea883640 1 -- 172.21.15.16:0/3551813295 shutdown_connections 2024-03-27T06:43:38.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.865+0000 7f16ea883640 1 -- 172.21.15.16:0/3551813295 wait complete. 2024-03-27T06:43:38.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.865+0000 7f16ea883640 1 Processor -- start 2024-03-27T06:43:38.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.865+0000 7f16ea883640 1 -- start start 2024-03-27T06:43:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16ea883640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16e4100a10 0x7f16e4197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16ea883640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16e41013d0 0x7f16e41978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16ea883640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16e4106a20 0x7f16e419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16ea883640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16e4110f50 con 0x7f16e4106a20 2024-03-27T06:43:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16ea883640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f16e4110dd0 con 0x7f16e41013d0 2024-03-27T06:43:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16ea883640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41110d0 con 0x7f16e4100a10 2024-03-27T06:43:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e8e20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16e4106a20 0x7f16e419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e8e20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16e4106a20 0x7f16e419bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51188/0 (socket says 172.21.15.16:51188) 2024-03-27T06:43:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e8e20640 1 -- 172.21.15.16:0/263402422 learned_addr learned my addr 172.21.15.16:0/263402422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:38.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e3fff640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16e4100a10 0x7f16e4197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e37fe640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16e41013d0 0x7f16e41978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e8e20640 1 -- 172.21.15.16:0/263402422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16e4100a10 msgr2=0x7f16e4197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e8e20640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16e4100a10 0x7f16e4197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e8e20640 1 -- 172.21.15.16:0/263402422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16e41013d0 msgr2=0x7f16e41978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:38.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e8e20640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16e41013d0 0x7f16e41978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:38.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e8e20640 1 -- 172.21.15.16:0/263402422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e419c360 con 0x7f16e4106a20 2024-03-27T06:43:38.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e3fff640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16e4100a10 0x7f16e4197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:38.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.866+0000 7f16e8e20640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16e4106a20 0x7f16e419bc30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f16dc004af0 tx=0x7f16dc032550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:38.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.867+0000 7f16e17fa640 1 -- 172.21.15.16:0/263402422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc039040 con 0x7f16e4106a20 2024-03-27T06:43:38.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.867+0000 7f16e17fa640 1 -- 172.21.15.16:0/263402422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16dc044070 con 0x7f16e4106a20 2024-03-27T06:43:38.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.867+0000 7f16ea883640 1 -- 172.21.15.16:0/263402422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e419c5f0 con 0x7f16e4106a20 2024-03-27T06:43:38.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.867+0000 7f16e17fa640 1 -- 172.21.15.16:0/263402422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0042a0 con 0x7f16e4106a20 2024-03-27T06:43:38.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.867+0000 7f16ea883640 1 -- 172.21.15.16:0/263402422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e41a3ee0 con 0x7f16e4106a20 2024-03-27T06:43:38.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.868+0000 7f16ea883640 1 -- 172.21.15.16:0/263402422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e4068d90 con 0x7f16e4106a20 2024-03-27T06:43:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.869+0000 7f16e17fa640 1 -- 172.21.15.16:0/263402422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7f16dc003970 con 0x7f16e4106a20 2024-03-27T06:43:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.869+0000 7f16e17fa640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f16cc0a38a0 0x7f16cc0a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.869+0000 7f16e17fa640 1 -- 172.21.15.16:0/263402422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f16dc0f2700 con 0x7f16e4106a20 2024-03-27T06:43:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.869+0000 7f16e3fff640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f16cc0a38a0 0x7f16cc0a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.870+0000 7f16e3fff640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f16cc0a38a0 0x7f16cc0a5d60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009f40 tx=0x7f16d4005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:38.874+0000 7f16e17fa640 1 -- 172.21.15.16:0/263402422 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f16dc0bef10 con 0x7f16e4106a20 2024-03-27T06:43:39.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.058+0000 7f16ea883640 1 -- 172.21.15.16:0/263402422 --> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16e4061f90 con 0x7f16cc0a38a0 2024-03-27T06:43:39.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.066+0000 7f16e17fa640 1 -- 172.21.15.16:0/263402422 <== mgr.15354 v2:172.21.15.16:6800/3715976676 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258017 (secure 0 0 0) 0x7f16e4061f90 con 0x7f16cc0a38a0 2024-03-27T06:43:39.063 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:39.064 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-03-27T06:43:39.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.068+0000 7f16ea883640 1 -- 172.21.15.16:0/263402422 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f16cc0a38a0 msgr2=0x7f16cc0a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.068+0000 7f16ea883640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f16cc0a38a0 0x7f16cc0a5d60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009f40 tx=0x7f16d4005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:39.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.068+0000 7f16ea883640 1 -- 172.21.15.16:0/263402422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16e4106a20 msgr2=0x7f16e419bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.068+0000 7f16ea883640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16e4106a20 0x7f16e419bc30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f16dc004af0 tx=0x7f16dc032550 comp rx=0 tx=0).stop 2024-03-27T06:43:39.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.069+0000 7f16ea883640 1 -- 172.21.15.16:0/263402422 shutdown_connections 2024-03-27T06:43:39.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.069+0000 7f16ea883640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16e4106a20 0x7f16e419bc30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.069+0000 7f16ea883640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16e41013d0 0x7f16e41978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.069+0000 7f16ea883640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f16cc0a38a0 0x7f16cc0a5d60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.069+0000 7f16ea883640 1 --2- 172.21.15.16:0/263402422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16e4100a10 0x7f16e4197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.069+0000 7f16ea883640 1 -- 172.21.15.16:0/263402422 >> 172.21.15.16:0/263402422 conn(0x7f16e4077af0 msgr2=0x7f16e40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.069+0000 7f16ea883640 1 -- 172.21.15.16:0/263402422 shutdown_connections 2024-03-27T06:43:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.069+0000 7f16ea883640 1 -- 172.21.15.16:0/263402422 wait complete. 2024-03-27T06:43:39.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: Updating smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:43:39.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: Updating smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:43:39.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: Updating smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:43:39.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:39.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/504699565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:39.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:43:39.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:43:39.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:43:39.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: Updating smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: Updating smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: Updating smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:43:39.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:43:39.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.076 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-03-27T06:43:37.799887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191594,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1471,"num_read_kb":902,"num_write":557,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1180960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":942,"ondisk_log_size":942,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369548,"kb_used_data":12052,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572276,"statfs":{"total":1151940427776,"available":1151562010624,"internally_reserved":0,"allocated":12341248,"data_stored":9141984,"data_compressed":205493,"data_compressed_allocated":266240,"data_compressed_original":3383296,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.986546"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044192+0000","last_change":"2024-03-27T06:41:29.408953+0000","last_active":"2024-03-27T06:43:34.044192+0000","last_peered":"2024-03-27T06:43:34.044192+0000","last_clean":"2024-03-27T06:43:34.044192+0000","last_became_active":"2024-03-27T06:41:29.408143+0000","last_became_peered":"2024-03-27T06:41:29.408143+0000","last_unstale":"2024-03-27T06:43:34.044192+0000","last_undegraded":"2024-03-27T06:43:34.044192+0000","last_fullsized":"2024-03-27T06:43:34.044192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:01:22.486951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044362+0000","last_change":"2024-03-27T06:41:29.405080+0000","last_active":"2024-03-27T06:43:34.044362+0000","last_peered":"2024-03-27T06:43:34.044362+0000","last_clean":"2024-03-27T06:43:34.044362+0000","last_became_active":"2024-03-27T06:41:29.404743+0000","last_became_peered":"2024-03-27T06:41:29.404743+0000","last_unstale":"2024-03-27T06:43:34.044362+0000","last_undegraded":"2024-03-27T06:43:34.044362+0000","last_fullsized":"2024-03-27T06:43:34.044362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:24:30.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044526+0000","last_change":"2024-03-27T06:41:29.409346+0000","last_active":"2024-03-27T06:43:34.044526+0000","last_peered":"2024-03-27T06:43:34.044526+0000","last_clean":"2024-03-27T06:43:34.044526+0000","last_became_active":"2024-03-27T06:41:29.408837+0000","last_became_peered":"2024-03-27T06:41:29.408837+0000","last_unstale":"2024-03-27T06:43:34.044526+0000","last_undegraded":"2024-03-27T06:43:34.044526+0000","last_fullsized":"2024-03-27T06:43:34.044526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:11:12.274869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052302+0000","last_change":"2024-03-27T06:41:29.412115+0000","last_active":"2024-03-27T06:43:34.052302+0000","last_peered":"2024-03-27T06:43:34.052302+0000","last_clean":"2024-03-27T06:43:34.052302+0000","last_became_active":"2024-03-27T06:41:29.411702+0000","last_became_peered":"2024-03-27T06:41:29.411702+0000","last_unstale":"2024-03-27T06:43:34.052302+0000","last_undegraded":"2024-03-27T06:43:34.052302+0000","last_fullsized":"2024-03-27T06:43:34.052302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:04:42.347229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890812+0000","last_change":"2024-03-27T06:41:29.409888+0000","last_active":"2024-03-27T06:43:33.890812+0000","last_peered":"2024-03-27T06:43:33.890812+0000","last_clean":"2024-03-27T06:43:33.890812+0000","last_became_active":"2024-03-27T06:41:29.409558+0000","last_became_peered":"2024-03-27T06:41:29.409558+0000","last_unstale":"2024-03-27T06:43:33.890812+0000","last_undegraded":"2024-03-27T06:43:33.890812+0000","last_fullsized":"2024-03-27T06:43:33.890812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:38:51.452793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902755+0000","last_change":"2024-03-27T06:41:29.404088+0000","last_active":"2024-03-27T06:43:33.902755+0000","last_peered":"2024-03-27T06:43:33.902755+0000","last_clean":"2024-03-27T06:43:33.902755+0000","last_became_active":"2024-03-27T06:41:29.403698+0000","last_became_peered":"2024-03-27T06:41:29.403698+0000","last_unstale":"2024-03-27T06:43:33.902755+0000","last_undegraded":"2024-03-27T06:43:33.902755+0000","last_fullsized":"2024-03-27T06:43:33.902755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:14:31.296523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788203+0000","last_change":"2024-03-27T06:41:29.413157+0000","last_active":"2024-03-27T06:43:33.788203+0000","last_peered":"2024-03-27T06:43:33.788203+0000","last_clean":"2024-03-27T06:43:33.788203+0000","last_became_active":"2024-03-27T06:41:29.412669+0000","last_became_peered":"2024-03-27T06:41:29.412669+0000","last_unstale":"2024-03-27T06:43:33.788203+0000","last_undegraded":"2024-03-27T06:43:33.788203+0000","last_fullsized":"2024-03-27T06:43:33.788203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:28:20.823770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890507+0000","last_change":"2024-03-27T06:41:29.417207+0000","last_active":"2024-03-27T06:43:33.890507+0000","last_peered":"2024-03-27T06:43:33.890507+0000","last_clean":"2024-03-27T06:43:33.890507+0000","last_became_active":"2024-03-27T06:41:29.415556+0000","last_became_peered":"2024-03-27T06:41:29.415556+0000","last_unstale":"2024-03-27T06:43:33.890507+0000","last_undegraded":"2024-03-27T06:43:33.890507+0000","last_fullsized":"2024-03-27T06:43:33.890507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:26:04.040296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.053665+0000","last_change":"2024-03-27T06:41:29.400032+0000","last_active":"2024-03-27T06:43:34.053665+0000","last_peered":"2024-03-27T06:43:34.053665+0000","last_clean":"2024-03-27T06:43:34.053665+0000","last_became_active":"2024-03-27T06:41:29.399274+0000","last_became_peered":"2024-03-27T06:41:29.399274+0000","last_unstale":"2024-03-27T06:43:34.053665+0000","last_undegraded":"2024-03-27T06:43:34.053665+0000","last_fullsized":"2024-03-27T06:43:34.053665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:22:55.306309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.903052+0000","last_change":"2024-03-27T06:41:29.412471+0000","last_active":"2024-03-27T06:43:33.903052+0000","last_peered":"2024-03-27T06:43:33.903052+0000","last_clean":"2024-03-27T06:43:33.903052+0000","last_became_active":"2024-03-27T06:41:29.412134+0000","last_became_peered":"2024-03-27T06:41:29.412134+0000","last_unstale":"2024-03-27T06:43:33.903052+0000","last_undegraded":"2024-03-27T06:43:33.903052+0000","last_fullsized":"2024-03-27T06:43:33.903052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:35:57.146944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788956+0000","last_change":"2024-03-27T06:41:29.407915+0000","last_active":"2024-03-27T06:43:33.788956+0000","last_peered":"2024-03-27T06:43:33.788956+0000","last_clean":"2024-03-27T06:43:33.788956+0000","last_became_active":"2024-03-27T06:41:29.407563+0000","last_became_peered":"2024-03-27T06:41:29.407563+0000","last_unstale":"2024-03-27T06:43:33.788956+0000","last_undegraded":"2024-03-27T06:43:33.788956+0000","last_fullsized":"2024-03-27T06:43:33.788956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:09:05.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052028+0000","last_change":"2024-03-27T06:41:29.413374+0000","last_active":"2024-03-27T06:43:34.052028+0000","last_peered":"2024-03-27T06:43:34.052028+0000","last_clean":"2024-03-27T06:43:34.052028+0000","last_became_active":"2024-03-27T06:41:29.413010+0000","last_became_peered":"2024-03-27T06:41:29.413010+0000","last_unstale":"2024-03-27T06:43:34.052028+0000","last_undegraded":"2024-03-27T06:43:34.052028+0000","last_fullsized":"2024-03-27T06:43:34.052028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:48:52.329611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050925+0000","last_change":"2024-03-27T06:41:29.408640+0000","last_active":"2024-03-27T06:43:34.050925+0000","last_peered":"2024-03-27T06:43:34.050925+0000","last_clean":"2024-03-27T06:43:34.050925+0000","last_became_active":"2024-03-27T06:41:29.408268+0000","last_became_peered":"2024-03-27T06:41:29.408268+0000","last_unstale":"2024-03-27T06:43:34.050925+0000","last_undegraded":"2024-03-27T06:43:34.050925+0000","last_fullsized":"2024-03-27T06:43:34.050925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:51:59.803596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043668+0000","last_change":"2024-03-27T06:41:29.420436+0000","last_active":"2024-03-27T06:43:34.043668+0000","last_peered":"2024-03-27T06:43:34.043668+0000","last_clean":"2024-03-27T06:43:34.043668+0000","last_became_active":"2024-03-27T06:41:29.419426+0000","last_became_peered":"2024-03-27T06:41:29.419426+0000","last_unstale":"2024-03-27T06:43:34.043668+0000","last_undegraded":"2024-03-27T06:43:34.043668+0000","last_fullsized":"2024-03-27T06:43:34.043668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:38:15.757620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043640+0000","last_change":"2024-03-27T06:41:29.415673+0000","last_active":"2024-03-27T06:43:34.043640+0000","last_peered":"2024-03-27T06:43:34.043640+0000","last_clean":"2024-03-27T06:43:34.043640+0000","last_became_active":"2024-03-27T06:41:29.414741+0000","last_became_peered":"2024-03-27T06:41:29.414741+0000","last_unstale":"2024-03-27T06:43:34.043640+0000","last_undegraded":"2024-03-27T06:43:34.043640+0000","last_fullsized":"2024-03-27T06:43:34.043640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:38:43.183517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051498+0000","last_change":"2024-03-27T06:41:29.408609+0000","last_active":"2024-03-27T06:43:34.051498+0000","last_peered":"2024-03-27T06:43:34.051498+0000","last_clean":"2024-03-27T06:43:34.051498+0000","last_became_active":"2024-03-27T06:41:29.408252+0000","last_became_peered":"2024-03-27T06:41:29.408252+0000","last_unstale":"2024-03-27T06:43:34.051498+0000","last_undegraded":"2024-03-27T06:43:34.051498+0000","last_fullsized":"2024-03-27T06:43:34.051498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:14:13.609113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044330+0000","last_change":"2024-03-27T06:41:29.409221+0000","last_active":"2024-03-27T06:43:34.044330+0000","last_peered":"2024-03-27T06:43:34.044330+0000","last_clean":"2024-03-27T06:43:34.044330+0000","last_became_active":"2024-03-27T06:41:29.408598+0000","last_became_peered":"2024-03-27T06:41:29.408598+0000","last_unstale":"2024-03-27T06:43:34.044330+0000","last_undegraded":"2024-03-27T06:43:34.044330+0000","last_fullsized":"2024-03-27T06:43:34.044330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:42:16.251149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.786349+0000","last_change":"2024-03-27T06:41:29.406143+0000","last_active":"2024-03-27T06:43:33.786349+0000","last_peered":"2024-03-27T06:43:33.786349+0000","last_clean":"2024-03-27T06:43:33.786349+0000","last_became_active":"2024-03-27T06:41:29.405754+0000","last_became_peered":"2024-03-27T06:41:29.405754+0000","last_unstale":"2024-03-27T06:43:33.786349+0000","last_undegraded":"2024-03-27T06:43:33.786349+0000","last_fullsized":"2024-03-27T06:43:33.786349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:42:47.787099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788822+0000","last_change":"2024-03-27T06:41:29.402471+0000","last_active":"2024-03-27T06:43:33.788822+0000","last_peered":"2024-03-27T06:43:33.788822+0000","last_clean":"2024-03-27T06:43:33.788822+0000","last_became_active":"2024-03-27T06:41:29.402113+0000","last_became_peered":"2024-03-27T06:41:29.402113+0000","last_unstale":"2024-03-27T06:43:33.788822+0000","last_undegraded":"2024-03-27T06:43:33.788822+0000","last_fullsized":"2024-03-27T06:43:33.788822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:34:16.408058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'14","reported_seq":157,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607533+0000","last_change":"2024-03-27T06:41:28.397815+0000","last_active":"2024-03-27T06:43:34.607533+0000","last_peered":"2024-03-27T06:43:34.607533+0000","last_clean":"2024-03-27T06:43:34.607533+0000","last_became_active":"2024-03-27T06:41:28.397503+0000","last_became_peered":"2024-03-27T06:41:28.397503+0000","last_unstale":"2024-03-27T06:43:34.607533+0000","last_undegraded":"2024-03-27T06:43:34.607533+0000","last_fullsized":"2024-03-27T06:43:34.607533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:15:53.751809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890734+0000","last_change":"2024-03-27T06:41:29.419528+0000","last_active":"2024-03-27T06:43:33.890734+0000","last_peered":"2024-03-27T06:43:33.890734+0000","last_clean":"2024-03-27T06:43:33.890734+0000","last_became_active":"2024-03-27T06:41:29.417091+0000","last_became_peered":"2024-03-27T06:41:29.417091+0000","last_unstale":"2024-03-27T06:43:33.890734+0000","last_undegraded":"2024-03-27T06:43:33.890734+0000","last_fullsized":"2024-03-27T06:43:33.890734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:13:14.629211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891107+0000","last_change":"2024-03-27T06:41:28.400468+0000","last_active":"2024-03-27T06:43:33.891107+0000","last_peered":"2024-03-27T06:43:33.891107+0000","last_clean":"2024-03-27T06:43:33.891107+0000","last_became_active":"2024-03-27T06:41:28.400190+0000","last_became_peered":"2024-03-27T06:41:28.400190+0000","last_unstale":"2024-03-27T06:43:33.891107+0000","last_undegraded":"2024-03-27T06:43:33.891107+0000","last_fullsized":"2024-03-27T06:43:33.891107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:24:17.059874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.045055+0000","last_change":"2024-03-27T06:41:29.409364+0000","last_active":"2024-03-27T06:43:34.045055+0000","last_peered":"2024-03-27T06:43:34.045055+0000","last_clean":"2024-03-27T06:43:34.045055+0000","last_became_active":"2024-03-27T06:41:29.408946+0000","last_became_peered":"2024-03-27T06:41:29.408946+0000","last_unstale":"2024-03-27T06:43:34.045055+0000","last_undegraded":"2024-03-27T06:43:34.045055+0000","last_fullsized":"2024-03-27T06:43:34.045055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:46:17.132071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890918+0000","last_change":"2024-03-27T06:41:28.401327+0000","last_active":"2024-03-27T06:43:33.890918+0000","last_peered":"2024-03-27T06:43:33.890918+0000","last_clean":"2024-03-27T06:43:33.890918+0000","last_became_active":"2024-03-27T06:41:28.400524+0000","last_became_peered":"2024-03-27T06:41:28.400524+0000","last_unstale":"2024-03-27T06:43:33.890918+0000","last_undegraded":"2024-03-27T06:43:33.890918+0000","last_fullsized":"2024-03-27T06:43:33.890918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:11:52.608690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051934+0000","last_change":"2024-03-27T06:41:29.403042+0000","last_active":"2024-03-27T06:43:34.051934+0000","last_peered":"2024-03-27T06:43:34.051934+0000","last_clean":"2024-03-27T06:43:34.051934+0000","last_became_active":"2024-03-27T06:41:29.402516+0000","last_became_peered":"2024-03-27T06:41:29.402516+0000","last_unstale":"2024-03-27T06:43:34.051934+0000","last_undegraded":"2024-03-27T06:43:34.051934+0000","last_fullsized":"2024-03-27T06:43:34.051934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:20:55.777498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891130+0000","last_change":"2024-03-27T06:41:28.395155+0000","last_active":"2024-03-27T06:43:33.891130+0000","last_peered":"2024-03-27T06:43:33.891130+0000","last_clean":"2024-03-27T06:43:33.891130+0000","last_became_active":"2024-03-27T06:41:28.394760+0000","last_became_peered":"2024-03-27T06:41:28.394760+0000","last_unstale":"2024-03-27T06:43:33.891130+0000","last_undegraded":"2024-03-27T06:43:33.891130+0000","last_fullsized":"2024-03-27T06:43:33.891130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:15:28.182581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899706+0000","last_change":"2024-03-27T06:41:29.409747+0000","last_active":"2024-03-27T06:43:33.899706+0000","last_peered":"2024-03-27T06:43:33.899706+0000","last_clean":"2024-03-27T06:43:33.899706+0000","last_became_active":"2024-03-27T06:41:29.409425+0000","last_became_peered":"2024-03-27T06:41:29.409425+0000","last_unstale":"2024-03-27T06:43:33.899706+0000","last_undegraded":"2024-03-27T06:43:33.899706+0000","last_fullsized":"2024-03-27T06:43:33.899706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:08:36.181279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.605066+0000","last_change":"2024-03-27T06:41:28.394726+0000","last_active":"2024-03-27T06:43:34.605066+0000","last_peered":"2024-03-27T06:43:34.605066+0000","last_clean":"2024-03-27T06:43:34.605066+0000","last_became_active":"2024-03-27T06:41:28.394378+0000","last_became_peered":"2024-03-27T06:41:28.394378+0000","last_unstale":"2024-03-27T06:43:34.605066+0000","last_undegraded":"2024-03-27T06:43:34.605066+0000","last_fullsized":"2024-03-27T06:43:34.605066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:30:22.194211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890573+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:43:33.890573+0000","last_peered":"2024-03-27T06:43:33.890573+0000","last_clean":"2024-03-27T06:43:33.890573+0000","last_became_active":"2024-03-27T06:41:29.408826+0000","last_became_peered":"2024-03-27T06:41:29.408826+0000","last_unstale":"2024-03-27T06:43:33.890573+0000","last_undegraded":"2024-03-27T06:43:33.890573+0000","last_fullsized":"2024-03-27T06:43:33.890573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:50:53.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902801+0000","last_change":"2024-03-27T06:41:28.393655+0000","last_active":"2024-03-27T06:43:33.902801+0000","last_peered":"2024-03-27T06:43:33.902801+0000","last_clean":"2024-03-27T06:43:33.902801+0000","last_became_active":"2024-03-27T06:41:28.393245+0000","last_became_peered":"2024-03-27T06:41:28.393245+0000","last_unstale":"2024-03-27T06:43:33.902801+0000","last_undegraded":"2024-03-27T06:43:33.902801+0000","last_fullsized":"2024-03-27T06:43:33.902801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:58:21.169444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899501+0000","last_change":"2024-03-27T06:41:29.417730+0000","last_active":"2024-03-27T06:43:33.899501+0000","last_peered":"2024-03-27T06:43:33.899501+0000","last_clean":"2024-03-27T06:43:33.899501+0000","last_became_active":"2024-03-27T06:41:29.417319+0000","last_became_peered":"2024-03-27T06:41:29.417319+0000","last_unstale":"2024-03-27T06:43:33.899501+0000","last_undegraded":"2024-03-27T06:43:33.899501+0000","last_fullsized":"2024-03-27T06:43:33.899501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:25.224220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050475+0000","last_change":"2024-03-27T06:41:28.402259+0000","last_active":"2024-03-27T06:43:34.050475+0000","last_peered":"2024-03-27T06:43:34.050475+0000","last_clean":"2024-03-27T06:43:34.050475+0000","last_became_active":"2024-03-27T06:41:28.401922+0000","last_became_peered":"2024-03-27T06:41:28.401922+0000","last_unstale":"2024-03-27T06:43:34.050475+0000","last_undegraded":"2024-03-27T06:43:34.050475+0000","last_fullsized":"2024-03-27T06:43:34.050475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:57:33.245183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890562+0000","last_change":"2024-03-27T06:41:29.409034+0000","last_active":"2024-03-27T06:43:33.890562+0000","last_peered":"2024-03-27T06:43:33.890562+0000","last_clean":"2024-03-27T06:43:33.890562+0000","last_became_active":"2024-03-27T06:41:29.408666+0000","last_became_peered":"2024-03-27T06:41:29.408666+0000","last_unstale":"2024-03-27T06:43:33.890562+0000","last_undegraded":"2024-03-27T06:43:33.890562+0000","last_fullsized":"2024-03-27T06:43:33.890562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:09:55.532059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899143+0000","last_change":"2024-03-27T06:41:28.393802+0000","last_active":"2024-03-27T06:43:33.899143+0000","last_peered":"2024-03-27T06:43:33.899143+0000","last_clean":"2024-03-27T06:43:33.899143+0000","last_became_active":"2024-03-27T06:41:28.393503+0000","last_became_peered":"2024-03-27T06:41:28.393503+0000","last_unstale":"2024-03-27T06:43:33.899143+0000","last_undegraded":"2024-03-27T06:43:33.899143+0000","last_fullsized":"2024-03-27T06:43:33.899143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:57:17.765709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.786232+0000","last_change":"2024-03-27T06:41:29.416557+0000","last_active":"2024-03-27T06:43:33.786232+0000","last_peered":"2024-03-27T06:43:33.786232+0000","last_clean":"2024-03-27T06:43:33.786232+0000","last_became_active":"2024-03-27T06:41:29.416223+0000","last_became_peered":"2024-03-27T06:41:29.416223+0000","last_unstale":"2024-03-27T06:43:33.786232+0000","last_undegraded":"2024-03-27T06:43:33.786232+0000","last_fullsized":"2024-03-27T06:43:33.786232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:54:20.929325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":268,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.604510+0000","last_change":"2024-03-27T06:41:28.396014+0000","last_active":"2024-03-27T06:43:34.604510+0000","last_peered":"2024-03-27T06:43:34.604510+0000","last_clean":"2024-03-27T06:43:34.604510+0000","last_became_active":"2024-03-27T06:41:28.395699+0000","last_became_peered":"2024-03-27T06:41:28.395699+0000","last_unstale":"2024-03-27T06:43:34.604510+0000","last_undegraded":"2024-03-27T06:43:34.604510+0000","last_fullsized":"2024-03-27T06:43:34.604510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:00:59.679843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.053523+0000","last_change":"2024-03-27T06:41:29.401668+0000","last_active":"2024-03-27T06:43:34.053523+0000","last_peered":"2024-03-27T06:43:34.053523+0000","last_clean":"2024-03-27T06:43:34.053523+0000","last_became_active":"2024-03-27T06:41:29.401328+0000","last_became_peered":"2024-03-27T06:41:29.401328+0000","last_unstale":"2024-03-27T06:43:34.053523+0000","last_undegraded":"2024-03-27T06:43:34.053523+0000","last_fullsized":"2024-03-27T06:43:34.053523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:32:54.273194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789142+0000","last_change":"2024-03-27T06:42:21.011596+0000","last_active":"2024-03-27T06:43:33.789142+0000","last_peered":"2024-03-27T06:43:33.789142+0000","last_clean":"2024-03-27T06:43:33.789142+0000","last_became_active":"2024-03-27T06:42:21.005035+0000","last_became_peered":"2024-03-27T06:42:21.005035+0000","last_unstale":"2024-03-27T06:43:33.789142+0000","last_undegraded":"2024-03-27T06:43:33.789142+0000","last_fullsized":"2024-03-27T06:43:33.789142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:51:35.652266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044846+0000","last_change":"2024-03-27T06:41:29.413919+0000","last_active":"2024-03-27T06:43:34.044846+0000","last_peered":"2024-03-27T06:43:34.044846+0000","last_clean":"2024-03-27T06:43:34.044846+0000","last_became_active":"2024-03-27T06:41:29.413566+0000","last_became_peered":"2024-03-27T06:41:29.413566+0000","last_unstale":"2024-03-27T06:43:34.044846+0000","last_undegraded":"2024-03-27T06:43:34.044846+0000","last_fullsized":"2024-03-27T06:43:34.044846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:45:03.705372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899758+0000","last_change":"2024-03-27T06:41:28.391661+0000","last_active":"2024-03-27T06:43:33.899758+0000","last_peered":"2024-03-27T06:43:33.899758+0000","last_clean":"2024-03-27T06:43:33.899758+0000","last_became_active":"2024-03-27T06:41:28.391230+0000","last_became_peered":"2024-03-27T06:41:28.391230+0000","last_unstale":"2024-03-27T06:43:33.899758+0000","last_undegraded":"2024-03-27T06:43:33.899758+0000","last_fullsized":"2024-03-27T06:43:33.899758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:41:22.789267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052236+0000","last_change":"2024-03-27T06:41:29.405727+0000","last_active":"2024-03-27T06:43:34.052236+0000","last_peered":"2024-03-27T06:43:34.052236+0000","last_clean":"2024-03-27T06:43:34.052236+0000","last_became_active":"2024-03-27T06:41:29.405413+0000","last_became_peered":"2024-03-27T06:41:29.405413+0000","last_unstale":"2024-03-27T06:43:34.052236+0000","last_undegraded":"2024-03-27T06:43:34.052236+0000","last_fullsized":"2024-03-27T06:43:34.052236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:35:21.575729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":308,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.605048+0000","last_change":"2024-03-27T06:41:28.391155+0000","last_active":"2024-03-27T06:43:34.605048+0000","last_peered":"2024-03-27T06:43:34.605048+0000","last_clean":"2024-03-27T06:43:34.605048+0000","last_became_active":"2024-03-27T06:41:28.390828+0000","last_became_peered":"2024-03-27T06:41:28.390828+0000","last_unstale":"2024-03-27T06:43:34.605048+0000","last_undegraded":"2024-03-27T06:43:34.605048+0000","last_fullsized":"2024-03-27T06:43:34.605048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:40.192560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66735,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":190,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890788+0000","last_change":"2024-03-27T06:41:29.413283+0000","last_active":"2024-03-27T06:43:33.890788+0000","last_peered":"2024-03-27T06:43:33.890788+0000","last_clean":"2024-03-27T06:43:33.890788+0000","last_became_active":"2024-03-27T06:41:29.412247+0000","last_became_peered":"2024-03-27T06:41:29.412247+0000","last_unstale":"2024-03-27T06:43:33.890788+0000","last_undegraded":"2024-03-27T06:43:33.890788+0000","last_fullsized":"2024-03-27T06:43:33.890788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:50:15.336003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788566+0000","last_change":"2024-03-27T06:41:28.395089+0000","last_active":"2024-03-27T06:43:33.788566+0000","last_peered":"2024-03-27T06:43:33.788566+0000","last_clean":"2024-03-27T06:43:33.788566+0000","last_became_active":"2024-03-27T06:41:28.394145+0000","last_became_peered":"2024-03-27T06:41:28.394145+0000","last_unstale":"2024-03-27T06:43:33.788566+0000","last_undegraded":"2024-03-27T06:43:33.788566+0000","last_fullsized":"2024-03-27T06:43:33.788566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:40.858404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050686+0000","last_change":"2024-03-27T06:41:29.416955+0000","last_active":"2024-03-27T06:43:34.050686+0000","last_peered":"2024-03-27T06:43:34.050686+0000","last_clean":"2024-03-27T06:43:34.050686+0000","last_became_active":"2024-03-27T06:41:29.416616+0000","last_became_peered":"2024-03-27T06:41:29.416616+0000","last_unstale":"2024-03-27T06:43:34.050686+0000","last_undegraded":"2024-03-27T06:43:34.050686+0000","last_fullsized":"2024-03-27T06:43:34.050686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:38:18.791602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899329+0000","last_change":"2024-03-27T06:41:28.393876+0000","last_active":"2024-03-27T06:43:33.899329+0000","last_peered":"2024-03-27T06:43:33.899329+0000","last_clean":"2024-03-27T06:43:33.899329+0000","last_became_active":"2024-03-27T06:41:28.393557+0000","last_became_peered":"2024-03-27T06:41:28.393557+0000","last_unstale":"2024-03-27T06:43:33.899329+0000","last_undegraded":"2024-03-27T06:43:33.899329+0000","last_fullsized":"2024-03-27T06:43:33.899329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:25:30.831039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788517+0000","last_change":"2024-03-27T06:42:20.660026+0000","last_active":"2024-03-27T06:43:33.788517+0000","last_peered":"2024-03-27T06:43:33.788517+0000","last_clean":"2024-03-27T06:43:33.788517+0000","last_became_active":"2024-03-27T06:42:20.659628+0000","last_became_peered":"2024-03-27T06:42:20.659628+0000","last_unstale":"2024-03-27T06:43:33.788517+0000","last_undegraded":"2024-03-27T06:43:33.788517+0000","last_fullsized":"2024-03-27T06:43:33.788517+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:18:59.396472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607543+0000","last_change":"2024-03-27T06:41:28.393849+0000","last_active":"2024-03-27T06:43:34.607543+0000","last_peered":"2024-03-27T06:43:34.607543+0000","last_clean":"2024-03-27T06:43:34.607543+0000","last_became_active":"2024-03-27T06:41:28.393529+0000","last_became_peered":"2024-03-27T06:41:28.393529+0000","last_unstale":"2024-03-27T06:43:34.607543+0000","last_undegraded":"2024-03-27T06:43:34.607543+0000","last_fullsized":"2024-03-27T06:43:34.607543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:24:23.251723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.787942+0000","last_change":"2024-03-27T06:41:29.411273+0000","last_active":"2024-03-27T06:43:33.787942+0000","last_peered":"2024-03-27T06:43:33.787942+0000","last_clean":"2024-03-27T06:43:33.787942+0000","last_became_active":"2024-03-27T06:41:29.410936+0000","last_became_peered":"2024-03-27T06:41:29.410936+0000","last_unstale":"2024-03-27T06:43:33.787942+0000","last_undegraded":"2024-03-27T06:43:33.787942+0000","last_fullsized":"2024-03-27T06:43:33.787942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:50:41.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043527+0000","last_change":"2024-03-27T06:42:20.673735+0000","last_active":"2024-03-27T06:43:34.043527+0000","last_peered":"2024-03-27T06:43:34.043527+0000","last_clean":"2024-03-27T06:43:34.043527+0000","last_became_active":"2024-03-27T06:42:20.655144+0000","last_became_peered":"2024-03-27T06:42:20.655144+0000","last_unstale":"2024-03-27T06:43:34.043527+0000","last_undegraded":"2024-03-27T06:43:34.043527+0000","last_fullsized":"2024-03-27T06:43:34.043527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:05:39.059633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899135+0000","last_change":"2024-03-27T06:41:29.409174+0000","last_active":"2024-03-27T06:43:33.899135+0000","last_peered":"2024-03-27T06:43:33.899135+0000","last_clean":"2024-03-27T06:43:33.899135+0000","last_became_active":"2024-03-27T06:41:29.408815+0000","last_became_peered":"2024-03-27T06:41:29.408815+0000","last_unstale":"2024-03-27T06:43:33.899135+0000","last_undegraded":"2024-03-27T06:43:33.899135+0000","last_fullsized":"2024-03-27T06:43:33.899135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:45.547747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891036+0000","last_change":"2024-03-27T06:41:28.398249+0000","last_active":"2024-03-27T06:43:33.891036+0000","last_peered":"2024-03-27T06:43:33.891036+0000","last_clean":"2024-03-27T06:43:33.891036+0000","last_became_active":"2024-03-27T06:41:28.397176+0000","last_became_peered":"2024-03-27T06:41:28.397176+0000","last_unstale":"2024-03-27T06:43:33.891036+0000","last_undegraded":"2024-03-27T06:43:33.891036+0000","last_fullsized":"2024-03-27T06:43:33.891036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:19:32.614558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044019+0000","last_change":"2024-03-27T06:41:29.412928+0000","last_active":"2024-03-27T06:43:34.044019+0000","last_peered":"2024-03-27T06:43:34.044019+0000","last_clean":"2024-03-27T06:43:34.044019+0000","last_became_active":"2024-03-27T06:41:29.412576+0000","last_became_peered":"2024-03-27T06:41:29.412576+0000","last_unstale":"2024-03-27T06:43:34.044019+0000","last_undegraded":"2024-03-27T06:43:34.044019+0000","last_fullsized":"2024-03-27T06:43:34.044019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:01:22.389424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.786409+0000","last_change":"2024-03-27T06:41:28.397135+0000","last_active":"2024-03-27T06:43:33.786409+0000","last_peered":"2024-03-27T06:43:33.786409+0000","last_clean":"2024-03-27T06:43:33.786409+0000","last_became_active":"2024-03-27T06:41:28.396483+0000","last_became_peered":"2024-03-27T06:41:28.396483+0000","last_unstale":"2024-03-27T06:43:33.786409+0000","last_undegraded":"2024-03-27T06:43:33.786409+0000","last_fullsized":"2024-03-27T06:43:33.786409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:05:02.038024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902974+0000","last_change":"2024-03-27T06:41:29.418629+0000","last_active":"2024-03-27T06:43:33.902974+0000","last_peered":"2024-03-27T06:43:33.902974+0000","last_clean":"2024-03-27T06:43:33.902974+0000","last_became_active":"2024-03-27T06:41:29.418278+0000","last_became_peered":"2024-03-27T06:41:29.418278+0000","last_unstale":"2024-03-27T06:43:33.902974+0000","last_undegraded":"2024-03-27T06:43:33.902974+0000","last_fullsized":"2024-03-27T06:43:33.902974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:33:54.461349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.608536+0000","last_change":"2024-03-27T06:41:28.400611+0000","last_active":"2024-03-27T06:43:34.608536+0000","last_peered":"2024-03-27T06:43:34.608536+0000","last_clean":"2024-03-27T06:43:34.608536+0000","last_became_active":"2024-03-27T06:41:28.400211+0000","last_became_peered":"2024-03-27T06:41:28.400211+0000","last_unstale":"2024-03-27T06:43:34.608536+0000","last_undegraded":"2024-03-27T06:43:34.608536+0000","last_fullsized":"2024-03-27T06:43:34.608536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:46:35.491071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.902979+0000","last_change":"2024-03-27T06:41:29.413246+0000","last_active":"2024-03-27T06:43:33.902979+0000","last_peered":"2024-03-27T06:43:33.902979+0000","last_clean":"2024-03-27T06:43:33.902979+0000","last_became_active":"2024-03-27T06:41:29.412946+0000","last_became_peered":"2024-03-27T06:41:29.412946+0000","last_unstale":"2024-03-27T06:43:33.902979+0000","last_undegraded":"2024-03-27T06:43:33.902979+0000","last_fullsized":"2024-03-27T06:43:33.902979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:41:51.441233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899393+0000","last_change":"2024-03-27T06:41:29.402160+0000","last_active":"2024-03-27T06:43:33.899393+0000","last_peered":"2024-03-27T06:43:33.899393+0000","last_clean":"2024-03-27T06:43:33.899393+0000","last_became_active":"2024-03-27T06:41:29.401789+0000","last_became_peered":"2024-03-27T06:41:29.401789+0000","last_unstale":"2024-03-27T06:43:33.899393+0000","last_undegraded":"2024-03-27T06:43:33.899393+0000","last_fullsized":"2024-03-27T06:43:33.899393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:31:27.821966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607060+0000","last_change":"2024-03-27T06:41:28.395978+0000","last_active":"2024-03-27T06:43:34.607060+0000","last_peered":"2024-03-27T06:43:34.607060+0000","last_clean":"2024-03-27T06:43:34.607060+0000","last_became_active":"2024-03-27T06:41:28.395650+0000","last_became_peered":"2024-03-27T06:41:28.395650+0000","last_unstale":"2024-03-27T06:43:34.607060+0000","last_undegraded":"2024-03-27T06:43:34.607060+0000","last_fullsized":"2024-03-27T06:43:34.607060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:01:15.841344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890934+0000","last_change":"2024-03-27T06:41:29.423303+0000","last_active":"2024-03-27T06:43:33.890934+0000","last_peered":"2024-03-27T06:43:33.890934+0000","last_clean":"2024-03-27T06:43:33.890934+0000","last_became_active":"2024-03-27T06:41:29.422977+0000","last_became_peered":"2024-03-27T06:41:29.422977+0000","last_unstale":"2024-03-27T06:43:33.890934+0000","last_undegraded":"2024-03-27T06:43:33.890934+0000","last_fullsized":"2024-03-27T06:43:33.890934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:14:01.877801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051856+0000","last_change":"2024-03-27T06:41:28.395141+0000","last_active":"2024-03-27T06:43:34.051856+0000","last_peered":"2024-03-27T06:43:34.051856+0000","last_clean":"2024-03-27T06:43:34.051856+0000","last_became_active":"2024-03-27T06:41:28.394826+0000","last_became_peered":"2024-03-27T06:41:28.394826+0000","last_unstale":"2024-03-27T06:43:34.051856+0000","last_undegraded":"2024-03-27T06:43:34.051856+0000","last_fullsized":"2024-03-27T06:43:34.051856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:29:49.997090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788039+0000","last_change":"2024-03-27T06:40:16.460939+0000","last_active":"2024-03-27T06:43:33.788039+0000","last_peered":"2024-03-27T06:43:33.788039+0000","last_clean":"2024-03-27T06:43:33.788039+0000","last_became_active":"2024-03-27T06:40:16.460453+0000","last_became_peered":"2024-03-27T06:40:16.460453+0000","last_unstale":"2024-03-27T06:43:33.788039+0000","last_undegraded":"2024-03-27T06:43:33.788039+0000","last_fullsized":"2024-03-27T06:43:33.788039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:36:48.325777+0000","last_clean_scrub_stamp":"2024-03-27T06:36:48.325777+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:09:28.241688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050380+0000","last_change":"2024-03-27T06:42:20.658690+0000","last_active":"2024-03-27T06:43:34.050380+0000","last_peered":"2024-03-27T06:43:34.050380+0000","last_clean":"2024-03-27T06:43:34.050380+0000","last_became_active":"2024-03-27T06:42:20.658348+0000","last_became_peered":"2024-03-27T06:42:20.658348+0000","last_unstale":"2024-03-27T06:43:34.050380+0000","last_undegraded":"2024-03-27T06:43:34.050380+0000","last_fullsized":"2024-03-27T06:43:34.050380+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:12:29.985173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788194+0000","last_change":"2024-03-27T06:41:28.399205+0000","last_active":"2024-03-27T06:43:33.788194+0000","last_peered":"2024-03-27T06:43:33.788194+0000","last_clean":"2024-03-27T06:43:33.788194+0000","last_became_active":"2024-03-27T06:41:28.398889+0000","last_became_peered":"2024-03-27T06:41:28.398889+0000","last_unstale":"2024-03-27T06:43:33.788194+0000","last_undegraded":"2024-03-27T06:43:33.788194+0000","last_fullsized":"2024-03-27T06:43:33.788194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:28:08.769187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043816+0000","last_change":"2024-03-27T06:42:20.663370+0000","last_active":"2024-03-27T06:43:34.043816+0000","last_peered":"2024-03-27T06:43:34.043816+0000","last_clean":"2024-03-27T06:43:34.043816+0000","last_became_active":"2024-03-27T06:42:20.654243+0000","last_became_peered":"2024-03-27T06:42:20.654243+0000","last_unstale":"2024-03-27T06:43:34.043816+0000","last_undegraded":"2024-03-27T06:43:34.043816+0000","last_fullsized":"2024-03-27T06:43:34.043816+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:43:46.566952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051012+0000","last_change":"2024-03-27T06:41:29.395466+0000","last_active":"2024-03-27T06:43:34.051012+0000","last_peered":"2024-03-27T06:43:34.051012+0000","last_clean":"2024-03-27T06:43:34.051012+0000","last_became_active":"2024-03-27T06:41:29.395094+0000","last_became_peered":"2024-03-27T06:41:29.395094+0000","last_unstale":"2024-03-27T06:43:34.051012+0000","last_undegraded":"2024-03-27T06:43:34.051012+0000","last_fullsized":"2024-03-27T06:43:34.051012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:29:01.746809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'48","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.011094+0000","last_change":"2024-03-27T06:41:28.400420+0000","last_active":"2024-03-27T06:43:34.011094+0000","last_peered":"2024-03-27T06:43:34.011094+0000","last_clean":"2024-03-27T06:43:34.011094+0000","last_became_active":"2024-03-27T06:41:28.400115+0000","last_became_peered":"2024-03-27T06:41:28.400115+0000","last_unstale":"2024-03-27T06:43:34.011094+0000","last_undegraded":"2024-03-27T06:43:34.011094+0000","last_fullsized":"2024-03-27T06:43:34.011094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:37:40.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891028+0000","last_change":"2024-03-27T06:41:29.411428+0000","last_active":"2024-03-27T06:43:33.891028+0000","last_peered":"2024-03-27T06:43:33.891028+0000","last_clean":"2024-03-27T06:43:33.891028+0000","last_became_active":"2024-03-27T06:41:29.411075+0000","last_became_peered":"2024-03-27T06:41:29.411075+0000","last_unstale":"2024-03-27T06:43:33.891028+0000","last_undegraded":"2024-03-27T06:43:33.891028+0000","last_fullsized":"2024-03-27T06:43:33.891028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:18:55.764459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.052106+0000","last_change":"2024-03-27T06:41:28.399799+0000","last_active":"2024-03-27T06:43:34.052106+0000","last_peered":"2024-03-27T06:43:34.052106+0000","last_clean":"2024-03-27T06:43:34.052106+0000","last_became_active":"2024-03-27T06:41:28.399385+0000","last_became_peered":"2024-03-27T06:41:28.399385+0000","last_unstale":"2024-03-27T06:43:34.052106+0000","last_undegraded":"2024-03-27T06:43:34.052106+0000","last_fullsized":"2024-03-27T06:43:34.052106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:16:18.005519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044636+0000","last_change":"2024-03-27T06:41:29.418991+0000","last_active":"2024-03-27T06:43:34.044636+0000","last_peered":"2024-03-27T06:43:34.044636+0000","last_clean":"2024-03-27T06:43:34.044636+0000","last_became_active":"2024-03-27T06:41:29.418527+0000","last_became_peered":"2024-03-27T06:41:29.418527+0000","last_unstale":"2024-03-27T06:43:34.044636+0000","last_undegraded":"2024-03-27T06:43:34.044636+0000","last_fullsized":"2024-03-27T06:43:34.044636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:49:55.913276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051676+0000","last_change":"2024-03-27T06:41:28.397771+0000","last_active":"2024-03-27T06:43:34.051676+0000","last_peered":"2024-03-27T06:43:34.051676+0000","last_clean":"2024-03-27T06:43:34.051676+0000","last_became_active":"2024-03-27T06:41:28.397470+0000","last_became_peered":"2024-03-27T06:41:28.397470+0000","last_unstale":"2024-03-27T06:43:34.051676+0000","last_undegraded":"2024-03-27T06:43:34.051676+0000","last_fullsized":"2024-03-27T06:43:34.051676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T11:37:21.783195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890518+0000","last_change":"2024-03-27T06:41:29.412280+0000","last_active":"2024-03-27T06:43:33.890518+0000","last_peered":"2024-03-27T06:43:33.890518+0000","last_clean":"2024-03-27T06:43:33.890518+0000","last_became_active":"2024-03-27T06:41:29.411336+0000","last_became_peered":"2024-03-27T06:41:29.411336+0000","last_unstale":"2024-03-27T06:43:33.890518+0000","last_undegraded":"2024-03-27T06:43:33.890518+0000","last_fullsized":"2024-03-27T06:43:33.890518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T09:13:03.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890676+0000","last_change":"2024-03-27T06:41:28.403529+0000","last_active":"2024-03-27T06:43:33.890676+0000","last_peered":"2024-03-27T06:43:33.890676+0000","last_clean":"2024-03-27T06:43:33.890676+0000","last_became_active":"2024-03-27T06:41:28.403155+0000","last_became_peered":"2024-03-27T06:41:28.403155+0000","last_unstale":"2024-03-27T06:43:33.890676+0000","last_undegraded":"2024-03-27T06:43:33.890676+0000","last_fullsized":"2024-03-27T06:43:33.890676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:23:53.971374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043609+0000","last_change":"2024-03-27T06:41:29.404929+0000","last_active":"2024-03-27T06:43:34.043609+0000","last_peered":"2024-03-27T06:43:34.043609+0000","last_clean":"2024-03-27T06:43:34.043609+0000","last_became_active":"2024-03-27T06:41:29.404602+0000","last_became_peered":"2024-03-27T06:41:29.404602+0000","last_unstale":"2024-03-27T06:43:34.043609+0000","last_undegraded":"2024-03-27T06:43:34.043609+0000","last_fullsized":"2024-03-27T06:43:34.043609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T12:19:07.379046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788341+0000","last_change":"2024-03-27T06:41:28.398602+0000","last_active":"2024-03-27T06:43:33.788341+0000","last_peered":"2024-03-27T06:43:33.788341+0000","last_clean":"2024-03-27T06:43:33.788341+0000","last_became_active":"2024-03-27T06:41:28.398068+0000","last_became_peered":"2024-03-27T06:41:28.398068+0000","last_unstale":"2024-03-27T06:43:33.788341+0000","last_undegraded":"2024-03-27T06:43:33.788341+0000","last_fullsized":"2024-03-27T06:43:33.788341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:10:53.312044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.891033+0000","last_change":"2024-03-27T06:41:29.420245+0000","last_active":"2024-03-27T06:43:33.891033+0000","last_peered":"2024-03-27T06:43:33.891033+0000","last_clean":"2024-03-27T06:43:33.891033+0000","last_became_active":"2024-03-27T06:41:29.419912+0000","last_became_peered":"2024-03-27T06:41:29.419912+0000","last_unstale":"2024-03-27T06:43:33.891033+0000","last_undegraded":"2024-03-27T06:43:33.891033+0000","last_fullsized":"2024-03-27T06:43:33.891033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:46:58.203083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890685+0000","last_change":"2024-03-27T06:41:28.398595+0000","last_active":"2024-03-27T06:43:33.890685+0000","last_peered":"2024-03-27T06:43:33.890685+0000","last_clean":"2024-03-27T06:43:33.890685+0000","last_became_active":"2024-03-27T06:41:28.398036+0000","last_became_peered":"2024-03-27T06:41:28.398036+0000","last_unstale":"2024-03-27T06:43:33.890685+0000","last_undegraded":"2024-03-27T06:43:33.890685+0000","last_fullsized":"2024-03-27T06:43:33.890685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:47:15.769760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788967+0000","last_change":"2024-03-27T06:41:29.406576+0000","last_active":"2024-03-27T06:43:33.788967+0000","last_peered":"2024-03-27T06:43:33.788967+0000","last_clean":"2024-03-27T06:43:33.788967+0000","last_became_active":"2024-03-27T06:41:29.406222+0000","last_became_peered":"2024-03-27T06:41:29.406222+0000","last_unstale":"2024-03-27T06:43:33.788967+0000","last_undegraded":"2024-03-27T06:43:33.788967+0000","last_fullsized":"2024-03-27T06:43:33.788967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:05:12.446154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.607123+0000","last_change":"2024-03-27T06:41:28.399361+0000","last_active":"2024-03-27T06:43:34.607123+0000","last_peered":"2024-03-27T06:43:34.607123+0000","last_clean":"2024-03-27T06:43:34.607123+0000","last_became_active":"2024-03-27T06:41:28.398021+0000","last_became_peered":"2024-03-27T06:41:28.398021+0000","last_unstale":"2024-03-27T06:43:34.607123+0000","last_undegraded":"2024-03-27T06:43:34.607123+0000","last_fullsized":"2024-03-27T06:43:34.607123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:51:34.106340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789247+0000","last_change":"2024-03-27T06:41:29.405696+0000","last_active":"2024-03-27T06:43:33.789247+0000","last_peered":"2024-03-27T06:43:33.789247+0000","last_clean":"2024-03-27T06:43:33.789247+0000","last_became_active":"2024-03-27T06:41:29.405273+0000","last_became_peered":"2024-03-27T06:41:29.405273+0000","last_unstale":"2024-03-27T06:43:33.789247+0000","last_undegraded":"2024-03-27T06:43:33.789247+0000","last_fullsized":"2024-03-27T06:43:33.789247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:02:58.778464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'41","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.006432+0000","last_change":"2024-03-27T06:41:28.391406+0000","last_active":"2024-03-27T06:43:34.006432+0000","last_peered":"2024-03-27T06:43:34.006432+0000","last_clean":"2024-03-27T06:43:34.006432+0000","last_became_active":"2024-03-27T06:41:28.390950+0000","last_became_peered":"2024-03-27T06:41:28.390950+0000","last_unstale":"2024-03-27T06:43:34.006432+0000","last_undegraded":"2024-03-27T06:43:34.006432+0000","last_fullsized":"2024-03-27T06:43:34.006432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:10:48.347139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788757+0000","last_change":"2024-03-27T06:41:29.410422+0000","last_active":"2024-03-27T06:43:33.788757+0000","last_peered":"2024-03-27T06:43:33.788757+0000","last_clean":"2024-03-27T06:43:33.788757+0000","last_became_active":"2024-03-27T06:41:29.409738+0000","last_became_peered":"2024-03-27T06:41:29.409738+0000","last_unstale":"2024-03-27T06:43:33.788757+0000","last_undegraded":"2024-03-27T06:43:33.788757+0000","last_fullsized":"2024-03-27T06:43:33.788757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:38:17.546504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051045+0000","last_change":"2024-03-27T06:42:21.008586+0000","last_active":"2024-03-27T06:43:34.051045+0000","last_peered":"2024-03-27T06:43:34.051045+0000","last_clean":"2024-03-27T06:43:34.051045+0000","last_became_active":"2024-03-27T06:42:21.004964+0000","last_became_peered":"2024-03-27T06:42:21.004964+0000","last_unstale":"2024-03-27T06:43:34.051045+0000","last_undegraded":"2024-03-27T06:43:34.051045+0000","last_fullsized":"2024-03-27T06:43:34.051045+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:27.371199+0000","last_clean_scrub_stamp":"2024-03-27T06:41:27.371199+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:11:13.246994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789419+0000","last_change":"2024-03-27T06:41:29.408809+0000","last_active":"2024-03-27T06:43:33.789419+0000","last_peered":"2024-03-27T06:43:33.789419+0000","last_clean":"2024-03-27T06:43:33.789419+0000","last_became_active":"2024-03-27T06:41:29.408447+0000","last_became_peered":"2024-03-27T06:41:29.408447+0000","last_unstale":"2024-03-27T06:43:33.789419+0000","last_undegraded":"2024-03-27T06:43:33.789419+0000","last_fullsized":"2024-03-27T06:43:33.789419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:04:55.269490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043772+0000","last_change":"2024-03-27T06:42:20.655859+0000","last_active":"2024-03-27T06:43:34.043772+0000","last_peered":"2024-03-27T06:43:34.043772+0000","last_clean":"2024-03-27T06:43:34.043772+0000","last_became_active":"2024-03-27T06:42:20.655446+0000","last_became_peered":"2024-03-27T06:42:20.655446+0000","last_unstale":"2024-03-27T06:43:34.043772+0000","last_undegraded":"2024-03-27T06:43:34.043772+0000","last_fullsized":"2024-03-27T06:43:34.043772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:43:07.920040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.787928+0000","last_change":"2024-03-27T06:41:29.409896+0000","last_active":"2024-03-27T06:43:33.787928+0000","last_peered":"2024-03-27T06:43:33.787928+0000","last_clean":"2024-03-27T06:43:33.787928+0000","last_became_active":"2024-03-27T06:41:29.408279+0000","last_became_peered":"2024-03-27T06:41:29.408279+0000","last_unstale":"2024-03-27T06:43:33.787928+0000","last_undegraded":"2024-03-27T06:43:33.787928+0000","last_fullsized":"2024-03-27T06:43:33.787928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T15:49:03.257543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.788142+0000","last_change":"2024-03-27T06:41:29.412526+0000","last_active":"2024-03-27T06:43:33.788142+0000","last_peered":"2024-03-27T06:43:33.788142+0000","last_clean":"2024-03-27T06:43:33.788142+0000","last_became_active":"2024-03-27T06:41:29.412180+0000","last_became_peered":"2024-03-27T06:41:29.412180+0000","last_unstale":"2024-03-27T06:43:33.788142+0000","last_undegraded":"2024-03-27T06:43:33.788142+0000","last_fullsized":"2024-03-27T06:43:33.788142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T14:44:47.461457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044158+0000","last_change":"2024-03-27T06:42:20.657735+0000","last_active":"2024-03-27T06:43:34.044158+0000","last_peered":"2024-03-27T06:43:34.044158+0000","last_clean":"2024-03-27T06:43:34.044158+0000","last_became_active":"2024-03-27T06:42:20.657378+0000","last_became_peered":"2024-03-27T06:42:20.657378+0000","last_unstale":"2024-03-27T06:43:34.044158+0000","last_undegraded":"2024-03-27T06:43:34.044158+0000","last_fullsized":"2024-03-27T06:43:34.044158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:13:56.368229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.051027+0000","last_change":"2024-03-27T06:41:29.402442+0000","last_active":"2024-03-27T06:43:34.051027+0000","last_peered":"2024-03-27T06:43:34.051027+0000","last_clean":"2024-03-27T06:43:34.051027+0000","last_became_active":"2024-03-27T06:41:29.402078+0000","last_became_peered":"2024-03-27T06:41:29.402078+0000","last_unstale":"2024-03-27T06:43:34.051027+0000","last_undegraded":"2024-03-27T06:43:34.051027+0000","last_fullsized":"2024-03-27T06:43:34.051027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T17:07:35.344881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.044286+0000","last_change":"2024-03-27T06:41:29.418563+0000","last_active":"2024-03-27T06:43:34.044286+0000","last_peered":"2024-03-27T06:43:34.044286+0000","last_clean":"2024-03-27T06:43:34.044286+0000","last_became_active":"2024-03-27T06:41:29.418209+0000","last_became_peered":"2024-03-27T06:41:29.418209+0000","last_unstale":"2024-03-27T06:43:34.044286+0000","last_undegraded":"2024-03-27T06:43:34.044286+0000","last_fullsized":"2024-03-27T06:43:34.044286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T08:53:58.536148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.789184+0000","last_change":"2024-03-27T06:41:29.400414+0000","last_active":"2024-03-27T06:43:33.789184+0000","last_peered":"2024-03-27T06:43:33.789184+0000","last_clean":"2024-03-27T06:43:33.789184+0000","last_became_active":"2024-03-27T06:41:29.397865+0000","last_became_peered":"2024-03-27T06:41:29.397865+0000","last_unstale":"2024-03-27T06:43:33.789184+0000","last_undegraded":"2024-03-27T06:43:33.789184+0000","last_fullsized":"2024-03-27T06:43:33.789184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T07:35:28.405347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890973+0000","last_change":"2024-03-27T06:41:29.416740+0000","last_active":"2024-03-27T06:43:33.890973+0000","last_peered":"2024-03-27T06:43:33.890973+0000","last_clean":"2024-03-27T06:43:33.890973+0000","last_became_active":"2024-03-27T06:41:29.416417+0000","last_became_peered":"2024-03-27T06:41:29.416417+0000","last_unstale":"2024-03-27T06:43:33.890973+0000","last_undegraded":"2024-03-27T06:43:33.890973+0000","last_fullsized":"2024-03-27T06:43:33.890973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:08:34.186527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.899169+0000","last_change":"2024-03-27T06:41:29.418797+0000","last_active":"2024-03-27T06:43:33.899169+0000","last_peered":"2024-03-27T06:43:33.899169+0000","last_clean":"2024-03-27T06:43:33.899169+0000","last_became_active":"2024-03-27T06:41:29.418472+0000","last_became_peered":"2024-03-27T06:41:29.418472+0000","last_unstale":"2024-03-27T06:43:33.899169+0000","last_undegraded":"2024-03-27T06:43:33.899169+0000","last_fullsized":"2024-03-27T06:43:33.899169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:30:55.540353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:33.890611+0000","last_change":"2024-03-27T06:41:29.416124+0000","last_active":"2024-03-27T06:43:33.890611+0000","last_peered":"2024-03-27T06:43:33.890611+0000","last_clean":"2024-03-27T06:43:33.890611+0000","last_became_active":"2024-03-27T06:41:29.415665+0000","last_became_peered":"2024-03-27T06:41:29.415665+0000","last_unstale":"2024-03-27T06:43:33.890611+0000","last_undegraded":"2024-03-27T06:43:33.890611+0000","last_fullsized":"2024-03-27T06:43:33.890611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T10:46:04.160705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.053062+0000","last_change":"2024-03-27T06:41:29.417165+0000","last_active":"2024-03-27T06:43:34.053062+0000","last_peered":"2024-03-27T06:43:34.053062+0000","last_clean":"2024-03-27T06:43:34.053062+0000","last_became_active":"2024-03-27T06:41:29.416856+0000","last_became_peered":"2024-03-27T06:41:29.416856+0000","last_unstale":"2024-03-27T06:43:34.053062+0000","last_undegraded":"2024-03-27T06:43:34.053062+0000","last_fullsized":"2024-03-27T06:43:34.053062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T13:55:20.025026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.050772+0000","last_change":"2024-03-27T06:41:29.412377+0000","last_active":"2024-03-27T06:43:34.050772+0000","last_peered":"2024-03-27T06:43:34.050772+0000","last_clean":"2024-03-27T06:43:34.050772+0000","last_became_active":"2024-03-27T06:41:29.411761+0000","last_became_peered":"2024-03-27T06:41:29.411761+0000","last_unstale":"2024-03-27T06:43:34.050772+0000","last_undegraded":"2024-03-27T06:43:34.050772+0000","last_fullsized":"2024-03-27T06:43:34.050772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T16:16:56.844342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-27T06:43:34.043855+0000","last_change":"2024-03-27T06:41:29.413660+0000","last_active":"2024-03-27T06:43:34.043855+0000","last_peered":"2024-03-27T06:43:34.043855+0000","last_clean":"2024-03-27T06:43:34.043855+0000","last_became_active":"2024-03-27T06:41:29.413201+0000","last_became_peered":"2024-03-27T06:41:29.413201+0000","last_unstale":"2024-03-27T06:43:34.043855+0000","last_undegraded":"2024-03-27T06:43:34.043855+0000","last_fullsized":"2024-03-27T06:43:34.043855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-27T06:41:28.376269+0000","last_clean_scrub_stamp":"2024-03-27T06:41:28.376269+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T18:14:46.446595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106419,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600992,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1335,"num_read_kb":783,"num_write":374,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1802976,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":460074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1155072,"data_stored":1259064,"data_compressed":44525,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":861923,"data_compressed":22042,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037813,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":862077,"data_compressed":22042,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":1063449,"data_compressed":38369,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":520990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":470545,"data_compressed":2896,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":537997,"data_compressed":2896,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":993065,"data_compressed":28198,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":459890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1193413,"data_compressed":44525,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35473,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135343,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":394492,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200994,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-27T06:43:39.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:43:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.225+0000 7fd85cd5c640 1 -- 172.21.15.16:0/4236210083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd858101410 msgr2=0x7fd85810d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.225+0000 7fd85cd5c640 1 --2- 172.21.15.16:0/4236210083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd858101410 0x7fd85810d920 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd848009cd0 tx=0x7fd84802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.225+0000 7fd85cd5c640 1 -- 172.21.15.16:0/4236210083 shutdown_connections 2024-03-27T06:43:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.225+0000 7fd85cd5c640 1 --2- 172.21.15.16:0/4236210083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd858101410 0x7fd85810d920 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.225+0000 7fd85cd5c640 1 --2- 172.21.15.16:0/4236210083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858100a50 0x7fd858100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.225+0000 7fd85cd5c640 1 --2- 172.21.15.16:0/4236210083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd858106a60 0x7fd858106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.225+0000 7fd85cd5c640 1 -- 172.21.15.16:0/4236210083 >> 172.21.15.16:0/4236210083 conn(0x7fd858077b50 msgr2=0x7fd858077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.225+0000 7fd85cd5c640 1 -- 172.21.15.16:0/4236210083 shutdown_connections 2024-03-27T06:43:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.225+0000 7fd85cd5c640 1 -- 172.21.15.16:0/4236210083 wait complete. 2024-03-27T06:43:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.225+0000 7fd85cd5c640 1 Processor -- start 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.225+0000 7fd85cd5c640 1 -- start start 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85cd5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd858100a50 0x7fd8581973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85cd5c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858101410 0x7fd858197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85cd5c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd858106a60 0x7fd85819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85cd5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd858110f90 con 0x7fd858100a50 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85cd5c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd858110e10 con 0x7fd858101410 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85cd5c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd858111110 con 0x7fd858106a60 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd858100a50 0x7fd8581973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd858100a50 0x7fd8581973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51218/0 (socket says 172.21.15.16:51218) 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85659c640 1 -- 172.21.15.16:0/3353034357 learned_addr learned my addr 172.21.15.16:0/3353034357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd855d9b640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858101410 0x7fd858197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd856d9d640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd858106a60 0x7fd85819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85659c640 1 -- 172.21.15.16:0/3353034357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd858106a60 msgr2=0x7fd85819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85659c640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd858106a60 0x7fd85819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85659c640 1 -- 172.21.15.16:0/3353034357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858101410 msgr2=0x7fd858197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85659c640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858101410 0x7fd858197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85659c640 1 -- 172.21.15.16:0/3353034357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85819c3e0 con 0x7fd858100a50 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85659c640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd858100a50 0x7fd8581973f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd85000e9f0 tx=0x7fd85000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd84f7fe640 1 -- 172.21.15.16:0/3353034357 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85000cd80 con 0x7fd858100a50 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd85cd5c640 1 -- 172.21.15.16:0/3353034357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85819c6d0 con 0x7fd858100a50 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.226+0000 7fd84f7fe640 1 -- 172.21.15.16:0/3353034357 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd850005520 con 0x7fd858100a50 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.227+0000 7fd84f7fe640 1 -- 172.21.15.16:0/3353034357 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850010690 con 0x7fd858100a50 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.227+0000 7fd85cd5c640 1 -- 172.21.15.16:0/3353034357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8581a3f60 con 0x7fd858100a50 2024-03-27T06:43:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.227+0000 7fd85cd5c640 1 -- 172.21.15.16:0/3353034357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd858068940 con 0x7fd858100a50 2024-03-27T06:43:39.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: Updating smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:43:39.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: Updating smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:43:39.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: Updating smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:43:39.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:39.226 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/504699565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: Updating smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: Updating smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: Updating smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.227 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:43:39.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:43:39.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:43:39.228 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.229+0000 7fd84f7fe640 1 -- 172.21.15.16:0/3353034357 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7fd8500046c0 con 0x7fd858100a50 2024-03-27T06:43:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.229+0000 7fd84f7fe640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd8440a38a0 0x7fd8440a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.229+0000 7fd84f7fe640 1 -- 172.21.15.16:0/3353034357 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd8500d0700 con 0x7fd858100a50 2024-03-27T06:43:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.229+0000 7fd855d9b640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd8440a38a0 0x7fd8440a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.230+0000 7fd855d9b640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd8440a38a0 0x7fd8440a5d60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd858105040 tx=0x7fd840005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.234+0000 7fd84f7fe640 1 -- 172.21.15.16:0/3353034357 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd85009cd70 con 0x7fd858100a50 2024-03-27T06:43:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: Updating smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:43:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: Updating smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:43:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: Updating smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-27T06:43:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/504699565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:43:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: Updating smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: Updating smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: Updating smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:43:39.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:43:39.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:43:39.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:43:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.421+0000 7fd85cd5c640 1 -- 172.21.15.16:0/3353034357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd858108fc0 con 0x7fd858100a50 2024-03-27T06:43:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.423+0000 7fd84f7fe640 1 -- 172.21.15.16:0/3353034357 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22982 (secure 0 0 0) 0x7fd85009c670 con 0x7fd858100a50 2024-03-27T06:43:39.418 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:39.418 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:43:33.780855+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3655179049":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3519122217":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6800/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/212881673":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/361067788":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:43:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 -- 172.21.15.16:0/3353034357 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd8440a38a0 msgr2=0x7fd8440a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd8440a38a0 0x7fd8440a5d60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd858105040 tx=0x7fd840005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 -- 172.21.15.16:0/3353034357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd858100a50 msgr2=0x7fd8581973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd858100a50 0x7fd8581973f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd85000e9f0 tx=0x7fd85000edf0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 -- 172.21.15.16:0/3353034357 shutdown_connections 2024-03-27T06:43:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd8440a38a0 0x7fd8440a5d60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd858106a60 0x7fd85819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd858101410 0x7fd858197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 --2- 172.21.15.16:0/3353034357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd858100a50 0x7fd8581973f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 -- 172.21.15.16:0/3353034357 >> 172.21.15.16:0/3353034357 conn(0x7fd858077b50 msgr2=0x7fd858103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 -- 172.21.15.16:0/3353034357 shutdown_connections 2024-03-27T06:43:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.425+0000 7fd85cd5c640 1 -- 172.21.15.16:0/3353034357 wait complete. 2024-03-27T06:43:39.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T06:41:26.393978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T06:41:27.707469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:43:39.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T06:43:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.542+0000 7f8551284640 1 -- 172.21.15.16:0/1888768716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f854c100dd0 msgr2=0x7f854c10e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.542+0000 7f8551284640 1 --2- 172.21.15.16:0/1888768716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f854c100dd0 0x7f854c10e9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f853c009d00 tx=0x7f853c02c180 comp rx=0 tx=0).stop 2024-03-27T06:43:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.542+0000 7f8551284640 1 -- 172.21.15.16:0/1888768716 shutdown_connections 2024-03-27T06:43:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.542+0000 7f8551284640 1 --2- 172.21.15.16:0/1888768716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f854c100dd0 0x7f854c10e9d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.542+0000 7f8551284640 1 --2- 172.21.15.16:0/1888768716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c100410 0x7f854c100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.542+0000 7f8551284640 1 --2- 172.21.15.16:0/1888768716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f854c107d50 0x7f854c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.542+0000 7f8551284640 1 -- 172.21.15.16:0/1888768716 >> 172.21.15.16:0/1888768716 conn(0x7f854c075d90 msgr2=0x7f854c0761a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.542+0000 7f8551284640 1 -- 172.21.15.16:0/1888768716 shutdown_connections 2024-03-27T06:43:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.542+0000 7f8551284640 1 -- 172.21.15.16:0/1888768716 wait complete. 2024-03-27T06:43:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.543+0000 7f8551284640 1 Processor -- start 2024-03-27T06:43:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.543+0000 7f8551284640 1 -- start start 2024-03-27T06:43:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.543+0000 7f8551284640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f854c100410 0x7f854c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.543+0000 7f8551284640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c100dd0 0x7f854c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.543+0000 7f8551284640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f854c107d50 0x7f854c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.543+0000 7f8551284640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1120d0 con 0x7f854c100dd0 2024-03-27T06:43:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.543+0000 7f8551284640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f854c111f50 con 0x7f854c100410 2024-03-27T06:43:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.543+0000 7f8551284640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f854c112250 con 0x7f854c107d50 2024-03-27T06:43:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.543+0000 7f854a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c100dd0 0x7f854c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f854a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c100dd0 0x7f854c198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51238/0 (socket says 172.21.15.16:51238) 2024-03-27T06:43:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f854a7fc640 1 -- 172.21.15.16:0/1966755628 learned_addr learned my addr 172.21.15.16:0/1966755628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f854affd640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f854c100410 0x7f854c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f854b7fe640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f854c107d50 0x7f854c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f854a7fc640 1 -- 172.21.15.16:0/1966755628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f854c107d50 msgr2=0x7f854c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f854a7fc640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f854c107d50 0x7f854c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f854a7fc640 1 -- 172.21.15.16:0/1966755628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f854c100410 msgr2=0x7f854c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f854a7fc640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f854c100410 0x7f854c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f854a7fc640 1 -- 172.21.15.16:0/1966755628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c19d510 con 0x7f854c100dd0 2024-03-27T06:43:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f854a7fc640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c100dd0 0x7f854c198ab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f853400ea60 tx=0x7f853400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f8543fff640 1 -- 172.21.15.16:0/1966755628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853400cde0 con 0x7f854c100dd0 2024-03-27T06:43:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f8551284640 1 -- 172.21.15.16:0/1966755628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c19d800 con 0x7f854c100dd0 2024-03-27T06:43:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f8551284640 1 -- 172.21.15.16:0/1966755628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c1a5090 con 0x7f854c100dd0 2024-03-27T06:43:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f8543fff640 1 -- 172.21.15.16:0/1966755628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85340035f0 con 0x7f854c100dd0 2024-03-27T06:43:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.544+0000 7f8543fff640 1 -- 172.21.15.16:0/1966755628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534018690 con 0x7f854c100dd0 2024-03-27T06:43:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.545+0000 7f8551284640 1 -- 172.21.15.16:0/1966755628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852c005290 con 0x7f854c100dd0 2024-03-27T06:43:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.546+0000 7f8543fff640 1 -- 172.21.15.16:0/1966755628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7f8534004020 con 0x7f854c100dd0 2024-03-27T06:43:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.547+0000 7f8543fff640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f85380a3920 0x7f85380a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.547+0000 7f854affd640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f85380a3920 0x7f85380a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.547+0000 7f8543fff640 1 -- 172.21.15.16:0/1966755628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f85340d05b0 con 0x7f854c100dd0 2024-03-27T06:43:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.547+0000 7f854affd640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f85380a3920 0x7f85380a5de0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85440095f0 tx=0x7f8544005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.551+0000 7f8543fff640 1 -- 172.21.15.16:0/1966755628 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f853409cbc0 con 0x7f854c100dd0 2024-03-27T06:43:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.737+0000 7f8551284640 1 -- 172.21.15.16:0/1966755628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f852c005850 con 0x7f854c100dd0 2024-03-27T06:43:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.738+0000 7f8543fff640 1 -- 172.21.15.16:0/1966755628 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f853409c4c0 con 0x7f854c100dd0 2024-03-27T06:43:39.733 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:43:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 -- 172.21.15.16:0/1966755628 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f85380a3920 msgr2=0x7f85380a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f85380a3920 0x7f85380a5de0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85440095f0 tx=0x7f8544005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 -- 172.21.15.16:0/1966755628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c100dd0 msgr2=0x7f854c198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c100dd0 0x7f854c198ab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f853400ea60 tx=0x7f853400ef20 comp rx=0 tx=0).stop 2024-03-27T06:43:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 -- 172.21.15.16:0/1966755628 shutdown_connections 2024-03-27T06:43:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f85380a3920 0x7f85380a5de0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f854c107d50 0x7f854c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f854c100dd0 0x7f854c198ab0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 --2- 172.21.15.16:0/1966755628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f854c100410 0x7f854c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 -- 172.21.15.16:0/1966755628 >> 172.21.15.16:0/1966755628 conn(0x7f854c075d90 msgr2=0x7f854c0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 -- 172.21.15.16:0/1966755628 shutdown_connections 2024-03-27T06:43:39.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.740+0000 7f8551284640 1 -- 172.21.15.16:0/1966755628 wait complete. 2024-03-27T06:43:39.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T06:43:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.859+0000 7f50dddeb640 1 -- 172.21.15.16:0/101947259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8101ac0 msgr2=0x7f50d8101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.859+0000 7f50dddeb640 1 --2- 172.21.15.16:0/101947259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8101ac0 0x7f50d8101f40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009930 tx=0x7f50cc02bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.859+0000 7f50dddeb640 1 -- 172.21.15.16:0/101947259 shutdown_connections 2024-03-27T06:43:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.859+0000 7f50dddeb640 1 --2- 172.21.15.16:0/101947259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50d8102480 0x7f50d810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.859+0000 7f50dddeb640 1 --2- 172.21.15.16:0/101947259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8101ac0 0x7f50d8101f40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.859+0000 7f50dddeb640 1 --2- 172.21.15.16:0/101947259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50d8107ad0 0x7f50d8107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.859+0000 7f50dddeb640 1 -- 172.21.15.16:0/101947259 >> 172.21.15.16:0/101947259 conn(0x7f50d80fd5b0 msgr2=0x7f50d80ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.859+0000 7f50dddeb640 1 -- 172.21.15.16:0/101947259 shutdown_connections 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.859+0000 7f50dddeb640 1 -- 172.21.15.16:0/101947259 wait complete. 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50dddeb640 1 Processor -- start 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50dddeb640 1 -- start start 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50dddeb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50d8101ac0 0x7f50d81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50dddeb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50d8102480 0x7f50d8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50dddeb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8107ad0 0x7f50d819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50dddeb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8111fb0 con 0x7f50d8107ad0 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50dddeb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8111e30 con 0x7f50d8101ac0 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50dddeb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8112130 con 0x7f50d8102480 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8107ad0 0x7f50d819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50d77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50d8101ac0 0x7f50d81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50d77fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50d8101ac0 0x7f50d81984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:43070/0 (socket says 172.21.15.16:43070) 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8107ad0 0x7f50d819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51244/0 (socket says 172.21.15.16:51244) 2024-03-27T06:43:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50d8102480 0x7f50d8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.860+0000 7f50d77fe640 1 -- 172.21.15.16:0/4235664240 learned_addr learned my addr 172.21.15.16:0/4235664240 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50d77fe640 1 -- 172.21.15.16:0/4235664240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50d8102480 msgr2=0x7f50d8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50d77fe640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50d8102480 0x7f50d8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50d77fe640 1 -- 172.21.15.16:0/4235664240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8107ad0 msgr2=0x7f50d819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50d77fe640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8107ad0 0x7f50d819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50d77fe640 1 -- 172.21.15.16:0/4235664240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d819d4e0 con 0x7f50d8101ac0 2024-03-27T06:43:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50d7fff640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8107ad0 0x7f50d819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:39.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50d77fe640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50d8101ac0 0x7f50d81984f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50c800e9f0 tx=0x7f50c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50d4ff9640 1 -- 172.21.15.16:0/4235664240 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c800cd80 con 0x7f50d8101ac0 2024-03-27T06:43:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50dddeb640 1 -- 172.21.15.16:0/4235664240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d819d7d0 con 0x7f50d8101ac0 2024-03-27T06:43:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50d4ff9640 1 -- 172.21.15.16:0/4235664240 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f50c8010930 con 0x7f50d8101ac0 2024-03-27T06:43:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50d4ff9640 1 -- 172.21.15.16:0/4235664240 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8018b00 con 0x7f50d8101ac0 2024-03-27T06:43:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.861+0000 7f50dddeb640 1 -- 172.21.15.16:0/4235664240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d81a5060 con 0x7f50d8101ac0 2024-03-27T06:43:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.862+0000 7f50dddeb640 1 -- 172.21.15.16:0/4235664240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d8068d90 con 0x7f50d8101ac0 2024-03-27T06:43:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.864+0000 7f50d4ff9640 1 -- 172.21.15.16:0/4235664240 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7f50c8018ca0 con 0x7f50d8101ac0 2024-03-27T06:43:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.864+0000 7f50d4ff9640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f50c00a38a0 0x7f50c00a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.864+0000 7f50d4ff9640 1 -- 172.21.15.16:0/4235664240 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f50c80d09c0 con 0x7f50d8101ac0 2024-03-27T06:43:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.864+0000 7f50d6ffd640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f50c00a38a0 0x7f50c00a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.865+0000 7f50d6ffd640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f50c00a38a0 0x7f50c00a5d60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f50cc005ab0 tx=0x7f50cc005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:39.872+0000 7f50d4ff9640 1 -- 172.21.15.16:0/4235664240 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f50c809c9e0 con 0x7f50d8101ac0 2024-03-27T06:43:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.055+0000 7f50dddeb640 1 -- 172.21.15.16:0/4235664240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f50d8199840 con 0x7f50d8101ac0 2024-03-27T06:43:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.055+0000 7f50d4ff9640 1 -- 172.21.15.16:0/4235664240 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f50c80a0ce0 con 0x7f50d8101ac0 2024-03-27T06:43:40.050 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.057+0000 7f50dddeb640 1 -- 172.21.15.16:0/4235664240 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f50c00a38a0 msgr2=0x7f50c00a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.057+0000 7f50dddeb640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f50c00a38a0 0x7f50c00a5d60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f50cc005ab0 tx=0x7f50cc005fd0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.057+0000 7f50dddeb640 1 -- 172.21.15.16:0/4235664240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50d8101ac0 msgr2=0x7f50d81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.057+0000 7f50dddeb640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50d8101ac0 0x7f50d81984f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50c800e9f0 tx=0x7f50c800eeb0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.058+0000 7f50dddeb640 1 -- 172.21.15.16:0/4235664240 shutdown_connections 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.058+0000 7f50dddeb640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8107ad0 0x7f50d819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.058+0000 7f50dddeb640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f50c00a38a0 0x7f50c00a5d60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.058+0000 7f50dddeb640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50d8102480 0x7f50d8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.058+0000 7f50dddeb640 1 --2- 172.21.15.16:0/4235664240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50d8101ac0 0x7f50d81984f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.058+0000 7f50dddeb640 1 -- 172.21.15.16:0/4235664240 >> 172.21.15.16:0/4235664240 conn(0x7f50d80fd5b0 msgr2=0x7f50d80fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.058+0000 7f50dddeb640 1 -- 172.21.15.16:0/4235664240 shutdown_connections 2024-03-27T06:43:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.058+0000 7f50dddeb640 1 -- 172.21.15.16:0/4235664240 wait complete. 2024-03-27T06:43:40.062 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:39 smithi016 ceph-mon[31908]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:43:40.062 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:39 smithi016 ceph-mon[31908]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:43:40.062 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:39 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3353034357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:40.062 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:39 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1966755628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:40.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T06:43:40.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.174+0000 7fea5daaf640 1 -- 172.21.15.16:0/4118533882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58107b90 msgr2=0x7fea58107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.174+0000 7fea5daaf640 1 --2- 172.21.15.16:0/4118533882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58107b90 0x7fea58107f70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fea48009cd0 tx=0x7fea4802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.174+0000 7fea5daaf640 1 -- 172.21.15.16:0/4118533882 shutdown_connections 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.174+0000 7fea5daaf640 1 --2- 172.21.15.16:0/4118533882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea581024b0 0x7fea5810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.174+0000 7fea5daaf640 1 --2- 172.21.15.16:0/4118533882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea58101af0 0x7fea58101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.174+0000 7fea5daaf640 1 --2- 172.21.15.16:0/4118533882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58107b90 0x7fea58107f70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.174+0000 7fea5daaf640 1 -- 172.21.15.16:0/4118533882 >> 172.21.15.16:0/4118533882 conn(0x7fea580fd580 msgr2=0x7fea580ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.174+0000 7fea5daaf640 1 -- 172.21.15.16:0/4118533882 shutdown_connections 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.175+0000 7fea5daaf640 1 -- 172.21.15.16:0/4118533882 wait complete. 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.175+0000 7fea5daaf640 1 Processor -- start 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.175+0000 7fea5daaf640 1 -- start start 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.175+0000 7fea5daaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58101af0 0x7fea58198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.175+0000 7fea5daaf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea581024b0 0x7fea58198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.175+0000 7fea5daaf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea58107b90 0x7fea5819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.175+0000 7fea5daaf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea58112030 con 0x7fea58101af0 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.175+0000 7fea5daaf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fea58111eb0 con 0x7fea58107b90 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.175+0000 7fea5daaf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea581121b0 con 0x7fea581024b0 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58101af0 0x7fea58198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58101af0 0x7fea58198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51266/0 (socket says 172.21.15.16:51266) 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea577fe640 1 -- 172.21.15.16:0/2098695264 learned_addr learned my addr 172.21.15.16:0/2098695264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea56ffd640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea581024b0 0x7fea58198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea57fff640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea58107b90 0x7fea5819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea577fe640 1 -- 172.21.15.16:0/2098695264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea581024b0 msgr2=0x7fea58198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea577fe640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea581024b0 0x7fea58198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea577fe640 1 -- 172.21.15.16:0/2098695264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea58107b90 msgr2=0x7fea5819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea577fe640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea58107b90 0x7fea5819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea577fe640 1 -- 172.21.15.16:0/2098695264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5819d4d0 con 0x7fea58101af0 2024-03-27T06:43:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea577fe640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58101af0 0x7fea58198530 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fea4802c700 tx=0x7fea48034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea54ff9640 1 -- 172.21.15.16:0/2098695264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48039040 con 0x7fea58101af0 2024-03-27T06:43:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea54ff9640 1 -- 172.21.15.16:0/2098695264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fea48044070 con 0x7fea58101af0 2024-03-27T06:43:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea5daaf640 1 -- 172.21.15.16:0/2098695264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5819d760 con 0x7fea58101af0 2024-03-27T06:43:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea54ff9640 1 -- 172.21.15.16:0/2098695264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48004500 con 0x7fea58101af0 2024-03-27T06:43:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.176+0000 7fea5daaf640 1 -- 172.21.15.16:0/2098695264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea581a5050 con 0x7fea58101af0 2024-03-27T06:43:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.177+0000 7fea5daaf640 1 -- 172.21.15.16:0/2098695264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea58068d90 con 0x7fea58101af0 2024-03-27T06:43:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.178+0000 7fea54ff9640 1 -- 172.21.15.16:0/2098695264 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7fea480046a0 con 0x7fea58101af0 2024-03-27T06:43:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.179+0000 7fea54ff9640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fea400a38a0 0x7fea400a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.179+0000 7fea56ffd640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fea400a38a0 0x7fea400a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.179+0000 7fea54ff9640 1 -- 172.21.15.16:0/2098695264 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fea480f3100 con 0x7fea58101af0 2024-03-27T06:43:40.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.180+0000 7fea56ffd640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fea400a38a0 0x7fea400a5d60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea581070b0 tx=0x7fea4c005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.183+0000 7fea54ff9640 1 -- 172.21.15.16:0/2098695264 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fea480bf820 con 0x7fea58101af0 2024-03-27T06:43:40.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:39 smithi156 ceph-mon[36261]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:43:40.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:39 smithi156 ceph-mon[36261]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:43:40.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:39 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3353034357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:40.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:39 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1966755628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:40.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:39 smithi170 ceph-mon[36438]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:43:40.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:39 smithi170 ceph-mon[36438]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-27T06:43:40.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:39 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3353034357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:40.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:39 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1966755628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:43:40.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.367+0000 7fea5daaf640 1 -- 172.21.15.16:0/2098695264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fea5810a060 con 0x7fea58101af0 2024-03-27T06:43:40.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.368+0000 7fea54ff9640 1 -- 172.21.15.16:0/2098695264 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fea480bf120 con 0x7fea58101af0 2024-03-27T06:43:40.363 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 -- 172.21.15.16:0/2098695264 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fea400a38a0 msgr2=0x7fea400a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fea400a38a0 0x7fea400a5d60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea581070b0 tx=0x7fea4c005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 -- 172.21.15.16:0/2098695264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58101af0 msgr2=0x7fea58198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58101af0 0x7fea58198530 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fea4802c700 tx=0x7fea48034440 comp rx=0 tx=0).stop 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 -- 172.21.15.16:0/2098695264 shutdown_connections 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea58107b90 0x7fea5819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fea400a38a0 0x7fea400a5d60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea581024b0 0x7fea58198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 --2- 172.21.15.16:0/2098695264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea58101af0 0x7fea58198530 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 -- 172.21.15.16:0/2098695264 >> 172.21.15.16:0/2098695264 conn(0x7fea580fd580 msgr2=0x7fea580fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 -- 172.21.15.16:0/2098695264 shutdown_connections 2024-03-27T06:43:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.370+0000 7fea5daaf640 1 -- 172.21.15.16:0/2098695264 wait complete. 2024-03-27T06:43:40.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:43:40.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.483+0000 7f5fb312c640 1 -- 172.21.15.16:0/69684587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac1024a0 msgr2=0x7f5fac10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.483+0000 7f5fb312c640 1 --2- 172.21.15.16:0/69684587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac1024a0 0x7f5fac10e9b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009cd0 tx=0x7f5fa402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 -- 172.21.15.16:0/69684587 shutdown_connections 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 --2- 172.21.15.16:0/69684587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac1024a0 0x7f5fac10e9b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 --2- 172.21.15.16:0/69684587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac101ae0 0x7f5fac101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 --2- 172.21.15.16:0/69684587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fac107b80 0x7f5fac107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 -- 172.21.15.16:0/69684587 >> 172.21.15.16:0/69684587 conn(0x7f5fac0fd5f0 msgr2=0x7f5fac0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 -- 172.21.15.16:0/69684587 shutdown_connections 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 -- 172.21.15.16:0/69684587 wait complete. 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 Processor -- start 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 -- start start 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac101ae0 0x7f5fac1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fac1024a0 0x7f5fac198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac107b80 0x7f5fac19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac111ff0 con 0x7f5fac101ae0 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac111e70 con 0x7f5fac1024a0 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb312c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac112170 con 0x7f5fac107b80 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb0ec8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac101ae0 0x7f5fac1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb0ec8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac101ae0 0x7f5fac1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51272/0 (socket says 172.21.15.16:51272) 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.484+0000 7f5fb0ec8640 1 -- 172.21.15.16:0/549991188 learned_addr learned my addr 172.21.15.16:0/549991188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fabfff640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fac1024a0 0x7f5fac198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fb16c9640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac107b80 0x7f5fac19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fb0ec8640 1 -- 172.21.15.16:0/549991188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac107b80 msgr2=0x7f5fac19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fb0ec8640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac107b80 0x7f5fac19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fb0ec8640 1 -- 172.21.15.16:0/549991188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fac1024a0 msgr2=0x7f5fac198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fb0ec8640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fac1024a0 0x7f5fac198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fb0ec8640 1 -- 172.21.15.16:0/549991188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac19d4c0 con 0x7f5fac101ae0 2024-03-27T06:43:40.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fb0ec8640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac101ae0 0x7f5fac1984d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000e9f0 tx=0x7f5fa000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:40.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fa9ffb640 1 -- 172.21.15.16:0/549991188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000cd80 con 0x7f5fac101ae0 2024-03-27T06:43:40.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fa9ffb640 1 -- 172.21.15.16:0/549991188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5fa0010930 con 0x7f5fac101ae0 2024-03-27T06:43:40.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fb312c640 1 -- 172.21.15.16:0/549991188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac19d7b0 con 0x7f5fac101ae0 2024-03-27T06:43:40.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fa9ffb640 1 -- 172.21.15.16:0/549991188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0018b00 con 0x7f5fac101ae0 2024-03-27T06:43:40.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.485+0000 7f5fb312c640 1 -- 172.21.15.16:0/549991188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1a5040 con 0x7f5fac101ae0 2024-03-27T06:43:40.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.486+0000 7f5fb312c640 1 -- 172.21.15.16:0/549991188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac068d90 con 0x7f5fac101ae0 2024-03-27T06:43:40.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.487+0000 7f5fa9ffb640 1 -- 172.21.15.16:0/549991188 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7f5fa0018ca0 con 0x7f5fac101ae0 2024-03-27T06:43:40.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.488+0000 7f5fa9ffb640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5f9c0a38a0 0x7f5f9c0a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.488+0000 7f5fa9ffb640 1 -- 172.21.15.16:0/549991188 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5fa00d09c0 con 0x7f5fac101ae0 2024-03-27T06:43:40.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.488+0000 7f5fabfff640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5f9c0a38a0 0x7f5f9c0a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.488+0000 7f5fabfff640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5f9c0a38a0 0x7f5f9c0a5d60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f980095f0 tx=0x7f5f98005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:40.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.491+0000 7f5fa9ffb640 1 -- 172.21.15.16:0/549991188 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5fa009c9e0 con 0x7f5fac101ae0 2024-03-27T06:43:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.717+0000 7f5fb312c640 1 -- 172.21.15.16:0/549991188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fac1a5210 con 0x7f5fac101ae0 2024-03-27T06:43:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.718+0000 7f5fa9ffb640 1 -- 172.21.15.16:0/549991188 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11932 (secure 0 0 0) 0x7f5fa00a0ce0 con 0x7f5fac101ae0 2024-03-27T06:43:40.713 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:40.713 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:43:32.824696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24467,24527,24500,24472,24493]},"id":1}]} 2024-03-27T06:43:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-03-27T06:43:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 -- 172.21.15.16:0/549991188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5f9c0a38a0 msgr2=0x7f5f9c0a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5f9c0a38a0 0x7f5f9c0a5d60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f980095f0 tx=0x7f5f98005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 -- 172.21.15.16:0/549991188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac101ae0 msgr2=0x7f5fac1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac101ae0 0x7f5fac1984d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000e9f0 tx=0x7f5fa000eeb0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 -- 172.21.15.16:0/549991188 shutdown_connections 2024-03-27T06:43:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5f9c0a38a0 0x7f5f9c0a5d60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fac107b80 0x7f5fac19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fac1024a0 0x7f5fac198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 --2- 172.21.15.16:0/549991188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac101ae0 0x7f5fac1984d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 -- 172.21.15.16:0/549991188 >> 172.21.15.16:0/549991188 conn(0x7f5fac0fd5f0 msgr2=0x7f5fac0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 -- 172.21.15.16:0/549991188 shutdown_connections 2024-03-27T06:43:40.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.720+0000 7f5fb312c640 1 -- 172.21.15.16:0/549991188 wait complete. 2024-03-27T06:43:40.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:43:40.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.835+0000 7f492f08f640 1 -- 172.21.15.16:0/815679624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4928100a10 msgr2=0x7f4928100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.835+0000 7f492f08f640 1 --2- 172.21.15.16:0/815679624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4928100a10 0x7f4928100e90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4914009930 tx=0x7f491402bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.835+0000 7f492f08f640 1 -- 172.21.15.16:0/815679624 shutdown_connections 2024-03-27T06:43:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.835+0000 7f492f08f640 1 --2- 172.21.15.16:0/815679624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49281013d0 0x7f492810d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.835+0000 7f492f08f640 1 --2- 172.21.15.16:0/815679624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4928100a10 0x7f4928100e90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.835+0000 7f492f08f640 1 --2- 172.21.15.16:0/815679624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928106a20 0x7f4928106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.835+0000 7f492f08f640 1 -- 172.21.15.16:0/815679624 >> 172.21.15.16:0/815679624 conn(0x7f4928077af0 msgr2=0x7f4928077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.835+0000 7f492f08f640 1 -- 172.21.15.16:0/815679624 shutdown_connections 2024-03-27T06:43:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.835+0000 7f492f08f640 1 -- 172.21.15.16:0/815679624 wait complete. 2024-03-27T06:43:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.835+0000 7f492f08f640 1 Processor -- start 2024-03-27T06:43:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.835+0000 7f492f08f640 1 -- start start 2024-03-27T06:43:40.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f492f08f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4928100a10 0x7f49281984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f492f08f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49281013d0 0x7f4928198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f492f08f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928106a20 0x7f492819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f492f08f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4928110f00 con 0x7f49281013d0 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f492f08f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f4928110d80 con 0x7f4928100a10 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f492f08f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4928111080 con 0x7f4928106a20 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4927fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49281013d0 0x7f4928198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f492ce2b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4928100a10 0x7f49281984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4927fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49281013d0 0x7f4928198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51288/0 (socket says 172.21.15.16:51288) 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4927fff640 1 -- 172.21.15.16:0/1265368055 learned_addr learned my addr 172.21.15.16:0/1265368055 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f492d62c640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928106a20 0x7f492819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4927fff640 1 -- 172.21.15.16:0/1265368055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928106a20 msgr2=0x7f492819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4927fff640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928106a20 0x7f492819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4927fff640 1 -- 172.21.15.16:0/1265368055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4928100a10 msgr2=0x7f49281984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:40.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4927fff640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4928100a10 0x7f49281984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4927fff640 1 -- 172.21.15.16:0/1265368055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492819d4e0 con 0x7f49281013d0 2024-03-27T06:43:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f492ce2b640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4928100a10 0x7f49281984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4927fff640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49281013d0 0x7f4928198a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4914009900 tx=0x7f4914004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4925ffb640 1 -- 172.21.15.16:0/1265368055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914043020 con 0x7f49281013d0 2024-03-27T06:43:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f492f08f640 1 -- 172.21.15.16:0/1265368055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492819d770 con 0x7f49281013d0 2024-03-27T06:43:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4925ffb640 1 -- 172.21.15.16:0/1265368055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4914004e50 con 0x7f49281013d0 2024-03-27T06:43:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.836+0000 7f4925ffb640 1 -- 172.21.15.16:0/1265368055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914003a10 con 0x7f49281013d0 2024-03-27T06:43:40.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.837+0000 7f492f08f640 1 -- 172.21.15.16:0/1265368055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49281a5060 con 0x7f49281013d0 2024-03-27T06:43:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.838+0000 7f492f08f640 1 -- 172.21.15.16:0/1265368055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4928068d90 con 0x7f49281013d0 2024-03-27T06:43:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.839+0000 7f4925ffb640 1 -- 172.21.15.16:0/1265368055 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7f4914046020 con 0x7f49281013d0 2024-03-27T06:43:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.839+0000 7f4925ffb640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f49180a38a0 0x7f49180a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.840+0000 7f492ce2b640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f49180a38a0 0x7f49180a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.840+0000 7f4925ffb640 1 -- 172.21.15.16:0/1265368055 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f49140f2500 con 0x7f49281013d0 2024-03-27T06:43:40.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.840+0000 7f492ce2b640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f49180a38a0 0x7f49180a5d60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f491c009f40 tx=0x7f491c005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:40.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:40.843+0000 7f4925ffb640 1 -- 172.21.15.16:0/1265368055 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f491404d020 con 0x7f49281013d0 2024-03-27T06:43:41.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.031+0000 7f492f08f640 1 -- 172.21.15.16:0/1265368055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4928109fe0 con 0x7f49281013d0 2024-03-27T06:43:41.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.032+0000 7f4925ffb640 1 -- 172.21.15.16:0/1265368055 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22982 (secure 0 0 0) 0x7f49140bec80 con 0x7f49281013d0 2024-03-27T06:43:41.027 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:43:41.027 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:43:33.780855+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3655179049":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3519122217":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6800/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/212881673":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/361067788":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:43:41.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 -- 172.21.15.16:0/1265368055 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f49180a38a0 msgr2=0x7f49180a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f49180a38a0 0x7f49180a5d60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f491c009f40 tx=0x7f491c005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 -- 172.21.15.16:0/1265368055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49281013d0 msgr2=0x7f4928198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49281013d0 0x7f4928198a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4914009900 tx=0x7f4914004c30 comp rx=0 tx=0).stop 2024-03-27T06:43:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 -- 172.21.15.16:0/1265368055 shutdown_connections 2024-03-27T06:43:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f49180a38a0 0x7f49180a5d60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928106a20 0x7f492819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49281013d0 0x7f4928198a30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 --2- 172.21.15.16:0/1265368055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4928100a10 0x7f49281984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 -- 172.21.15.16:0/1265368055 >> 172.21.15.16:0/1265368055 conn(0x7f4928077af0 msgr2=0x7f492806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 -- 172.21.15.16:0/1265368055 shutdown_connections 2024-03-27T06:43:41.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.034+0000 7f492f08f640 1 -- 172.21.15.16:0/1265368055 wait complete. 2024-03-27T06:43:41.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:40 smithi016 ceph-mon[31908]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:41.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:40 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/4235664240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:41.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:40 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2098695264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:41.039 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:40 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/549991188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:41.041 DEBUG:tasks.fwd_scrub:Yielding 2024-03-27T06:43:41.041 INFO:teuthology.run_tasks:Running task exec... 2024-03-27T06:43:41.051 INFO:teuthology.task.exec:Executing custom commands... 2024-03-27T06:43:41.051 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-03-27T06:43:41.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-27T06:43:41.051 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-27T06:43:41.052 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-27T06:43:41.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:43:41.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-27T06:43:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7f66592c0640 1 -- 172.21.15.16:0/1882204451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6654101660 msgr2=0x7f665410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7f66592c0640 1 --2- 172.21.15.16:0/1882204451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6654101660 0x7f665410d880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f664c009cd0 tx=0x7f664c02c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7f66592c0640 1 -- 172.21.15.16:0/1882204451 shutdown_connections 2024-03-27T06:43:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7f66592c0640 1 --2- 172.21.15.16:0/1882204451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6654101660 0x7f665410d880 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7f66592c0640 1 --2- 172.21.15.16:0/1882204451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6654100ca0 0x7f6654101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7f66592c0640 1 --2- 172.21.15.16:0/1882204451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66541059f0 0x7f6654105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7f66592c0640 1 -- 172.21.15.16:0/1882204451 >> 172.21.15.16:0/1882204451 conn(0x7f66540778a0 msgr2=0x7f6654077cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7f66592c0640 1 -- 172.21.15.16:0/1882204451 shutdown_connections 2024-03-27T06:43:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7f66592c0640 1 -- 172.21.15.16:0/1882204451 wait complete. 2024-03-27T06:43:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7fbb6582a640 1 -- 172.21.15.16:0/3716061530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb601013b0 msgr2=0x7fbb6010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7fbb6582a640 1 --2- 172.21.15.16:0/3716061530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb601013b0 0x7fbb6010d870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009ca0 tx=0x7fbb5802c180 comp rx=0 tx=0).stop 2024-03-27T06:43:41.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7fbb6582a640 1 -- 172.21.15.16:0/3716061530 shutdown_connections 2024-03-27T06:43:41.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7fbb6582a640 1 --2- 172.21.15.16:0/3716061530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb601013b0 0x7fbb6010d870 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7fbb6582a640 1 --2- 172.21.15.16:0/3716061530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb60100a10 0x7fbb60100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7fbb6582a640 1 --2- 172.21.15.16:0/3716061530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb601058f0 0x7fbb60105cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7fbb6582a640 1 -- 172.21.15.16:0/3716061530 >> 172.21.15.16:0/3716061530 conn(0x7fbb60077ae0 msgr2=0x7fbb60077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7fbb6582a640 1 -- 172.21.15.16:0/3716061530 shutdown_connections 2024-03-27T06:43:41.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7fbb6582a640 1 -- 172.21.15.16:0/3716061530 wait complete. 2024-03-27T06:43:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7f66592c0640 1 Processor -- start 2024-03-27T06:43:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.164+0000 7f66592c0640 1 -- start start 2024-03-27T06:43:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f66592c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654100ca0 0x7f66541026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f66592c0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6654101660 0x7f6654102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f66592c0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66541059f0 0x7f6654103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f66592c0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6654110ff0 con 0x7f6654100ca0 2024-03-27T06:43:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f66592c0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f6654110e70 con 0x7f6654101660 2024-03-27T06:43:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f66592c0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6654111170 con 0x7f66541059f0 2024-03-27T06:43:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb6582a640 1 Processor -- start 2024-03-27T06:43:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654100ca0 0x7f66541026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654100ca0 0x7f66541026c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51308/0 (socket says 172.21.15.16:51308) 2024-03-27T06:43:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f6652ffd640 1 -- 172.21.15.16:0/265364538 learned_addr learned my addr 172.21.15.16:0/265364538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f66537fe640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66541059f0 0x7f6654103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f664bfff640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6654101660 0x7f6654102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb6582a640 1 -- start start 2024-03-27T06:43:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f6652ffd640 1 -- 172.21.15.16:0/265364538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66541059f0 msgr2=0x7f6654103140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f6652ffd640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66541059f0 0x7f6654103140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f6652ffd640 1 -- 172.21.15.16:0/265364538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6654101660 msgr2=0x7f6654102c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f6652ffd640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6654101660 0x7f6654102c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f6652ffd640 1 -- 172.21.15.16:0/265364538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66541a69a0 con 0x7f6654100ca0 2024-03-27T06:43:41.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb6582a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100a10 0x7fbb6019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f664bfff640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6654101660 0x7f6654102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb6582a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb601013b0 0x7fbb6019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb6582a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb601058f0 0x7fbb6019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb6582a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60110f00 con 0x7fbb601013b0 2024-03-27T06:43:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb6582a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60110d80 con 0x7fbb60100a10 2024-03-27T06:43:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb6582a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60111080 con 0x7fbb601058f0 2024-03-27T06:43:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb5e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb601013b0 0x7fbb6019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100a10 0x7fbb6019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb5f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb601058f0 0x7fbb6019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb5e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb601013b0 0x7fbb6019ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51320/0 (socket says 172.21.15.16:51320) 2024-03-27T06:43:41.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7fbb5e7fc640 1 -- 172.21.15.16:0/859487887 learned_addr learned my addr 172.21.15.16:0/859487887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.165+0000 7f6652ffd640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654100ca0 0x7f66541026c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f664400e9f0 tx=0x7f664400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7f6650ff9640 1 -- 172.21.15.16:0/265364538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664400cd80 con 0x7f6654100ca0 2024-03-27T06:43:41.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb5e7fc640 1 -- 172.21.15.16:0/859487887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb601058f0 msgr2=0x7fbb6019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb5e7fc640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb601058f0 0x7fbb6019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb5e7fc640 1 -- 172.21.15.16:0/859487887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100a10 msgr2=0x7fbb6019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb5e7fc640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100a10 0x7fbb6019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb5e7fc640 1 -- 172.21.15.16:0/859487887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6019f730 con 0x7fbb601013b0 2024-03-27T06:43:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb5f7fe640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb601058f0 0x7fbb6019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb5effd640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100a10 0x7fbb6019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7f66592c0640 1 -- 172.21.15.16:0/265364538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66541a6c90 con 0x7f6654100ca0 2024-03-27T06:43:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7f6650ff9640 1 -- 172.21.15.16:0/265364538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6644010930 con 0x7f6654100ca0 2024-03-27T06:43:41.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7f6650ff9640 1 -- 172.21.15.16:0/265364538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644018b00 con 0x7f6654100ca0 2024-03-27T06:43:41.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7f66592c0640 1 -- 172.21.15.16:0/265364538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66541a7180 con 0x7f6654100ca0 2024-03-27T06:43:41.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb5e7fc640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb601013b0 0x7fbb6019ac80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400e9f0 tx=0x7fbb5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb64828640 1 -- 172.21.15.16:0/859487887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5400cd80 con 0x7fbb601013b0 2024-03-27T06:43:41.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb64828640 1 -- 172.21.15.16:0/859487887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb54010930 con 0x7fbb601013b0 2024-03-27T06:43:41.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb6582a640 1 -- 172.21.15.16:0/859487887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6019fa20 con 0x7fbb601013b0 2024-03-27T06:43:41.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.166+0000 7fbb64828640 1 -- 172.21.15.16:0/859487887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54018b00 con 0x7fbb601013b0 2024-03-27T06:43:41.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.167+0000 7fbb6582a640 1 -- 172.21.15.16:0/859487887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb601a72b0 con 0x7fbb601013b0 2024-03-27T06:43:41.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.167+0000 7f66592c0640 1 -- 172.21.15.16:0/265364538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665404f8b0 con 0x7f6654100ca0 2024-03-27T06:43:41.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.168+0000 7f6650ff9640 1 -- 172.21.15.16:0/265364538 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7f6644018ca0 con 0x7f6654100ca0 2024-03-27T06:43:41.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.169+0000 7fbb64828640 1 -- 172.21.15.16:0/859487887 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7fbb54018ca0 con 0x7fbb601013b0 2024-03-27T06:43:41.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.169+0000 7f6650ff9640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f66400a3940 0x7f66400a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.169+0000 7f6650ff9640 1 -- 172.21.15.16:0/265364538 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f66440d0690 con 0x7f6654100ca0 2024-03-27T06:43:41.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.169+0000 7f664bfff640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f66400a3940 0x7f66400a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.169+0000 7fbb64828640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbb480a3920 0x7fbb480a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.169+0000 7fbb5effd640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbb480a3920 0x7fbb480a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.169+0000 7fbb64828640 1 -- 172.21.15.16:0/859487887 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbb540d06a0 con 0x7fbb601013b0 2024-03-27T06:43:41.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7f664bfff640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f66400a3940 0x7f66400a5e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f663c0095f0 tx=0x7f663c005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb68412740 1 Processor -- start 2024-03-27T06:43:41.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb5effd640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbb480a3920 0x7fbb480a5de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009f40 tx=0x7fbb50005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb68412740 1 -- start start 2024-03-27T06:43:41.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb68412740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84811e50 0x55ae84812230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb68412740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae8481ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb68412740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae8481b350 0x55ae8481d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb68412740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae8471cc20 con 0x55ae8481b350 2024-03-27T06:43:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb68412740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55ae84775430 con 0x55ae84811e50 2024-03-27T06:43:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb68412740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ae8471d900 con 0x55ae84812770 2024-03-27T06:43:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84811e50 0x55ae84812230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb5e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae8481ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84811e50 0x55ae84812230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:43172/0 (socket says 172.21.15.16:43172) 2024-03-27T06:43:41.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb5f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae8481b350 0x55ae8481d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.170+0000 7fbb5effd640 1 -- 172.21.15.16:0/3122591833 learned_addr learned my addr 172.21.15.16:0/3122591833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.171+0000 7fbb5effd640 1 -- 172.21.15.16:0/3122591833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 msgr2=0x55ae8481ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.171+0000 7fbb5effd640 1 --2- 172.21.15.16:0/3122591833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae8481ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.171+0000 7fbb5effd640 1 -- 172.21.15.16:0/3122591833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae8481b350 msgr2=0x55ae8481d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.171+0000 7fbb5effd640 1 --2- 172.21.15.16:0/3122591833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae8481b350 0x55ae8481d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.171+0000 7fbb5effd640 1 -- 172.21.15.16:0/3122591833 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae84770180 con 0x55ae84811e50 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.171+0000 7fbb5e7fc640 1 --2- 172.21.15.16:0/3122591833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae8481ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.171+0000 7fbb5effd640 1 --2- 172.21.15.16:0/3122591833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84811e50 0x55ae84812230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb50004230 tx=0x7fbb50014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2249f92843e22408 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.171+0000 7fbb45ffb640 1 -- 172.21.15.16:0/3122591833 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50003e30 con 0x55ae84811e50 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.171+0000 7fbb45ffb640 1 -- 172.21.15.16:0/3122591833 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb5000f980 con 0x55ae84811e50 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.171+0000 7fbb45ffb640 1 -- 172.21.15.16:0/3122591833 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5001eb40 con 0x55ae84811e50 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.172+0000 7fbb68412740 1 -- 172.21.15.16:0/3122591833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84811e50 msgr2=0x55ae84812230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.172+0000 7fbb68412740 1 --2- 172.21.15.16:0/3122591833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84811e50 0x55ae84812230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb50004230 tx=0x7fbb50014000 comp rx=0 tx=0).stop 2024-03-27T06:43:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.172+0000 7fbb68412740 1 -- 172.21.15.16:0/3122591833 shutdown_connections 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.172+0000 7fbb68412740 1 --2- 172.21.15.16:0/3122591833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae8481b350 0x55ae8481d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.172+0000 7fbb68412740 1 --2- 172.21.15.16:0/3122591833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae8481ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.172+0000 7fbb68412740 1 --2- 172.21.15.16:0/3122591833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84811e50 0x55ae84812230 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.172+0000 7fbb68412740 1 -- 172.21.15.16:0/3122591833 >> 172.21.15.16:0/3122591833 conn(0x55ae8480c580 msgr2=0x55ae8480c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.172+0000 7fbb68412740 1 -- 172.21.15.16:0/3122591833 shutdown_connections 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.172+0000 7fbb68412740 1 -- 172.21.15.16:0/3122591833 wait complete. 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.172+0000 7fbb68412740 1 -- start start 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.172+0000 7fbb68412740 1 Processor -- start 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb68412740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae8481acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb68412740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae84818050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb68412740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae8481b350 0x55ae848185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb68412740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae84730cc0 con 0x55ae84811e50 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb68412740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55ae847713e0 con 0x55ae8481b350 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb68412740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ae8481e140 con 0x55ae84812770 2024-03-27T06:43:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb68412740 10 client.0 _ll_register_callbacks cb 0x55ae8476bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae8481acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae8481acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51342/0 (socket says 172.21.15.16:51342) 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb5effd640 1 -- 172.21.15.16:0/3320726430 learned_addr learned my addr 172.21.15.16:0/3320726430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb5e7fc640 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae84818050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb5f7fe640 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae8481b350 0x55ae848185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb5effd640 1 -- 172.21.15.16:0/3320726430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 msgr2=0x55ae84818050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb5effd640 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae84818050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb5effd640 1 -- 172.21.15.16:0/3320726430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae8481b350 msgr2=0x55ae848185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb5effd640 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae8481b350 0x55ae848185b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb5effd640 1 -- 172.21.15.16:0/3320726430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae84737ef0 con 0x55ae84811e50 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb5effd640 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae8481acc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbb50004200 tx=0x7fbb5001e600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26f96950e538079 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.173+0000 7fbb3effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb3effd640 1 -- 172.21.15.16:0/3320726430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5000f710 con 0x55ae84811e50 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb3effd640 1 -- 172.21.15.16:0/3320726430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbb5000fb30 con 0x55ae84811e50 2024-03-27T06:43:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb68412740 1 -- 172.21.15.16:0/3320726430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb5000f980 con 0x55ae84811e50 2024-03-27T06:43:41.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7f6650ff9640 1 -- 172.21.15.16:0/265364538 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f664409c6b0 con 0x7f6654100ca0 2024-03-27T06:43:41.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb3effd640 1 -- 172.21.15.16:0/3320726430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb5000fd30 con 0x55ae84811e50 2024-03-27T06:43:41.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb68412740 10 client.15558 fetch_fsmap learned FSMap version 25 2024-03-27T06:43:41.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb68412740 1 -- 172.21.15.16:0/3320726430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae845517e0 con 0x55ae84811e50 2024-03-27T06:43:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb3effd640 1 -- 172.21.15.16:0/3320726430 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb5000f980 con 0x55ae84811e50 2024-03-27T06:43:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb3effd640 1 -- 172.21.15.16:0/3320726430 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbb50031ae0 con 0x55ae84811e50 2024-03-27T06:43:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb68412740 10 client.15558 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:43:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb68412740 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:43:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb68412740 20 client.15558 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb68412740 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae8482b160 0x55ae8482d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb68412740 4 client.15558 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:43:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.175+0000 7fbb68412740 1 -- 172.21.15.16:0/3320726430 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae84782970 con 0x55ae8482b160 2024-03-27T06:43:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.176+0000 7fbb5e7fc640 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae8482b160 0x55ae8482d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.176+0000 7fbb5e7fc640 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae8482b160 0x55ae8482d550 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3273f4d5ebfb9dc server_cookie=f5f7e374f3106d42 in_seq=0 out_seq=0 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.176+0000 7fbb3effd640 10 client.15558 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb3effd640 1 -- 172.21.15.16:0/3320726430 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ae84782970 con 0x55ae8482b160 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb3effd640 10 client.15558 handle_command_reply: tid=0 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 1 client.15558 shutdown 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 1 -- 172.21.15.16:0/3320726430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 msgr2=0x55ae8481acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae8481acc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbb50004200 tx=0x7fbb5001e600 comp rx=0 tx=0).stop 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 1 -- 172.21.15.16:0/3320726430 shutdown_connections 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae8482b160 0x55ae8482d550 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae8481b350 0x55ae848185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae84818050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb3effd640 0 client.15558 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 1 --2- 172.21.15.16:0/3320726430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae8481acc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 1 -- 172.21.15.16:0/3320726430 >> 172.21.15.16:0/3320726430 conn(0x55ae8480c580 msgr2=0x55ae8480e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 1 -- 172.21.15.16:0/3320726430 shutdown_connections 2024-03-27T06:43:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 1 -- 172.21.15.16:0/3320726430 wait complete. 2024-03-27T06:43:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.177+0000 7fbb68412740 20 client.15558 trim_cache size 0 max 16384 2024-03-27T06:43:41.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb68412740 1 Processor -- start 2024-03-27T06:43:41.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb68412740 1 -- start start 2024-03-27T06:43:41.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb68412740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84811e50 0x55ae84826d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb68412740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84812770 0x55ae84832b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb68412740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae8481b350 0x55ae8481a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb68412740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601afab0 con 0x55ae8481b350 2024-03-27T06:43:41.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb68412740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601b15f0 con 0x55ae84812770 2024-03-27T06:43:41.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb68412740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601b0180 con 0x55ae84811e50 2024-03-27T06:43:41.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb5f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae8481b350 0x55ae8481a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84811e50 0x55ae84826d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb5f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae8481b350 0x55ae8481a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51358/0 (socket says 172.21.15.16:51358) 2024-03-27T06:43:41.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb5e7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84812770 0x55ae84832b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb5f7fe640 1 -- 172.21.15.16:0/3329677283 learned_addr learned my addr 172.21.15.16:0/3329677283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84811e50 0x55ae84826d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:46724/0 (socket says 172.21.15.16:46724) 2024-03-27T06:43:41.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb5effd640 1 -- 172.21.15.16:0/3329677283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84812770 msgr2=0x55ae84832b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb5effd640 1 --2- 172.21.15.16:0/3329677283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84812770 0x55ae84832b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb5effd640 1 -- 172.21.15.16:0/3329677283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae8481b350 msgr2=0x55ae8481a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb5effd640 1 --2- 172.21.15.16:0/3329677283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae8481b350 0x55ae8481a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.195+0000 7fbb5effd640 1 -- 172.21.15.16:0/3329677283 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae848125a0 con 0x55ae84811e50 2024-03-27T06:43:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.196+0000 7fbb5effd640 1 --2- 172.21.15.16:0/3329677283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84811e50 0x55ae84826d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb50031f30 tx=0x7fbb500266b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ca0c2e66a359f54 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.196+0000 7fbb457fa640 1 -- 172.21.15.16:0/3329677283 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5000f710 con 0x55ae84811e50 2024-03-27T06:43:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.196+0000 7fbb457fa640 1 -- 172.21.15.16:0/3329677283 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb5000f8b0 con 0x55ae84811e50 2024-03-27T06:43:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.196+0000 7fbb457fa640 1 -- 172.21.15.16:0/3329677283 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5001ead0 con 0x55ae84811e50 2024-03-27T06:43:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 -- 172.21.15.16:0/3329677283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84811e50 msgr2=0x55ae84826d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 --2- 172.21.15.16:0/3329677283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84811e50 0x55ae84826d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb50031f30 tx=0x7fbb500266b0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 -- 172.21.15.16:0/3329677283 shutdown_connections 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 --2- 172.21.15.16:0/3329677283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae8481b350 0x55ae8481a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 --2- 172.21.15.16:0/3329677283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae84812770 0x55ae84832b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 --2- 172.21.15.16:0/3329677283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84811e50 0x55ae84826d60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 -- 172.21.15.16:0/3329677283 >> 172.21.15.16:0/3329677283 conn(0x55ae8480c580 msgr2=0x55ae84817080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 -- 172.21.15.16:0/3329677283 shutdown_connections 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 -- 172.21.15.16:0/3329677283 wait complete. 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 -- start start 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 Processor -- start 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae84823390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae84820360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae8481b350 0x55ae848208a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601b0f20 con 0x55ae84811e50 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601b0850 con 0x55ae8481b350 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601ada90 con 0x55ae84812770 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb68412740 10 client.0 _ll_register_callbacks cb 0x7fbb50005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.197+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae84823390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb5e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae84820360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae84823390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51370/0 (socket says 172.21.15.16:51370) 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb5effd640 1 -- 172.21.15.16:0/1488347694 learned_addr learned my addr 172.21.15.16:0/1488347694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb5f7fe640 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae8481b350 0x55ae848208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb5effd640 1 -- 172.21.15.16:0/1488347694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 msgr2=0x55ae84820360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb5effd640 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae84820360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb5effd640 1 -- 172.21.15.16:0/1488347694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae8481b350 msgr2=0x55ae848208a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb5effd640 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae8481b350 0x55ae848208a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb5effd640 1 -- 172.21.15.16:0/1488347694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae845b1f70 con 0x55ae84811e50 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb5e7fc640 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae84820360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb5effd640 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae84823390 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbb50004230 tx=0x7fbb50011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cc87c6e29b7114 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb3effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb3effd640 1 -- 172.21.15.16:0/1488347694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5003e070 con 0x55ae84811e50 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb68412740 1 -- 172.21.15.16:0/1488347694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb5000f8b0 con 0x55ae84811e50 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb3effd640 1 -- 172.21.15.16:0/1488347694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbb50005030 con 0x55ae84811e50 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb3effd640 1 -- 172.21.15.16:0/1488347694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb5000f710 con 0x55ae84811e50 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb68412740 10 client.15564 fetch_fsmap learned FSMap version 25 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.198+0000 7fbb68412740 1 -- 172.21.15.16:0/1488347694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae84449170 con 0x55ae84811e50 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.199+0000 7fbb3effd640 1 -- 172.21.15.16:0/1488347694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb5000f8b0 con 0x55ae84811e50 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.199+0000 7fbb3effd640 1 -- 172.21.15.16:0/1488347694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbb50004630 con 0x55ae84811e50 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.199+0000 7fbb68412740 10 client.15564 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:43:41.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.199+0000 7fbb68412740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:43:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.199+0000 7fbb68412740 20 client.15564 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.199+0000 7fbb68412740 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae84891eb0 0x55ae848942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.199+0000 7fbb68412740 4 client.15564 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-27T06:43:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.199+0000 7fbb68412740 1 -- 172.21.15.16:0/1488347694 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ae8477ccc0 con 0x55ae84891eb0 2024-03-27T06:43:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.199+0000 7fbb5e7fc640 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae84891eb0 0x55ae848942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.199+0000 7fbb3effd640 10 client.15564 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.199+0000 7fbb5e7fc640 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae84891eb0 0x55ae848942d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf4c6040017a549 server_cookie=d2627ffa37f99634 in_seq=0 out_seq=0 2024-03-27T06:43:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.200+0000 7fbb3effd640 1 -- 172.21.15.16:0/1488347694 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ae8477ccc0 con 0x55ae84891eb0 2024-03-27T06:43:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.200+0000 7fbb3effd640 10 client.15564 handle_command_reply: tid=0 2024-03-27T06:43:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.200+0000 7fbb68412740 1 client.15564 shutdown 2024-03-27T06:43:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb68412740 1 -- 172.21.15.16:0/1488347694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 msgr2=0x55ae84823390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb68412740 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae84823390 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbb50004230 tx=0x7fbb50011d20 comp rx=0 tx=0).stop 2024-03-27T06:43:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb68412740 1 -- 172.21.15.16:0/1488347694 shutdown_connections 2024-03-27T06:43:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb68412740 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae84891eb0 0x55ae848942d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb68412740 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae8481b350 0x55ae848208a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb68412740 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae84812770 0x55ae84820360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb68412740 1 --2- 172.21.15.16:0/1488347694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae84811e50 0x55ae84823390 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb3effd640 0 client.15564 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb68412740 1 -- 172.21.15.16:0/1488347694 >> 172.21.15.16:0/1488347694 conn(0x55ae8480c580 msgr2=0x55ae848167f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb68412740 1 -- 172.21.15.16:0/1488347694 shutdown_connections 2024-03-27T06:43:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb68412740 1 -- 172.21.15.16:0/1488347694 wait complete. 2024-03-27T06:43:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb68412740 20 client.15564 trim_cache size 0 max 16384 2024-03-27T06:43:41.202 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:43:41.203 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-03-27T06:43:41.203 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "fb4110ef-a146-4874-937e-c9d570087b3f", 2024-03-27T06:43:41.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-03-27T06:43:41.203 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:43:41.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb6582a640 1 -- 172.21.15.16:0/859487887 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbb480a3920 msgr2=0x7fbb480a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb6582a640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbb480a3920 0x7fbb480a5de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009f40 tx=0x7fbb50005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb6582a640 1 -- 172.21.15.16:0/859487887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb601013b0 msgr2=0x7fbb6019ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb6582a640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb601013b0 0x7fbb6019ac80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400e9f0 tx=0x7fbb5400eeb0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb6582a640 1 -- 172.21.15.16:0/859487887 shutdown_connections 2024-03-27T06:43:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb6582a640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbb480a3920 0x7fbb480a5de0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb6582a640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb601058f0 0x7fbb6019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb6582a640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb601013b0 0x7fbb6019ac80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb6582a640 1 --2- 172.21.15.16:0/859487887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb60100a10 0x7fbb6019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.201+0000 7fbb6582a640 1 -- 172.21.15.16:0/859487887 >> 172.21.15.16:0/859487887 conn(0x7fbb60077ae0 msgr2=0x7fbb60069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.202+0000 7fbb6582a640 1 -- 172.21.15.16:0/859487887 shutdown_connections 2024-03-27T06:43:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.202+0000 7fbb6582a640 1 -- 172.21.15.16:0/859487887 wait complete. 2024-03-27T06:43:41.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T06:43:41.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T06:43:41.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:40 smithi156 ceph-mon[36261]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:41.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:40 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/4235664240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:41.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:40 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2098695264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:41.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:40 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/549991188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.318+0000 7f5833702640 1 -- 172.21.15.16:0/744464541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f582c1069d0 msgr2=0x7f582c106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.318+0000 7f5833702640 1 --2- 172.21.15.16:0/744464541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f582c1069d0 0x7f582c106db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5828009890 tx=0x7f582802bb70 comp rx=0 tx=0).stop 2024-03-27T06:43:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.318+0000 7f5833702640 1 -- 172.21.15.16:0/744464541 shutdown_connections 2024-03-27T06:43:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.318+0000 7f5833702640 1 --2- 172.21.15.16:0/744464541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c101380 0x7f582c10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.318+0000 7f5833702640 1 --2- 172.21.15.16:0/744464541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f582c1009c0 0x7f582c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.318+0000 7f5833702640 1 --2- 172.21.15.16:0/744464541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f582c1069d0 0x7f582c106db0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.318+0000 7f5833702640 1 -- 172.21.15.16:0/744464541 >> 172.21.15.16:0/744464541 conn(0x7f582c077b50 msgr2=0x7f582c077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.318+0000 7f5833702640 1 -- 172.21.15.16:0/744464541 shutdown_connections 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.318+0000 7f5833702640 1 -- 172.21.15.16:0/744464541 wait complete. 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5833702640 1 Processor -- start 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5833702640 1 -- start start 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5833702640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f582c1009c0 0x7f582c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5833702640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c101380 0x7f582c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5833702640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f582c1069d0 0x7f582c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5833702640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f582c110ed0 con 0x7f582c101380 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5833702640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f582c110d50 con 0x7f582c1009c0 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5833702640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f582c111050 con 0x7f582c1069d0 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5830c9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c101380 0x7f582c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5830c9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c101380 0x7f582c19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51388/0 (socket says 172.21.15.16:51388) 2024-03-27T06:43:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5830c9d640 1 -- 172.21.15.16:0/2310360123 learned_addr learned my addr 172.21.15.16:0/2310360123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f583149e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f582c1009c0 0x7f582c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.319+0000 7f5831c9f640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f582c1069d0 0x7f582c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f5830c9d640 1 -- 172.21.15.16:0/2310360123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f582c1069d0 msgr2=0x7f582c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f5830c9d640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f582c1069d0 0x7f582c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f5830c9d640 1 -- 172.21.15.16:0/2310360123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f582c1009c0 msgr2=0x7f582c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f5830c9d640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f582c1009c0 0x7f582c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f5830c9d640 1 -- 172.21.15.16:0/2310360123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582c19f6c0 con 0x7f582c101380 2024-03-27T06:43:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f583149e640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f582c1009c0 0x7f582c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f5830c9d640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c101380 0x7f582c19ac60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f581c00e9f0 tx=0x7f581c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f58227fc640 1 -- 172.21.15.16:0/2310360123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c00cd80 con 0x7f582c101380 2024-03-27T06:43:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f58227fc640 1 -- 172.21.15.16:0/2310360123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f581c010930 con 0x7f582c101380 2024-03-27T06:43:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f5833702640 1 -- 172.21.15.16:0/2310360123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582c19f9b0 con 0x7f582c101380 2024-03-27T06:43:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f58227fc640 1 -- 172.21.15.16:0/2310360123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c018b00 con 0x7f582c101380 2024-03-27T06:43:41.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.320+0000 7f5833702640 1 -- 172.21.15.16:0/2310360123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582c1a7240 con 0x7f582c101380 2024-03-27T06:43:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.322+0000 7f58227fc640 1 -- 172.21.15.16:0/2310360123 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7f581c018ca0 con 0x7f582c101380 2024-03-27T06:43:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.322+0000 7f58363c6740 1 Processor -- start 2024-03-27T06:43:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f58363c6740 1 -- start start 2024-03-27T06:43:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f58363c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544250 0x564be6544630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f58363c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544b70 0x564be654d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f58363c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be654fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f58363c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564be64a8860 con 0x564be6544250 2024-03-27T06:43:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f58363c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x564be64508d0 con 0x564be6544b70 2024-03-27T06:43:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f58363c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564be64a4810 con 0x564be654d750 2024-03-27T06:43:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f5831c9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544250 0x564be6544630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f5831c9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544250 0x564be6544630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51404/0 (socket says 172.21.15.16:51404) 2024-03-27T06:43:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f5830c9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be654fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f5831c9f640 1 -- 172.21.15.16:0/2027123662 learned_addr learned my addr 172.21.15.16:0/2027123662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f583149e640 1 --2- 172.21.15.16:0/2027123662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544b70 0x564be654d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f58227fc640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f58180a38a0 0x7f58180a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f5831c9f640 1 -- 172.21.15.16:0/2027123662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 msgr2=0x564be654fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f5831c9f640 1 --2- 172.21.15.16:0/2027123662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be654fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f5831c9f640 1 -- 172.21.15.16:0/2027123662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544b70 msgr2=0x564be654d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f5831c9f640 1 --2- 172.21.15.16:0/2027123662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544b70 0x564be654d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f5831c9f640 1 -- 172.21.15.16:0/2027123662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be645b120 con 0x564be6544250 2024-03-27T06:43:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f5830c9d640 1 --2- 172.21.15.16:0/2027123662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be654fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f583149e640 1 --2- 172.21.15.16:0/2027123662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544b70 0x564be654d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f58227fc640 1 -- 172.21.15.16:0/2310360123 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f581c0d0710 con 0x7f582c101380 2024-03-27T06:43:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.323+0000 7f5831c9f640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f58180a38a0 0x7f58180a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.324+0000 7f5831c9f640 1 --2- 172.21.15.16:0/2027123662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544250 0x564be6544630 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f582400e520 tx=0x7f582400e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8cd4007133413c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.324+0000 7f58177fe640 1 -- 172.21.15.16:0/2027123662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582400d580 con 0x564be6544250 2024-03-27T06:43:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.324+0000 7f58177fe640 1 -- 172.21.15.16:0/2027123662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f582400dc20 con 0x564be6544250 2024-03-27T06:43:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.324+0000 7f58177fe640 1 -- 172.21.15.16:0/2027123662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582401ba50 con 0x564be6544250 2024-03-27T06:43:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.324+0000 7f58363c6740 1 -- 172.21.15.16:0/2027123662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544250 msgr2=0x564be6544630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.324+0000 7f58363c6740 1 --2- 172.21.15.16:0/2027123662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544250 0x564be6544630 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f582400e520 tx=0x7f582400e9e0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.324+0000 7f5831c9f640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f58180a38a0 0x7f58180a5d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5824023000 tx=0x7f5824024000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 -- 172.21.15.16:0/2027123662 shutdown_connections 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 --2- 172.21.15.16:0/2027123662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be654fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 --2- 172.21.15.16:0/2027123662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544b70 0x564be654d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 --2- 172.21.15.16:0/2027123662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544250 0x564be6544630 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 -- 172.21.15.16:0/2027123662 >> 172.21.15.16:0/2027123662 conn(0x564be653e980 msgr2=0x564be653ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 -- 172.21.15.16:0/2027123662 shutdown_connections 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 -- 172.21.15.16:0/2027123662 wait complete. 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 -- start start 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 Processor -- start 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be654a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be6544b70 0x564be654ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be654d750 0x564be654b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564be6550540 con 0x564be654d750 2024-03-27T06:43:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x564be6463cd0 con 0x564be6544250 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564be65506c0 con 0x564be6544b70 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.325+0000 7f58363c6740 10 client.0 _ll_register_callbacks cb 0x564be6455ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5831c9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be654d750 0x564be654b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5831c9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be654d750 0x564be654b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51406/0 (socket says 172.21.15.16:51406) 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5831c9f640 1 -- 172.21.15.16:0/2670342216 learned_addr learned my addr 172.21.15.16:0/2670342216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f583149e640 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be654a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5830c9d640 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be6544b70 0x564be654ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5831c9f640 1 -- 172.21.15.16:0/2670342216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be6544b70 msgr2=0x564be654ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5831c9f640 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be6544b70 0x564be654ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5831c9f640 1 -- 172.21.15.16:0/2670342216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 msgr2=0x564be654a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5831c9f640 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be654a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5831c9f640 1 -- 172.21.15.16:0/2670342216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be62867d0 con 0x564be654d750 2024-03-27T06:43:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f583149e640 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be654a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5831c9f640 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be654d750 0x564be654b1b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5824009850 tx=0x7f5824009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e66ced765927ed1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5814ff9640 1 -- 172.21.15.16:0/2670342216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824036060 con 0x564be654d750 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5814ff9640 1 -- 172.21.15.16:0/2670342216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f582401b7d0 con 0x564be654d750 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f58363c6740 1 -- 172.21.15.16:0/2670342216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582400dc20 con 0x564be654d750 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f5814ff9640 1 -- 172.21.15.16:0/2670342216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f582401ba10 con 0x564be654d750 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f58363c6740 10 client.15576 fetch_fsmap learned FSMap version 25 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.326+0000 7f58363c6740 1 -- 172.21.15.16:0/2670342216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564be61bc3f0 con 0x564be654d750 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.327+0000 7f5814ff9640 1 -- 172.21.15.16:0/2670342216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f582400dc20 con 0x564be654d750 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.327+0000 7f5814ff9640 1 -- 172.21.15.16:0/2670342216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f582401a030 con 0x564be654d750 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.327+0000 7f58363c6740 10 client.15576 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.327+0000 7f58363c6740 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.327+0000 7f58363c6740 20 client.15576 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.327+0000 7f58363c6740 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564be655d910 0x564be655fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.327+0000 7f58363c6740 4 client.15576 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.327+0000 7f58363c6740 1 -- 172.21.15.16:0/2670342216 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564be64b5da0 con 0x564be655d910 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.327+0000 7f583149e640 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564be655d910 0x564be655fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.327+0000 7f583149e640 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564be655d910 0x564be655fd00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea4030300f3a404b server_cookie=f21a6e79022fa473 in_seq=0 out_seq=0 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.327+0000 7f5814ff9640 10 client.15576 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f5814ff9640 1 -- 172.21.15.16:0/2670342216 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564be64b5da0 con 0x564be655d910 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f5814ff9640 10 client.15576 handle_command_reply: tid=0 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 1 client.15576 shutdown 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 1 -- 172.21.15.16:0/2670342216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be654d750 msgr2=0x564be654b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be654d750 0x564be654b1b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5824009850 tx=0x7f5824009cc0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 1 -- 172.21.15.16:0/2670342216 shutdown_connections 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564be655d910 0x564be655fd00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be654d750 0x564be654b1b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f5814ff9640 0 client.15576 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be6544b70 0x564be654ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 1 --2- 172.21.15.16:0/2670342216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be654a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 1 -- 172.21.15.16:0/2670342216 >> 172.21.15.16:0/2670342216 conn(0x564be653e980 msgr2=0x564be65406d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 1 -- 172.21.15.16:0/2670342216 shutdown_connections 2024-03-27T06:43:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 1 -- 172.21.15.16:0/2670342216 wait complete. 2024-03-27T06:43:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.328+0000 7f58363c6740 20 client.15576 trim_cache size 0 max 16384 2024-03-27T06:43:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f58363c6740 1 Processor -- start 2024-03-27T06:43:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f58363c6740 1 -- start start 2024-03-27T06:43:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f58363c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be654ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f58363c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be65592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f58363c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be6549ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f58363c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1afaa0 con 0x564be6544b70 2024-03-27T06:43:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f58363c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b15e0 con 0x564be6544250 2024-03-27T06:43:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f58363c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b0170 con 0x564be654d750 2024-03-27T06:43:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f5830c9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be65592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f5830c9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be65592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51420/0 (socket says 172.21.15.16:51420) 2024-03-27T06:43:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f5830c9d640 1 -- 172.21.15.16:0/2534074673 learned_addr learned my addr 172.21.15.16:0/2534074673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f583149e640 1 --2- 172.21.15.16:0/2534074673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be654ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f5831c9f640 1 --2- 172.21.15.16:0/2534074673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be6549ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f5830c9d640 1 -- 172.21.15.16:0/2534074673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 msgr2=0x564be6549ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f5830c9d640 1 --2- 172.21.15.16:0/2534074673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be6549ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f5830c9d640 1 -- 172.21.15.16:0/2534074673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 msgr2=0x564be654ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f5830c9d640 1 --2- 172.21.15.16:0/2534074673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be654ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f5830c9d640 1 -- 172.21.15.16:0/2534074673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be617c0f0 con 0x564be6544b70 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f5831c9f640 1 --2- 172.21.15.16:0/2534074673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be6549ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.346+0000 7f583149e640 1 --2- 172.21.15.16:0/2534074673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be654ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.347+0000 7f5830c9d640 1 --2- 172.21.15.16:0/2534074673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be65592e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f581c0ce490 tx=0x7f581c0d6510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393afbc839e51f76 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.347+0000 7f5816ffd640 1 -- 172.21.15.16:0/2534074673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c09c090 con 0x564be6544b70 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.347+0000 7f5816ffd640 1 -- 172.21.15.16:0/2534074673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f581c0a52f0 con 0x564be6544b70 2024-03-27T06:43:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.347+0000 7f5816ffd640 1 -- 172.21.15.16:0/2534074673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c0d0bc0 con 0x564be6544b70 2024-03-27T06:43:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:40 smithi170 ceph-mon[36438]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-27T06:43:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:40 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/4235664240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:43:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:40 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2098695264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:43:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:40 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/549991188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:43:41.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.348+0000 7f58363c6740 1 -- 172.21.15.16:0/2534074673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 msgr2=0x564be65592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.348+0000 7f58363c6740 1 --2- 172.21.15.16:0/2534074673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be65592e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f581c0ce490 tx=0x7f581c0d6510 comp rx=0 tx=0).stop 2024-03-27T06:43:41.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.348+0000 7f58363c6740 1 -- 172.21.15.16:0/2534074673 shutdown_connections 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.348+0000 7f58363c6740 1 --2- 172.21.15.16:0/2534074673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be6549ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.348+0000 7f58363c6740 1 --2- 172.21.15.16:0/2534074673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be65592e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.348+0000 7f58363c6740 1 --2- 172.21.15.16:0/2534074673 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be654ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.348+0000 7f58363c6740 1 -- 172.21.15.16:0/2534074673 >> 172.21.15.16:0/2534074673 conn(0x564be653e980 msgr2=0x564be654af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.348+0000 7f58363c6740 1 -- 172.21.15.16:0/2534074673 shutdown_connections 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.348+0000 7f58363c6740 1 -- 172.21.15.16:0/2534074673 wait complete. 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.348+0000 7f58363c6740 1 -- start start 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.348+0000 7f58363c6740 1 Processor -- start 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f58363c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be65508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f58363c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be6550e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f58363c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be65c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f58363c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b0f10 con 0x564be6544b70 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f58363c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b0840 con 0x564be6544250 2024-03-27T06:43:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f58363c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1ada80 con 0x564be654d750 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f58363c6740 10 client.0 _ll_register_callbacks cb 0x7f5824009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5830c9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be6550e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5830c9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be6550e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51424/0 (socket says 172.21.15.16:51424) 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f583149e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be65508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5830c9d640 1 -- 172.21.15.16:0/3058786891 learned_addr learned my addr 172.21.15.16:0/3058786891 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5831c9f640 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be65c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5830c9d640 1 -- 172.21.15.16:0/3058786891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 msgr2=0x564be65c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5830c9d640 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be65c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5830c9d640 1 -- 172.21.15.16:0/3058786891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 msgr2=0x564be65508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5830c9d640 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be65508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5830c9d640 1 -- 172.21.15.16:0/3058786891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be65516d0 con 0x564be6544b70 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5830c9d640 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be6550e10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f581c0cde20 tx=0x7f581c099070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbf44129920ddaa server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5814ff9640 1 -- 172.21.15.16:0/3058786891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c0b4070 con 0x564be6544b70 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f5814ff9640 1 -- 172.21.15.16:0/3058786891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f581c0cdff0 con 0x564be6544b70 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.349+0000 7f58363c6740 1 -- 172.21.15.16:0/3058786891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f581c0a52f0 con 0x564be6544b70 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f5814ff9640 1 -- 172.21.15.16:0/3058786891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f581c09c090 con 0x564be6544b70 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f58363c6740 10 client.15582 fetch_fsmap learned FSMap version 25 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f58363c6740 1 -- 172.21.15.16:0/3058786891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564be65c3220 con 0x564be6544b70 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f5814ff9640 1 -- 172.21.15.16:0/3058786891 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f581c0a52f0 con 0x564be6544b70 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f5814ff9640 1 -- 172.21.15.16:0/3058786891 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f581c09bb80 con 0x564be6544b70 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f58363c6740 10 client.15582 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f58363c6740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f58363c6740 20 client.15582 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f58363c6740 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564be65c3860 0x564be65c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f58363c6740 4 client.15582 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f58363c6740 1 -- 172.21.15.16:0/3058786891 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564be64b00f0 con 0x564be65c3860 2024-03-27T06:43:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.350+0000 7f583149e640 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564be65c3860 0x564be65c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f5814ff9640 10 client.15582 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f583149e640 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564be65c3860 0x564be65c5c80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b28f1d39d955e51d server_cookie=5e0162cfe1f86726 in_seq=0 out_seq=0 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f5814ff9640 1 -- 172.21.15.16:0/3058786891 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564be64b00f0 con 0x564be65c3860 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f5814ff9640 10 client.15582 handle_command_reply: tid=0 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 1 client.15582 shutdown 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 1 -- 172.21.15.16:0/3058786891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 msgr2=0x564be6550e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be6550e10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f581c0cde20 tx=0x7f581c099070 comp rx=0 tx=0).stop 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 1 -- 172.21.15.16:0/3058786891 shutdown_connections 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564be65c3860 0x564be65c5c80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564be654d750 0x564be65c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564be6544b70 0x564be6550e10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f5814ff9640 0 client.15582 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 1 --2- 172.21.15.16:0/3058786891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564be6544250 0x564be65508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 1 -- 172.21.15.16:0/3058786891 >> 172.21.15.16:0/3058786891 conn(0x564be653e980 msgr2=0x564be654aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 1 -- 172.21.15.16:0/3058786891 shutdown_connections 2024-03-27T06:43:41.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 1 -- 172.21.15.16:0/3058786891 wait complete. 2024-03-27T06:43:41.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.351+0000 7f58363c6740 20 client.15582 trim_cache size 0 max 16384 2024-03-27T06:43:41.353 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:43:41.353 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-27T06:43:41.353 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-03-27T06:43:41.353 INFO:teuthology.orchestra.run.smithi016.stdout: "fb4110ef-a146-4874-937e-c9d570087b3f": { 2024-03-27T06:43:41.353 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-03-27T06:43:41.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "fb4110ef-a146-4874-937e-c9d570087b3f", 2024-03-27T06:43:41.353 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-03-27T06:43:41.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:43:41.354 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:43:41.354 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:43:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 -- 172.21.15.16:0/2310360123 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f58180a38a0 msgr2=0x7f58180a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f58180a38a0 0x7f58180a5d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5824023000 tx=0x7f5824024000 comp rx=0 tx=0).stop 2024-03-27T06:43:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 -- 172.21.15.16:0/2310360123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c101380 msgr2=0x7f582c19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c101380 0x7f582c19ac60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f581c00e9f0 tx=0x7f581c00eeb0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 -- 172.21.15.16:0/2310360123 shutdown_connections 2024-03-27T06:43:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f58180a38a0 0x7f58180a5d60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f582c1069d0 0x7f582c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c101380 0x7f582c19ac60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 --2- 172.21.15.16:0/2310360123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f582c1009c0 0x7f582c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 -- 172.21.15.16:0/2310360123 >> 172.21.15.16:0/2310360123 conn(0x7f582c077b50 msgr2=0x7f582c068e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 -- 172.21.15.16:0/2310360123 shutdown_connections 2024-03-27T06:43:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.352+0000 7f5833702640 1 -- 172.21.15.16:0/2310360123 wait complete. 2024-03-27T06:43:41.357 INFO:tasks.cephfs.filesystem:scrub status for tag:fb4110ef-a146-4874-937e-c9d570087b3f - {'path': '/', 'tag': 'fb4110ef-a146-4874-937e-c9d570087b3f', 'options': 'recursive,force'} 2024-03-27T06:43:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.390+0000 7f66592c0640 1 -- 172.21.15.16:0/265364538 --> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f6654061f90 con 0x7f66400a3940 2024-03-27T06:43:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.447+0000 7f6650ff9640 1 -- 172.21.15.16:0/265364538 <== mgr.15354 v2:172.21.15.16:6800/3715976676 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6654061f90 con 0x7f66400a3940 2024-03-27T06:43:41.442 INFO:teuthology.orchestra.run.smithi016.stdout:Schedule set for path / 2024-03-27T06:43:41.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 -- 172.21.15.16:0/265364538 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f66400a3940 msgr2=0x7f66400a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f66400a3940 0x7f66400a5e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f663c0095f0 tx=0x7f663c005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:41.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 -- 172.21.15.16:0/265364538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654100ca0 msgr2=0x7f66541026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654100ca0 0x7f66541026c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f664400e9f0 tx=0x7f664400eeb0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 -- 172.21.15.16:0/265364538 shutdown_connections 2024-03-27T06:43:41.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f66400a3940 0x7f66400a5e00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66541059f0 0x7f6654103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6654101660 0x7f6654102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 --2- 172.21.15.16:0/265364538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654100ca0 0x7f66541026c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 -- 172.21.15.16:0/265364538 >> 172.21.15.16:0/265364538 conn(0x7f66540778a0 msgr2=0x7f6654069410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 -- 172.21.15.16:0/265364538 shutdown_connections 2024-03-27T06:43:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.449+0000 7f66592c0640 1 -- 172.21.15.16:0/265364538 wait complete. 2024-03-27T06:43:41.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-27T06:43:41.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.606+0000 7f3be98d4640 1 -- 172.21.15.16:0/4240406993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3be41002c0 msgr2=0x7f3be41006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.607+0000 7f3be98d4640 1 --2- 172.21.15.16:0/4240406993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3be41002c0 0x7f3be41006a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009900 tx=0x7f3bdc02bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.607+0000 7f3be98d4640 1 -- 172.21.15.16:0/4240406993 shutdown_connections 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.607+0000 7f3be98d4640 1 --2- 172.21.15.16:0/4240406993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3be4106e80 0x7f3be410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.607+0000 7f3be98d4640 1 --2- 172.21.15.16:0/4240406993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3be4100c70 0x7f3be41067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.607+0000 7f3be98d4640 1 --2- 172.21.15.16:0/4240406993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3be41002c0 0x7f3be41006a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.607+0000 7f3be98d4640 1 -- 172.21.15.16:0/4240406993 >> 172.21.15.16:0/4240406993 conn(0x7f3be4077010 msgr2=0x7f3be40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.607+0000 7f3be98d4640 1 -- 172.21.15.16:0/4240406993 shutdown_connections 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.607+0000 7f3be98d4640 1 -- 172.21.15.16:0/4240406993 wait complete. 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.607+0000 7f3be98d4640 1 Processor -- start 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.607+0000 7f3be98d4640 1 -- start start 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.607+0000 7f3be98d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3be41002c0 0x7f3be41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be98d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3be4106e80 0x7f3be4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be98d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3be419cd80 0x7f3be419d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be98d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4112020 con 0x7f3be4106e80 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be98d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4111ea0 con 0x7f3be41002c0 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be98d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41121a0 con 0x7f3be419cd80 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3be4106e80 0x7f3be4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3be4106e80 0x7f3be4198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51450/0 (socket says 172.21.15.16:51450) 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be27fc640 1 -- 172.21.15.16:0/4224637560 learned_addr learned my addr 172.21.15.16:0/4224637560 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be37fe640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3be419cd80 0x7f3be419d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be2ffd640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3be41002c0 0x7f3be41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be27fc640 1 -- 172.21.15.16:0/4224637560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3be419cd80 msgr2=0x7f3be419d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be27fc640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3be419cd80 0x7f3be419d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be27fc640 1 -- 172.21.15.16:0/4224637560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3be41002c0 msgr2=0x7f3be41984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be27fc640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3be41002c0 0x7f3be41984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be27fc640 1 -- 172.21.15.16:0/4224637560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be419d870 con 0x7f3be4106e80 2024-03-27T06:43:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be27fc640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3be4106e80 0x7f3be4198a00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00e9c0 tx=0x7f3bcc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be88d2640 1 -- 172.21.15.16:0/4224637560 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00cd80 con 0x7f3be4106e80 2024-03-27T06:43:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be88d2640 1 -- 172.21.15.16:0/4224637560 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3bcc005520 con 0x7f3be4106e80 2024-03-27T06:43:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be98d4640 1 -- 172.21.15.16:0/4224637560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be419db60 con 0x7f3be4106e80 2024-03-27T06:43:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be88d2640 1 -- 172.21.15.16:0/4224637560 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc010690 con 0x7f3be4106e80 2024-03-27T06:43:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.608+0000 7f3be98d4640 1 -- 172.21.15.16:0/4224637560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be41a5350 con 0x7f3be4106e80 2024-03-27T06:43:41.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.609+0000 7f3be98d4640 1 -- 172.21.15.16:0/4224637560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be404f8b0 con 0x7f3be4106e80 2024-03-27T06:43:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.611+0000 7f3be88d2640 1 -- 172.21.15.16:0/4224637560 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7f3bcc0046c0 con 0x7f3be4106e80 2024-03-27T06:43:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.612+0000 7f3be88d2640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f3bd00a3940 0x7f3bd00a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.612+0000 7f3be2ffd640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f3bd00a3940 0x7f3bd00a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.612+0000 7f3be88d2640 1 -- 172.21.15.16:0/4224637560 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3bcc0d0980 con 0x7f3be4106e80 2024-03-27T06:43:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.612+0000 7f3be2ffd640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f3bd00a3940 0x7f3bd00a5e00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc006010 tx=0x7f3bdc005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.616+0000 7f3be88d2640 1 -- 172.21.15.16:0/4224637560 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f3bcc053cb0 con 0x7f3be4106e80 2024-03-27T06:43:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.826+0000 7f3be98d4640 1 -- 172.21.15.16:0/4224637560 --> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f3be4100c70 con 0x7f3bd00a3940 2024-03-27T06:43:41.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.877+0000 7f3be88d2640 1 -- 172.21.15.16:0/4224637560 <== mgr.15354 v2:172.21.15.16:6800/3715976676 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f3be4100c70 con 0x7f3bd00a3940 2024-03-27T06:43:41.872 INFO:teuthology.orchestra.run.smithi016.stdout:Retention added to path / 2024-03-27T06:43:41.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 -- 172.21.15.16:0/4224637560 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f3bd00a3940 msgr2=0x7f3bd00a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f3bd00a3940 0x7f3bd00a5e00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc006010 tx=0x7f3bdc005e40 comp rx=0 tx=0).stop 2024-03-27T06:43:41.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 -- 172.21.15.16:0/4224637560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3be4106e80 msgr2=0x7f3be4198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3be4106e80 0x7f3be4198a00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00e9c0 tx=0x7f3bcc00ee80 comp rx=0 tx=0).stop 2024-03-27T06:43:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 -- 172.21.15.16:0/4224637560 shutdown_connections 2024-03-27T06:43:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f3bd00a3940 0x7f3bd00a5e00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3be419cd80 0x7f3be419d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3be4106e80 0x7f3be4198a00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 --2- 172.21.15.16:0/4224637560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3be41002c0 0x7f3be41984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 -- 172.21.15.16:0/4224637560 >> 172.21.15.16:0/4224637560 conn(0x7f3be4077010 msgr2=0x7f3be4105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 -- 172.21.15.16:0/4224637560 shutdown_connections 2024-03-27T06:43:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.879+0000 7f3be98d4640 1 -- 172.21.15.16:0/4224637560 wait complete. 2024-03-27T06:43:41.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-27T06:43:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.995+0000 7fd788183640 1 -- 172.21.15.16:0/2976926329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd780101660 msgr2=0x7fd78010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.995+0000 7fd788183640 1 --2- 172.21.15.16:0/2976926329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd780101660 0x7fd78010d880 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009ca0 tx=0x7fd77c02c0f0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.995+0000 7fd788183640 1 -- 172.21.15.16:0/2976926329 shutdown_connections 2024-03-27T06:43:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.995+0000 7fd788183640 1 --2- 172.21.15.16:0/2976926329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd780101660 0x7fd78010d880 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.995+0000 7fd788183640 1 --2- 172.21.15.16:0/2976926329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd780100ca0 0x7fd780101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.995+0000 7fd788183640 1 --2- 172.21.15.16:0/2976926329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7801059f0 0x7fd780105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.995+0000 7fd788183640 1 -- 172.21.15.16:0/2976926329 >> 172.21.15.16:0/2976926329 conn(0x7fd7800778a0 msgr2=0x7fd780077cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd788183640 1 -- 172.21.15.16:0/2976926329 shutdown_connections 2024-03-27T06:43:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd788183640 1 -- 172.21.15.16:0/2976926329 wait complete. 2024-03-27T06:43:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd788183640 1 Processor -- start 2024-03-27T06:43:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd788183640 1 -- start start 2024-03-27T06:43:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd788183640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd780100ca0 0x7fd7801026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd788183640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd780101660 0x7fd780102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd788183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7801059f0 0x7fd780103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd788183640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd780111030 con 0x7fd7801059f0 2024-03-27T06:43:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd788183640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd780110eb0 con 0x7fd780101660 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd788183640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7801111b0 con 0x7fd780100ca0 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd786720640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7801059f0 0x7fd780103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd785f1f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd780100ca0 0x7fd7801026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd786720640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7801059f0 0x7fd780103140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51480/0 (socket says 172.21.15.16:51480) 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd78571e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd780101660 0x7fd780102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.996+0000 7fd786720640 1 -- 172.21.15.16:0/3046998555 learned_addr learned my addr 172.21.15.16:0/3046998555 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.997+0000 7fd786720640 1 -- 172.21.15.16:0/3046998555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd780100ca0 msgr2=0x7fd7801026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.997+0000 7fd786720640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd780100ca0 0x7fd7801026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.997+0000 7fd786720640 1 -- 172.21.15.16:0/3046998555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd780101660 msgr2=0x7fd780102c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.997+0000 7fd786720640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd780101660 0x7fd780102c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.997+0000 7fd786720640 1 -- 172.21.15.16:0/3046998555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7801a6960 con 0x7fd7801059f0 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.997+0000 7fd786720640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7801059f0 0x7fd780103140 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd77c005e30 tx=0x7fd77c034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.997+0000 7fd776ffd640 1 -- 172.21.15.16:0/3046998555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c039050 con 0x7fd7801059f0 2024-03-27T06:43:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.997+0000 7fd776ffd640 1 -- 172.21.15.16:0/3046998555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd77c045070 con 0x7fd7801059f0 2024-03-27T06:43:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.997+0000 7fd788183640 1 -- 172.21.15.16:0/3046998555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7801a6bf0 con 0x7fd7801059f0 2024-03-27T06:43:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.997+0000 7fd776ffd640 1 -- 172.21.15.16:0/3046998555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c004630 con 0x7fd7801059f0 2024-03-27T06:43:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.997+0000 7fd788183640 1 -- 172.21.15.16:0/3046998555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7801a70e0 con 0x7fd7801059f0 2024-03-27T06:43:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.998+0000 7fd788183640 1 -- 172.21.15.16:0/3046998555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd78004f8b0 con 0x7fd7801059f0 2024-03-27T06:43:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:41.999+0000 7fd776ffd640 1 -- 172.21.15.16:0/3046998555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7fd77c003970 con 0x7fd7801059f0 2024-03-27T06:43:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.000+0000 7fd776ffd640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd76c0a3940 0x7fd76c0a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.000+0000 7fd776ffd640 1 -- 172.21.15.16:0/3046998555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd77c0f3620 con 0x7fd7801059f0 2024-03-27T06:43:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.000+0000 7fd785f1f640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd76c0a3940 0x7fd76c0a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.000+0000 7fd785f1f640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd76c0a3940 0x7fd76c0a5e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd778009b50 tx=0x7fd778005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.003+0000 7fd776ffd640 1 -- 172.21.15.16:0/3046998555 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd77c0bf6f0 con 0x7fd7801059f0 2024-03-27T06:43:42.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:41 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1265368055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:42.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:41 smithi016 ceph-mon[31908]: scrub queued for path: / 2024-03-27T06:43:42.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:41 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:43:42.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:41 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:43:42.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:41 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:43:42.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:41 smithi016 ceph-mon[31908]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.209+0000 7fd788183640 1 -- 172.21.15.16:0/3046998555 --> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fd7800699e0 con 0x7fd76c0a3940 2024-03-27T06:43:42.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.216+0000 7fd776ffd640 1 -- 172.21.15.16:0/3046998555 <== mgr.15354 v2:172.21.15.16:6800/3715976676 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fd7800699e0 con 0x7fd76c0a3940 2024-03-27T06:43:42.211 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-27T00:00:00", "created": "2024-03-27T06:43:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-27T06:43:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 -- 172.21.15.16:0/3046998555 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd76c0a3940 msgr2=0x7fd76c0a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd76c0a3940 0x7fd76c0a5e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd778009b50 tx=0x7fd778005f90 comp rx=0 tx=0).stop 2024-03-27T06:43:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 -- 172.21.15.16:0/3046998555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7801059f0 msgr2=0x7fd780103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7801059f0 0x7fd780103140 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd77c005e30 tx=0x7fd77c034440 comp rx=0 tx=0).stop 2024-03-27T06:43:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 -- 172.21.15.16:0/3046998555 shutdown_connections 2024-03-27T06:43:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7801059f0 0x7fd780103140 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd780101660 0x7fd780102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd76c0a3940 0x7fd76c0a5e00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 --2- 172.21.15.16:0/3046998555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd780100ca0 0x7fd7801026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 -- 172.21.15.16:0/3046998555 >> 172.21.15.16:0/3046998555 conn(0x7fd7800778a0 msgr2=0x7fd7800692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 -- 172.21.15.16:0/3046998555 shutdown_connections 2024-03-27T06:43:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.218+0000 7fd788183640 1 -- 172.21.15.16:0/3046998555 wait complete. 2024-03-27T06:43:42.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:41 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1265368055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:42.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:41 smithi156 ceph-mon[36261]: scrub queued for path: / 2024-03-27T06:43:42.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:41 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:43:42.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:41 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:43:42.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:41 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:43:42.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:41 smithi156 ceph-mon[36261]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:42.223 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-27T06:43:42.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.337+0000 7f5ede70f640 1 -- 172.21.15.16:0/2858975420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed80691d0 msgr2=0x7f5ed8069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.337+0000 7f5ede70f640 1 --2- 172.21.15.16:0/2858975420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed80691d0 0x7f5ed8069650 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8009900 tx=0x7f5ec802bcd0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.337+0000 7f5ede70f640 1 -- 172.21.15.16:0/2858975420 shutdown_connections 2024-03-27T06:43:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.337+0000 7f5ede70f640 1 --2- 172.21.15.16:0/2858975420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ed8069b90 0x7f5ed810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.337+0000 7f5ede70f640 1 --2- 172.21.15.16:0/2858975420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed80691d0 0x7f5ed8069650 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.337+0000 7f5ede70f640 1 --2- 172.21.15.16:0/2858975420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed8107c60 0x7f5ed8108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.337+0000 7f5ede70f640 1 -- 172.21.15.16:0/2858975420 >> 172.21.15.16:0/2858975420 conn(0x7f5ed80778a0 msgr2=0x7f5ed8077cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:42.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.337+0000 7f5ede70f640 1 -- 172.21.15.16:0/2858975420 shutdown_connections 2024-03-27T06:43:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.337+0000 7f5ede70f640 1 -- 172.21.15.16:0/2858975420 wait complete. 2024-03-27T06:43:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.337+0000 7f5ede70f640 1 Processor -- start 2024-03-27T06:43:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.337+0000 7f5ede70f640 1 -- start start 2024-03-27T06:43:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ede70f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed80691d0 0x7f5ed8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ede70f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ed8069b90 0x7f5ed8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ede70f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed8107c60 0x7f5ed819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ede70f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed8112020 con 0x7f5ed80691d0 2024-03-27T06:43:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ede70f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed8111ea0 con 0x7f5ed8069b90 2024-03-27T06:43:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ede70f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81121a0 con 0x7f5ed8107c60 2024-03-27T06:43:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed80691d0 0x7f5ed8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:42.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed80691d0 0x7f5ed8198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51492/0 (socket says 172.21.15.16:51492) 2024-03-27T06:43:42.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed7fff640 1 -- 172.21.15.16:0/2901653392 learned_addr learned my addr 172.21.15.16:0/2901653392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:42.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed77fe640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ed8069b90 0x7f5ed8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:42.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5edccac640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed8107c60 0x7f5ed819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:42.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed7fff640 1 -- 172.21.15.16:0/2901653392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed8107c60 msgr2=0x7f5ed819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:42.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed7fff640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed8107c60 0x7f5ed819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed7fff640 1 -- 172.21.15.16:0/2901653392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ed8069b90 msgr2=0x7f5ed8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:42.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed7fff640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ed8069b90 0x7f5ed8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed7fff640 1 -- 172.21.15.16:0/2901653392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed819d3a0 con 0x7f5ed80691d0 2024-03-27T06:43:42.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed77fe640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ed8069b90 0x7f5ed8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:42.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed7fff640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed80691d0 0x7f5ed8198400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00e9f0 tx=0x7f5ecc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:42.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.338+0000 7f5ed57fa640 1 -- 172.21.15.16:0/2901653392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc00cd80 con 0x7f5ed80691d0 2024-03-27T06:43:42.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.339+0000 7f5ed57fa640 1 -- 172.21.15.16:0/2901653392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5ecc005520 con 0x7f5ed80691d0 2024-03-27T06:43:42.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.339+0000 7f5ede70f640 1 -- 172.21.15.16:0/2901653392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed819d690 con 0x7f5ed80691d0 2024-03-27T06:43:42.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.339+0000 7f5ed57fa640 1 -- 172.21.15.16:0/2901653392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc010690 con 0x7f5ed80691d0 2024-03-27T06:43:42.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.339+0000 7f5ede70f640 1 -- 172.21.15.16:0/2901653392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed81a4f20 con 0x7f5ed80691d0 2024-03-27T06:43:42.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.340+0000 7f5ede70f640 1 -- 172.21.15.16:0/2901653392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed804f8b0 con 0x7f5ed80691d0 2024-03-27T06:43:42.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.341+0000 7f5ed57fa640 1 -- 172.21.15.16:0/2901653392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7f5ecc0046c0 con 0x7f5ed80691d0 2024-03-27T06:43:42.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.341+0000 7f5ed57fa640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5ec00a39c0 0x7f5ec00a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.342+0000 7f5ed77fe640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5ec00a39c0 0x7f5ec00a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.342+0000 7f5ed57fa640 1 -- 172.21.15.16:0/2901653392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5ecc0d0710 con 0x7f5ed80691d0 2024-03-27T06:43:42.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.342+0000 7f5ed77fe640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5ec00a39c0 0x7f5ec00a5e80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8006010 tx=0x7f5ec8005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:42.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.346+0000 7f5ed57fa640 1 -- 172.21.15.16:0/2901653392 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5ecc09cd80 con 0x7f5ed80691d0 2024-03-27T06:43:42.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:41 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1265368055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:43:42.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:41 smithi170 ceph-mon[36438]: scrub queued for path: / 2024-03-27T06:43:42.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:41 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:43:42.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:41 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:43:42.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:41 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:43:42.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:41 smithi170 ceph-mon[36438]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.554+0000 7f5ede70f640 1 -- 172.21.15.16:0/2901653392 --> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5ed80ffe80 con 0x7f5ec00a39c0 2024-03-27T06:43:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.560+0000 7f5ed57fa640 1 -- 172.21.15.16:0/2901653392 <== mgr.15354 v2:172.21.15.16:6800/3715976676 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5ed80ffe80 con 0x7f5ec00a39c0 2024-03-27T06:43:42.555 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-03-27T06:43:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 -- 172.21.15.16:0/2901653392 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5ec00a39c0 msgr2=0x7f5ec00a5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5ec00a39c0 0x7f5ec00a5e80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8006010 tx=0x7f5ec8005e40 comp rx=0 tx=0).stop 2024-03-27T06:43:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 -- 172.21.15.16:0/2901653392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed80691d0 msgr2=0x7f5ed8198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed80691d0 0x7f5ed8198400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00e9f0 tx=0x7f5ecc00edf0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 -- 172.21.15.16:0/2901653392 shutdown_connections 2024-03-27T06:43:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5ec00a39c0 0x7f5ec00a5e80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed8107c60 0x7f5ed819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ed8069b90 0x7f5ed8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 --2- 172.21.15.16:0/2901653392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed80691d0 0x7f5ed8198400 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 -- 172.21.15.16:0/2901653392 >> 172.21.15.16:0/2901653392 conn(0x7f5ed80778a0 msgr2=0x7f5ed80ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 -- 172.21.15.16:0/2901653392 shutdown_connections 2024-03-27T06:43:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:43:42.562+0000 7f5ede70f640 1 -- 172.21.15.16:0/2901653392 wait complete. 2024-03-27T06:43:42.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-27T06:43:42.596 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-27T06:43:42.607 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-27T06:43:42.617 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-03-27T06:43:42.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-27T06:43:42.639 INFO:teuthology.run_tasks:Running task workunit... 2024-03-27T06:43:42.653 INFO:tasks.workunit:Pulling workunits from ref 155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:43:42.653 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-27T06:43:42.653 INFO:tasks.workunit:timeout=3h 2024-03-27T06:43:42.653 INFO:tasks.workunit:cleanup=True 2024-03-27T06:43:42.653 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-27T06:43:42.668 INFO:teuthology.orchestra.run.smithi016.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-27T06:43:42.668 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-27T06:43:42.668 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-27T06:43:42.668 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-27T06:43:42.668 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-27T06:43:42.668 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-03-27 06:41:34.939869228 +0000 2024-03-27T06:43:42.669 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-03-27 06:41:34.939869228 +0000 2024-03-27T06:43:42.669 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-03-27 06:43:32.125308598 +0000 2024-03-27T06:43:42.669 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-03-27 06:41:34.939869228 +0000 2024-03-27T06:43:42.669 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-27T06:43:42.669 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-27T06:43:42.743 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 155268c4e432a12433aa833f174f9fe3b1016ae0 2024-03-27T06:43:42.797 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-27T06:43:43.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:42 smithi156 ceph-mon[36261]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 25 op/s 2024-03-27T06:43:43.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:42 smithi156 ceph-mon[36261]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:43.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:42 smithi156 ceph-mon[36261]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:43.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:42 smithi156 ceph-mon[36261]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:43.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:42 smithi170 ceph-mon[36438]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 25 op/s 2024-03-27T06:43:43.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:42 smithi170 ceph-mon[36438]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:43.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:42 smithi170 ceph-mon[36438]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:43.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:42 smithi170 ceph-mon[36438]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:43.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:42 smithi016 ceph-mon[31908]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 25 op/s 2024-03-27T06:43:43.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:42 smithi016 ceph-mon[31908]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:43.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:42 smithi016 ceph-mon[31908]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:43.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:42 smithi016 ceph-mon[31908]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:43:45.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:44 smithi156 ceph-mon[36261]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 797 B/s wr, 20 op/s 2024-03-27T06:43:45.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:44 smithi156 ceph-mon[36261]: scrub summary: idle 2024-03-27T06:43:45.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:44 smithi170 ceph-mon[36438]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 797 B/s wr, 20 op/s 2024-03-27T06:43:45.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:44 smithi170 ceph-mon[36438]: scrub summary: idle 2024-03-27T06:43:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:44 smithi016 ceph-mon[31908]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 797 B/s wr, 20 op/s 2024-03-27T06:43:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:44 smithi016 ceph-mon[31908]: scrub summary: idle 2024-03-27T06:43:47.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:46 smithi156 ceph-mon[36261]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-03-27T06:43:47.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:46 smithi170 ceph-mon[36438]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-03-27T06:43:47.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:46 smithi016 ceph-mon[31908]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-03-27T06:43:49.186 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:48 smithi016 ceph-mon[31908]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-27T06:43:49.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:48 smithi156 ceph-mon[36261]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-27T06:43:49.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:48 smithi170 ceph-mon[36438]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-27T06:43:51.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:50 smithi156 ceph-mon[36261]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-27T06:43:51.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:50 smithi170 ceph-mon[36438]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-27T06:43:51.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:50 smithi016 ceph-mon[31908]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-27T06:43:53.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:52 smithi156 ceph-mon[36261]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-03-27T06:43:53.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:52 smithi170 ceph-mon[36438]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-03-27T06:43:53.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:52 smithi016 ceph-mon[31908]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-03-27T06:43:55.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:54 smithi156 ceph-mon[36261]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-03-27T06:43:55.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:54 smithi170 ceph-mon[36438]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-03-27T06:43:55.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:54 smithi016 ceph-mon[31908]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-03-27T06:43:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.784+0000 7f81a34aa640 1 -- 172.21.15.16:0/1686666432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f819c103770 msgr2=0x7f819c10bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.784+0000 7f81a34aa640 1 --2- 172.21.15.16:0/1686666432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f819c103770 0x7f819c10bbc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f818c009900 tx=0x7f818c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.784+0000 7f81a34aa640 1 -- 172.21.15.16:0/1686666432 shutdown_connections 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.784+0000 7f81a34aa640 1 --2- 172.21.15.16:0/1686666432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c10c2d0 0x7f819c10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.784+0000 7f81a34aa640 1 --2- 172.21.15.16:0/1686666432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f819c103770 0x7f819c10bbc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.784+0000 7f81a34aa640 1 --2- 172.21.15.16:0/1686666432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c102dc0 0x7f819c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.784+0000 7f81a34aa640 1 -- 172.21.15.16:0/1686666432 >> 172.21.15.16:0/1686666432 conn(0x7f819c077b70 msgr2=0x7f819c077f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.784+0000 7f81a34aa640 1 -- 172.21.15.16:0/1686666432 shutdown_connections 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.784+0000 7f81a34aa640 1 -- 172.21.15.16:0/1686666432 wait complete. 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a34aa640 1 Processor -- start 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a34aa640 1 -- start start 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a34aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c102dc0 0x7f819c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a34aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f819c10c2d0 0x7f819c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a34aa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c19ce80 0x7f819c19d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a34aa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f819c112190 con 0x7f819c102dc0 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a34aa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f819c112010 con 0x7f819c10c2d0 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a34aa640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f819c112310 con 0x7f819c19ce80 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a1246640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c102dc0 0x7f819c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a1246640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c102dc0 0x7f819c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34434/0 (socket says 172.21.15.16:34434) 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a1246640 1 -- 172.21.15.16:0/2070255660 learned_addr learned my addr 172.21.15.16:0/2070255660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a0a45640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f819c10c2d0 0x7f819c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.785+0000 7f81a1a47640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c19ce80 0x7f819c19d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81a1246640 1 -- 172.21.15.16:0/2070255660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c19ce80 msgr2=0x7f819c19d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81a1246640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c19ce80 0x7f819c19d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81a1246640 1 -- 172.21.15.16:0/2070255660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f819c10c2d0 msgr2=0x7f819c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81a1246640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f819c10c2d0 0x7f819c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81a1246640 1 -- 172.21.15.16:0/2070255660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c19d970 con 0x7f819c102dc0 2024-03-27T06:43:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81a0a45640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f819c10c2d0 0x7f819c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81a1246640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c102dc0 0x7f819c198610 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f819800e9f0 tx=0x7f819800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81927fc640 1 -- 172.21.15.16:0/2070255660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819800cd80 con 0x7f819c102dc0 2024-03-27T06:43:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81927fc640 1 -- 172.21.15.16:0/2070255660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8198010930 con 0x7f819c102dc0 2024-03-27T06:43:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81a34aa640 1 -- 172.21.15.16:0/2070255660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c19dc60 con 0x7f819c102dc0 2024-03-27T06:43:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81927fc640 1 -- 172.21.15.16:0/2070255660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198018b00 con 0x7f819c102dc0 2024-03-27T06:43:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.786+0000 7f81a34aa640 1 -- 172.21.15.16:0/2070255660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c1a5420 con 0x7f819c102dc0 2024-03-27T06:43:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.788+0000 7f81927fc640 1 -- 172.21.15.16:0/2070255660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138597+0+0 (secure 0 0 0) 0x7f8198018ca0 con 0x7f819c102dc0 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.789+0000 7f81a6168740 1 Processor -- start 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.789+0000 7f81a6168740 1 -- start start 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.789+0000 7f81927fc640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f81880a3940 0x7f81880a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.789+0000 7f81a6168740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.789+0000 7f81a6168740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cba10 0x5575c78d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.789+0000 7f81a6168740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78d47e0 0x5575c78d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.789+0000 7f81a6168740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5575c7891cd0 con 0x5575c78cb0f0 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.789+0000 7f81a6168740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5575c77d6950 con 0x5575c78d47e0 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.789+0000 7f81a6168740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5575c782ee50 con 0x5575c78cba10 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.789+0000 7f81a1a47640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f81880a3940 0x7f81880a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a1246640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81927fc640 1 -- 172.21.15.16:0/2070255660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f81980d0770 con 0x7f819c102dc0 2024-03-27T06:43:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a0a45640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cba10 0x5575c78d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a1246640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34450/0 (socket says 172.21.15.16:34450) 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a1246640 1 -- 172.21.15.16:0/2829092058 learned_addr learned my addr 172.21.15.16:0/2829092058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a1a47640 1 --2- 172.21.15.16:0/2829092058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78d47e0 0x5575c78d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a0a45640 1 -- 172.21.15.16:0/2829092058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78d47e0 msgr2=0x5575c78d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a0a45640 1 --2- 172.21.15.16:0/2829092058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78d47e0 0x5575c78d6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a0a45640 1 -- 172.21.15.16:0/2829092058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 msgr2=0x5575c78cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a0a45640 1 --2- 172.21.15.16:0/2829092058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a0a45640 1 -- 172.21.15.16:0/2829092058 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575c7543210 con 0x5575c78cba10 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a1246640 1 --2- 172.21.15.16:0/2829092058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a1a47640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f81880a3940 0x7f81880a5e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8194009fd0 tx=0x7f8194009d80 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81a0a45640 1 --2- 172.21.15.16:0/2829092058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cba10 0x5575c78d42a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f818c02c170 tx=0x7f818c009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9975a37b8edd9d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81877fe640 1 -- 172.21.15.16:0/2829092058 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c043070 con 0x5575c78cba10 2024-03-27T06:43:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81877fe640 1 -- 172.21.15.16:0/2829092058 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f818c02c770 con 0x5575c78cba10 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.790+0000 7f81877fe640 1 -- 172.21.15.16:0/2829092058 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c0039c0 con 0x5575c78cba10 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.791+0000 7f81a6168740 1 -- 172.21.15.16:0/2829092058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cba10 msgr2=0x5575c78d42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.791+0000 7f81a6168740 1 --2- 172.21.15.16:0/2829092058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cba10 0x5575c78d42a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f818c02c170 tx=0x7f818c009610 comp rx=0 tx=0).stop 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.791+0000 7f81a6168740 1 -- 172.21.15.16:0/2829092058 shutdown_connections 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.791+0000 7f81a6168740 1 --2- 172.21.15.16:0/2829092058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78d47e0 0x5575c78d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.791+0000 7f81a6168740 1 --2- 172.21.15.16:0/2829092058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cba10 0x5575c78d42a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.791+0000 7f81a6168740 1 --2- 172.21.15.16:0/2829092058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.791+0000 7f81a6168740 1 -- 172.21.15.16:0/2829092058 >> 172.21.15.16:0/2829092058 conn(0x5575c78c5a10 msgr2=0x5575c78c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.791+0000 7f81a6168740 1 -- 172.21.15.16:0/2829092058 shutdown_connections 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.791+0000 7f81a6168740 1 -- 172.21.15.16:0/2829092058 wait complete. 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.791+0000 7f81a6168740 1 -- start start 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a6168740 1 Processor -- start 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a6168740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 0x5575c78d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a6168740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a6168740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 0x5575c78d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a6168740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5575c782ae00 con 0x5575c78d47e0 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a6168740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5575c77d7520 con 0x5575c78cba10 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a6168740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5575c77ea8e0 con 0x5575c78cb0f0 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a6168740 10 client.0 _ll_register_callbacks cb 0x5575c7825a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a1a47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 0x5575c78d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a1a47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 0x5575c78d2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34456/0 (socket says 172.21.15.16:34456) 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a1a47640 1 -- 172.21.15.16:0/855539199 learned_addr learned my addr 172.21.15.16:0/855539199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a1246640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 0x5575c78d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a0a45640 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a1a47640 1 -- 172.21.15.16:0/855539199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 msgr2=0x5575c78d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a1a47640 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 0x5575c78d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a1a47640 1 -- 172.21.15.16:0/855539199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 msgr2=0x5575c78d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.792+0000 7f81a1a47640 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f81a1a47640 1 -- 172.21.15.16:0/855539199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575c760f620 con 0x5575c78d47e0 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f81a1a47640 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 0x5575c78d2220 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f819400fe00 tx=0x7f819401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82e208d58009b84 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f8184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f8184ff9640 1 -- 172.21.15.16:0/855539199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194013330 con 0x5575c78d47e0 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f8184ff9640 1 -- 172.21.15.16:0/855539199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f819401c040 con 0x5575c78d47e0 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f81a6168740 1 -- 172.21.15.16:0/855539199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f818c02c770 con 0x5575c78d47e0 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f8184ff9640 1 -- 172.21.15.16:0/855539199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f819400fe30 con 0x5575c78d47e0 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f81a6168740 10 client.15612 fetch_fsmap learned FSMap version 25 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f81a6168740 1 -- 172.21.15.16:0/855539199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575c78e3300 con 0x5575c78d47e0 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f8184ff9640 1 -- 172.21.15.16:0/855539199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f818c02c770 con 0x5575c78d47e0 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f8184ff9640 1 -- 172.21.15.16:0/855539199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8194020050 con 0x5575c78d47e0 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f81a6168740 10 client.15612 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:43:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.793+0000 7f81a6168740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.794+0000 7f81a6168740 20 client.15612 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.794+0000 7f81a6168740 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5575c78e4830 0x5575c78e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.794+0000 7f81a6168740 4 client.15612 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.794+0000 7f81a6168740 1 -- 172.21.15.16:0/855539199 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575c783c390 con 0x5575c78e4830 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.794+0000 7f81a1246640 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5575c78e4830 0x5575c78e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.794+0000 7f81a1246640 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5575c78e4830 0x5575c78e6c20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30010cdae9e27cd8 server_cookie=48b7cb0a12082014 in_seq=0 out_seq=0 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.794+0000 7f8184ff9640 10 client.15612 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f8184ff9640 1 -- 172.21.15.16:0/855539199 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5575c783c390 con 0x5575c78e4830 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f8184ff9640 10 client.15612 handle_command_reply: tid=0 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 1 client.15612 shutdown 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 1 -- 172.21.15.16:0/855539199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 msgr2=0x5575c78d2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 0x5575c78d2220 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f819400fe00 tx=0x7f819401d000 comp rx=0 tx=0).stop 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 1 -- 172.21.15.16:0/855539199 shutdown_connections 2024-03-27T06:43:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5575c78e4830 0x5575c78e6c20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 0x5575c78d2220 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f8184ff9640 0 client.15612 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 1 --2- 172.21.15.16:0/855539199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 0x5575c78d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 1 -- 172.21.15.16:0/855539199 >> 172.21.15.16:0/855539199 conn(0x5575c78c5a10 msgr2=0x5575c78c7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 1 -- 172.21.15.16:0/855539199 shutdown_connections 2024-03-27T06:43:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 1 -- 172.21.15.16:0/855539199 wait complete. 2024-03-27T06:43:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.795+0000 7f81a6168740 20 client.15612 trim_cache size 0 max 16384 2024-03-27T06:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a6168740 1 Processor -- start 2024-03-27T06:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a6168740 1 -- start start 2024-03-27T06:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a6168740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a6168740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78dfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a6168740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78d47e0 0x5575c78ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a6168740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1ab4b0 con 0x5575c78cb0f0 2024-03-27T06:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a6168740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1acff0 con 0x5575c78cba10 2024-03-27T06:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a6168740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1abb80 con 0x5575c78d47e0 2024-03-27T06:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a1246640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a1246640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78d3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34464/0 (socket says 172.21.15.16:34464) 2024-03-27T06:43:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a1246640 1 -- 172.21.15.16:0/475405029 learned_addr learned my addr 172.21.15.16:0/475405029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a0a45640 1 --2- 172.21.15.16:0/475405029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78dfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.810+0000 7f81a1a47640 1 --2- 172.21.15.16:0/475405029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78d47e0 0x5575c78ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.811+0000 7f81a1246640 1 -- 172.21.15.16:0/475405029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78d47e0 msgr2=0x5575c78ec530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.811+0000 7f81a1246640 1 --2- 172.21.15.16:0/475405029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78d47e0 0x5575c78ec530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.811+0000 7f81a1246640 1 -- 172.21.15.16:0/475405029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 msgr2=0x5575c78dfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.811+0000 7f81a1246640 1 --2- 172.21.15.16:0/475405029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78dfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.811+0000 7f81a1246640 1 -- 172.21.15.16:0/475405029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575c78cb840 con 0x5575c78cb0f0 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.811+0000 7f81a1246640 1 --2- 172.21.15.16:0/475405029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78d3ed0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8198098b90 tx=0x7f8198018520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e1188b2ea0f1b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.811+0000 7f8186ffd640 1 -- 172.21.15.16:0/475405029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198026040 con 0x5575c78cb0f0 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.811+0000 7f8186ffd640 1 -- 172.21.15.16:0/475405029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f81980a52f0 con 0x5575c78cb0f0 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.811+0000 7f8186ffd640 1 -- 172.21.15.16:0/475405029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819804e760 con 0x5575c78cb0f0 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 -- 172.21.15.16:0/475405029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 msgr2=0x5575c78d3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 --2- 172.21.15.16:0/475405029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78d3ed0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8198098b90 tx=0x7f8198018520 comp rx=0 tx=0).stop 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 -- 172.21.15.16:0/475405029 shutdown_connections 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 --2- 172.21.15.16:0/475405029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78d47e0 0x5575c78ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 --2- 172.21.15.16:0/475405029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78dfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 --2- 172.21.15.16:0/475405029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78cb0f0 0x5575c78d3ed0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 -- 172.21.15.16:0/475405029 >> 172.21.15.16:0/475405029 conn(0x5575c78c5a10 msgr2=0x5575c78cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 -- 172.21.15.16:0/475405029 shutdown_connections 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 -- 172.21.15.16:0/475405029 wait complete. 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 -- start start 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 Processor -- start 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 0x5575c78d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78d83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 0x5575c78d8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1ac920 con 0x5575c78d47e0 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1ac250 con 0x5575c78cba10 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1a9490 con 0x5575c78cb0f0 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.812+0000 7f81a6168740 10 client.0 _ll_register_callbacks cb 0x7f8194005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a1246640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 0x5575c78d7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a1a47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 0x5575c78d8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a1246640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 0x5575c78d7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:56866/0 (socket says 172.21.15.16:56866) 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a1246640 1 -- 172.21.15.16:0/2813710857 learned_addr learned my addr 172.21.15.16:0/2813710857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a0a45640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78d83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a1246640 1 -- 172.21.15.16:0/2813710857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 msgr2=0x5575c78d83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a1246640 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78d83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a1246640 1 -- 172.21.15.16:0/2813710857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 msgr2=0x5575c78d8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a1246640 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 0x5575c78d8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a1246640 1 -- 172.21.15.16:0/2813710857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575c7949c90 con 0x5575c78cb0f0 2024-03-27T06:43:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a1a47640 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 0x5575c78d8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a1246640 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 0x5575c78d7ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8198099100 tx=0x7f8198009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3a75822b07c7aa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f8184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f8184ff9640 1 -- 172.21.15.16:0/2813710857 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81980ba060 con 0x5575c78cb0f0 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f8184ff9640 1 -- 172.21.15.16:0/2813710857 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8198010ae0 con 0x5575c78cb0f0 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a6168740 1 -- 172.21.15.16:0/2813710857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81980a52f0 con 0x5575c78cb0f0 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f8184ff9640 1 -- 172.21.15.16:0/2813710857 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8198010ce0 con 0x5575c78cb0f0 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a6168740 10 client.25456 fetch_fsmap learned FSMap version 25 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.813+0000 7f81a6168740 1 -- 172.21.15.16:0/2813710857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5575c794af80 con 0x5575c78cb0f0 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.814+0000 7f8184ff9640 1 -- 172.21.15.16:0/2813710857 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f81980a52f0 con 0x5575c78cb0f0 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.814+0000 7f8184ff9640 1 -- 172.21.15.16:0/2813710857 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f81980ce080 con 0x5575c78cb0f0 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.814+0000 7f81a6168740 10 client.25456 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.814+0000 7f81a6168740 10 client.25456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.814+0000 7f81a6168740 20 client.25456 populate_metadata read hostname 'smithi016' 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.814+0000 7f81a6168740 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5575c794b590 0x5575c794d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.814+0000 7f81a6168740 4 client.25456 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:43:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.814+0000 7f81a6168740 1 -- 172.21.15.16:0/2813710857 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575c78366e0 con 0x5575c794b590 2024-03-27T06:43:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.814+0000 7f81a0a45640 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5575c794b590 0x5575c794d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:43:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.814+0000 7f81a0a45640 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5575c794b590 0x5575c794d9b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df6ad83bc18cab01 server_cookie=bcbd5e66760e8beb in_seq=0 out_seq=0 2024-03-27T06:43:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.814+0000 7f8184ff9640 10 client.25456 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.825+0000 7f8184ff9640 1 -- 172.21.15.16:0/2813710857 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575c78366e0 con 0x5575c794b590 2024-03-27T06:43:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f8184ff9640 10 client.25456 handle_command_reply: tid=0 2024-03-27T06:43:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 1 client.25456 shutdown 2024-03-27T06:43:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 1 -- 172.21.15.16:0/2813710857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 msgr2=0x5575c78d7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 0x5575c78d7ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8198099100 tx=0x7f8198009700 comp rx=0 tx=0).stop 2024-03-27T06:43:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 1 -- 172.21.15.16:0/2813710857 shutdown_connections 2024-03-27T06:43:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5575c794b590 0x5575c794d9b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575c78d47e0 0x5575c78d8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5575c78cba10 0x5575c78d83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 1 --2- 172.21.15.16:0/2813710857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575c78cb0f0 0x5575c78d7ea0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f8184ff9640 0 client.25456 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:43:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 1 -- 172.21.15.16:0/2813710857 >> 172.21.15.16:0/2813710857 conn(0x5575c78c5a10 msgr2=0x5575c78d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 1 -- 172.21.15.16:0/2813710857 shutdown_connections 2024-03-27T06:43:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 1 -- 172.21.15.16:0/2813710857 wait complete. 2024-03-27T06:43:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a6168740 20 client.25456 trim_cache size 0 max 16384 2024-03-27T06:43:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:43:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:43:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:43:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:43:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a34aa640 1 -- 172.21.15.16:0/2070255660 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f81880a3940 msgr2=0x7f81880a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.826+0000 7f81a34aa640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f81880a3940 0x7f81880a5e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8194009fd0 tx=0x7f8194009d80 comp rx=0 tx=0).stop 2024-03-27T06:43:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.827+0000 7f81a34aa640 1 -- 172.21.15.16:0/2070255660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c102dc0 msgr2=0x7f819c198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:43:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.827+0000 7f81a34aa640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c102dc0 0x7f819c198610 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f819800e9f0 tx=0x7f819800eeb0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.827+0000 7f81a34aa640 1 -- 172.21.15.16:0/2070255660 shutdown_connections 2024-03-27T06:43:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.827+0000 7f81a34aa640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f81880a3940 0x7f81880a5e00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.827+0000 7f81a34aa640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c19ce80 0x7f819c19d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.827+0000 7f81a34aa640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f819c10c2d0 0x7f819c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.827+0000 7f81a34aa640 1 --2- 172.21.15.16:0/2070255660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819c102dc0 0x7f819c198610 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:43:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.827+0000 7f81a34aa640 1 -- 172.21.15.16:0/2070255660 >> 172.21.15.16:0/2070255660 conn(0x7f819c077b70 msgr2=0x7f819c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:43:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.827+0000 7f81a34aa640 1 -- 172.21.15.16:0/2070255660 shutdown_connections 2024-03-27T06:43:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:43:55.827+0000 7f81a34aa640 1 -- 172.21.15.16:0/2070255660 wait complete. 2024-03-27T06:43:57.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:56 smithi156 ceph-mon[36261]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-03-27T06:43:57.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:56 smithi170 ceph-mon[36438]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-03-27T06:43:57.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:56 smithi016 ceph-mon[31908]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-03-27T06:43:59.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:43:58 smithi156 ceph-mon[36261]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:43:59.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:43:58 smithi170 ceph-mon[36438]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:43:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:43:58 smithi016 ceph-mon[31908]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:44:01.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:00 smithi156 ceph-mon[36261]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-27T06:44:01.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:00 smithi170 ceph-mon[36438]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-27T06:44:01.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:00 smithi016 ceph-mon[31908]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-27T06:44:01.362 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:44:00 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:44:00.886+0000 7f703daaf700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:44:01.362 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:44:00 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:44:00.886+0000 7f703daaf700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:44:01.362 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:44:00 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:44:00.886+0000 7f703daaf700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:44:01.362 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:44:00 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:44:00.886+0000 7f703daaf700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:44:01.362 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:44:00 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:44:00.886+0000 7f703daaf700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-27T06:44:03.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:02 smithi156 ceph-mon[36261]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 16 op/s 2024-03-27T06:44:03.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:02 smithi170 ceph-mon[36438]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 16 op/s 2024-03-27T06:44:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:02 smithi016 ceph-mon[31908]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 16 op/s 2024-03-27T06:44:04.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:03 smithi156 ceph-mon[36261]: mgrmap e24: x(active, since 29s), standbys: y, z 2024-03-27T06:44:04.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:03 smithi170 ceph-mon[36438]: mgrmap e24: x(active, since 29s), standbys: y, z 2024-03-27T06:44:04.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:03 smithi016 ceph-mon[31908]: mgrmap e24: x(active, since 29s), standbys: y, z 2024-03-27T06:44:05.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:04 smithi156 ceph-mon[36261]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-27T06:44:05.462 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:04 smithi016 ceph-mon[31908]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-27T06:44:05.462 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:04 smithi170 ceph-mon[36438]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-27T06:44:07.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:06 smithi156 ceph-mon[36261]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-03-27T06:44:07.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:06 smithi170 ceph-mon[36438]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-03-27T06:44:07.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:06 smithi016 ceph-mon[31908]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-03-27T06:44:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.926+0000 7ff47060e640 1 -- 172.21.15.16:0/2323749040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4681016d0 msgr2=0x7ff46810d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.926+0000 7ff47060e640 1 --2- 172.21.15.16:0/2323749040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4681016d0 0x7ff46810d7f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff460009ca0 tx=0x7ff46002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.926+0000 7ff47060e640 1 -- 172.21.15.16:0/2323749040 shutdown_connections 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.926+0000 7ff47060e640 1 --2- 172.21.15.16:0/2323749040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4681016d0 0x7ff46810d7f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.926+0000 7ff47060e640 1 --2- 172.21.15.16:0/2323749040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff468100d30 0x7ff468101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.926+0000 7ff47060e640 1 --2- 172.21.15.16:0/2323749040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff468106a90 0x7ff468106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.926+0000 7ff47060e640 1 -- 172.21.15.16:0/2323749040 >> 172.21.15.16:0/2323749040 conn(0x7ff468077770 msgr2=0x7ff468077b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.926+0000 7ff47060e640 1 -- 172.21.15.16:0/2323749040 shutdown_connections 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.926+0000 7ff47060e640 1 -- 172.21.15.16:0/2323749040 wait complete. 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff47060e640 1 Processor -- start 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff47060e640 1 -- start start 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff47060e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff468100d30 0x7ff468102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff47060e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4681016d0 0x7ff4681029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff47060e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468106a90 0x7ff468102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff47060e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff468110f20 con 0x7ff468106a90 2024-03-27T06:44:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff47060e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ff468110da0 con 0x7ff468100d30 2024-03-27T06:44:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff47060e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681110a0 con 0x7ff4681016d0 2024-03-27T06:44:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff46e3aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff468100d30 0x7ff468102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff46ebab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468106a90 0x7ff468102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff46e3aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff468100d30 0x7ff468102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60148/0 (socket says 172.21.15.16:60148) 2024-03-27T06:44:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff46e3aa640 1 -- 172.21.15.16:0/1150980845 learned_addr learned my addr 172.21.15.16:0/1150980845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.927+0000 7ff46dba9640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4681016d0 0x7ff4681029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff46ebab640 1 -- 172.21.15.16:0/1150980845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4681016d0 msgr2=0x7ff4681029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff46ebab640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4681016d0 0x7ff4681029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff46ebab640 1 -- 172.21.15.16:0/1150980845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff468100d30 msgr2=0x7ff468102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff46ebab640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff468100d30 0x7ff468102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff46ebab640 1 -- 172.21.15.16:0/1150980845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4681a6830 con 0x7ff468106a90 2024-03-27T06:44:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff468100d30 0x7ff468102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff46dba9640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4681016d0 0x7ff4681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff46ebab640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468106a90 0x7ff468102ef0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff46002c700 tx=0x7ff460034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff45f7fe640 1 -- 172.21.15.16:0/1150980845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460039040 con 0x7ff468106a90 2024-03-27T06:44:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff45f7fe640 1 -- 172.21.15.16:0/1150980845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff460044070 con 0x7ff468106a90 2024-03-27T06:44:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff47060e640 1 -- 172.21.15.16:0/1150980845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4681a6ac0 con 0x7ff468106a90 2024-03-27T06:44:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff45f7fe640 1 -- 172.21.15.16:0/1150980845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4600043a0 con 0x7ff468106a90 2024-03-27T06:44:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.928+0000 7ff47060e640 1 -- 172.21.15.16:0/1150980845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4681a6fd0 con 0x7ff468106a90 2024-03-27T06:44:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.931+0000 7ff45f7fe640 1 -- 172.21.15.16:0/1150980845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7ff460003b00 con 0x7ff468106a90 2024-03-27T06:44:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff4731fa740 1 Processor -- start 2024-03-27T06:44:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff4731fa740 1 -- start start 2024-03-27T06:44:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 0x555bfdf1e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff45f7fe640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff4540a3a50 0x7ff4540a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf20bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff4731fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555bfdedbcd0 con 0x555bfdf150f0 2024-03-27T06:44:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff4731fa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555bfde20950 con 0x555bfdf1e7e0 2024-03-27T06:44:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff4731fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bfde78e50 con 0x555bfdf15a10 2024-03-27T06:44:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff46dba9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff46dba9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf154d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33210/0 (socket says 172.21.15.16:33210) 2024-03-27T06:44:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff46dba9640 1 -- 172.21.15.16:0/3137531031 learned_addr learned my addr 172.21.15.16:0/3137531031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff46ebab640 1 --2- 172.21.15.16:0/3137531031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf20bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/3137531031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 0x555bfdf1e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff46dba9640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff4540a3a50 0x7ff4540a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.932+0000 7ff45f7fe640 1 -- 172.21.15.16:0/1150980845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff4600f26a0 con 0x7ff468106a90 2024-03-27T06:44:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff46dba9640 1 -- 172.21.15.16:0/3137531031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 msgr2=0x555bfdf1e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff46dba9640 1 --2- 172.21.15.16:0/3137531031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 0x555bfdf1e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff46dba9640 1 -- 172.21.15.16:0/3137531031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 msgr2=0x555bfdf20bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff46dba9640 1 --2- 172.21.15.16:0/3137531031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf20bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff46dba9640 1 -- 172.21.15.16:0/3137531031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bfdb8d210 con 0x555bfdf150f0 2024-03-27T06:44:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff46ebab640 1 --2- 172.21.15.16:0/3137531031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf20bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/3137531031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 0x555bfdf1e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff46dba9640 1 --2- 172.21.15.16:0/3137531031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf154d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff458011ed0 tx=0x7ff45800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7765e6f3a8d762c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff46dba9640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff4540a3a50 0x7ff4540a5f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff45800fc90 tx=0x7ff45801e000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff45cbfe640 1 -- 172.21.15.16:0/3137531031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45802b070 con 0x555bfdf150f0 2024-03-27T06:44:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff45cbfe640 1 -- 172.21.15.16:0/3137531031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff45801dcb0 con 0x555bfdf150f0 2024-03-27T06:44:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.933+0000 7ff45cbfe640 1 -- 172.21.15.16:0/3137531031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45801e400 con 0x555bfdf150f0 2024-03-27T06:44:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.934+0000 7ff4731fa740 1 -- 172.21.15.16:0/3137531031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 msgr2=0x555bfdf154d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.934+0000 7ff4731fa740 1 --2- 172.21.15.16:0/3137531031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf154d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff458011ed0 tx=0x7ff45800f790 comp rx=0 tx=0).stop 2024-03-27T06:44:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.934+0000 7ff4731fa740 1 -- 172.21.15.16:0/3137531031 shutdown_connections 2024-03-27T06:44:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.934+0000 7ff4731fa740 1 --2- 172.21.15.16:0/3137531031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf20bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.934+0000 7ff4731fa740 1 --2- 172.21.15.16:0/3137531031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 0x555bfdf1e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.934+0000 7ff4731fa740 1 --2- 172.21.15.16:0/3137531031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf154d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.934+0000 7ff4731fa740 1 -- 172.21.15.16:0/3137531031 >> 172.21.15.16:0/3137531031 conn(0x555bfdf0fa10 msgr2=0x555bfdf0fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.934+0000 7ff4731fa740 1 -- 172.21.15.16:0/3137531031 shutdown_connections 2024-03-27T06:44:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.934+0000 7ff4731fa740 1 -- 172.21.15.16:0/3137531031 wait complete. 2024-03-27T06:44:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.934+0000 7ff4731fa740 1 -- start start 2024-03-27T06:44:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.934+0000 7ff4731fa740 1 Processor -- start 2024-03-27T06:44:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf150f0 0x555bfdf1b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf1bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf1c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff4731fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555bfde74e00 con 0x555bfdf15a10 2024-03-27T06:44:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff4731fa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555bfde21520 con 0x555bfdf1e7e0 2024-03-27T06:44:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff4731fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bfde348e0 con 0x555bfdf150f0 2024-03-27T06:44:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff4731fa740 10 client.0 _ll_register_callbacks cb 0x555bfde6fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46dba9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf1bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46e3aa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf150f0 0x555bfdf1b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46ebab640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf1c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46e3aa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf150f0 0x555bfdf1b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:40502/0 (socket says 172.21.15.16:40502) 2024-03-27T06:44:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46e3aa640 1 -- 172.21.15.16:0/3644162056 learned_addr learned my addr 172.21.15.16:0/3644162056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46dba9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf1bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33220/0 (socket says 172.21.15.16:33220) 2024-03-27T06:44:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46e3aa640 1 -- 172.21.15.16:0/3644162056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 msgr2=0x555bfdf1c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf1c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46e3aa640 1 -- 172.21.15.16:0/3644162056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 msgr2=0x555bfdf1bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf1bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46e3aa640 1 -- 172.21.15.16:0/3644162056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bfdc59620 con 0x555bfdf150f0 2024-03-27T06:44:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46dba9640 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf1bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf150f0 0x555bfdf1b820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff464012d50 tx=0x7ff46400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ca9285e588f6c76 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.935+0000 7ff451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:44:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff451ffb640 1 -- 172.21.15.16:0/3644162056 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46400fe20 con 0x555bfdf150f0 2024-03-27T06:44:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff4731fa740 1 -- 172.21.15.16:0/3644162056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45801dcb0 con 0x555bfdf150f0 2024-03-27T06:44:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff451ffb640 1 -- 172.21.15.16:0/3644162056 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff464010430 con 0x555bfdf150f0 2024-03-27T06:44:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff451ffb640 1 -- 172.21.15.16:0/3644162056 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff464010630 con 0x555bfdf150f0 2024-03-27T06:44:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff4731fa740 10 client.25474 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff4731fa740 1 -- 172.21.15.16:0/3644162056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bfdf2d3e0 con 0x555bfdf150f0 2024-03-27T06:44:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff451ffb640 1 -- 172.21.15.16:0/3644162056 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff45801dcb0 con 0x555bfdf150f0 2024-03-27T06:44:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff451ffb640 1 -- 172.21.15.16:0/3644162056 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff46401c940 con 0x555bfdf150f0 2024-03-27T06:44:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff4731fa740 10 client.25474 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff4731fa740 10 client.25474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff4731fa740 20 client.25474 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff4731fa740 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555bfdf2e910 0x555bfdf30d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff4731fa740 4 client.25474 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.936+0000 7ff4731fa740 1 -- 172.21.15.16:0/3644162056 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bfde86390 con 0x555bfdf2e910 2024-03-27T06:44:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.937+0000 7ff46dba9640 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555bfdf2e910 0x555bfdf30d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.937+0000 7ff46dba9640 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555bfdf2e910 0x555bfdf30d80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df03be3761810ce server_cookie=807dfd821d271cec in_seq=0 out_seq=0 2024-03-27T06:44:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.937+0000 7ff451ffb640 10 client.25474 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff451ffb640 1 -- 172.21.15.16:0/3644162056 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555bfde86390 con 0x555bfdf2e910 2024-03-27T06:44:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff451ffb640 10 client.25474 handle_command_reply: tid=0 2024-03-27T06:44:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 1 client.25474 shutdown 2024-03-27T06:44:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 1 -- 172.21.15.16:0/3644162056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf150f0 msgr2=0x555bfdf1b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf150f0 0x555bfdf1b820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff464012d50 tx=0x7ff46400f790 comp rx=0 tx=0).stop 2024-03-27T06:44:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 1 -- 172.21.15.16:0/3644162056 shutdown_connections 2024-03-27T06:44:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555bfdf2e910 0x555bfdf30d80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf1c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf1bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff451ffb640 0 client.25474 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 1 --2- 172.21.15.16:0/3644162056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf150f0 0x555bfdf1b820 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 1 -- 172.21.15.16:0/3644162056 >> 172.21.15.16:0/3644162056 conn(0x555bfdf0fa10 msgr2=0x555bfdf11760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 1 -- 172.21.15.16:0/3644162056 shutdown_connections 2024-03-27T06:44:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 1 -- 172.21.15.16:0/3644162056 wait complete. 2024-03-27T06:44:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.938+0000 7ff4731fa740 20 client.25474 trim_cache size 0 max 16384 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.953+0000 7ff4731fa740 1 Processor -- start 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.953+0000 7ff4731fa740 1 -- start start 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.953+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf150f0 0x555bfdf1df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.953+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf29e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.953+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf1e7e0 0x555bfdf36690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.953+0000 7ff4731fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681af950 con 0x555bfdf15a10 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.953+0000 7ff4731fa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681b1490 con 0x555bfdf150f0 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.953+0000 7ff4731fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681b0020 con 0x555bfdf1e7e0 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46e3aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf150f0 0x555bfdf1df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46dba9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf29e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46e3aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf150f0 0x555bfdf1df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60172/0 (socket says 172.21.15.16:60172) 2024-03-27T06:44:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46dba9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf29e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33232/0 (socket says 172.21.15.16:33232) 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46e3aa640 1 -- 172.21.15.16:0/2551164949 learned_addr learned my addr 172.21.15.16:0/2551164949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46ebab640 1 --2- 172.21.15.16:0/2551164949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf1e7e0 0x555bfdf36690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46e3aa640 1 -- 172.21.15.16:0/2551164949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf1e7e0 msgr2=0x555bfdf36690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/2551164949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf1e7e0 0x555bfdf36690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46e3aa640 1 -- 172.21.15.16:0/2551164949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 msgr2=0x555bfdf29e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/2551164949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf29e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46e3aa640 1 -- 172.21.15.16:0/2551164949 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bfdf2a390 con 0x555bfdf150f0 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46ebab640 1 --2- 172.21.15.16:0/2551164949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf1e7e0 0x555bfdf36690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46dba9640 1 --2- 172.21.15.16:0/2551164949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf29e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/2551164949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf150f0 0x555bfdf1df50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff464012870 tx=0x7ff464005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=516af8794615a8a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff453fff640 1 -- 172.21.15.16:0/2551164949 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464029030 con 0x555bfdf150f0 2024-03-27T06:44:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff453fff640 1 -- 172.21.15.16:0/2551164949 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff464010430 con 0x555bfdf150f0 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.954+0000 7ff453fff640 1 -- 172.21.15.16:0/2551164949 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464010ba0 con 0x555bfdf150f0 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 -- 172.21.15.16:0/2551164949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf150f0 msgr2=0x555bfdf1df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 --2- 172.21.15.16:0/2551164949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf150f0 0x555bfdf1df50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff464012870 tx=0x7ff464005c50 comp rx=0 tx=0).stop 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 -- 172.21.15.16:0/2551164949 shutdown_connections 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 --2- 172.21.15.16:0/2551164949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf1e7e0 0x555bfdf36690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 --2- 172.21.15.16:0/2551164949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf15a10 0x555bfdf29e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 --2- 172.21.15.16:0/2551164949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf150f0 0x555bfdf1df50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 -- 172.21.15.16:0/2551164949 >> 172.21.15.16:0/2551164949 conn(0x555bfdf0fa10 msgr2=0x555bfdf16b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 -- 172.21.15.16:0/2551164949 shutdown_connections 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 -- 172.21.15.16:0/2551164949 wait complete. 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 -- start start 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 Processor -- start 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.955+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf26480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 0x555bfdf23450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff4731fa740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf23990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff4731fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681b0dc0 con 0x555bfdf150f0 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff4731fa740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681b06f0 con 0x555bfdf1e7e0 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff4731fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681ad930 con 0x555bfdf15a10 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff4731fa740 10 client.0 _ll_register_callbacks cb 0x7ff464009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46e3aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf26480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46dba9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 0x555bfdf23450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46ebab640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf23990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46e3aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf26480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33248/0 (socket says 172.21.15.16:33248) 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46e3aa640 1 -- 172.21.15.16:0/4236090299 learned_addr learned my addr 172.21.15.16:0/4236090299 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46dba9640 1 -- 172.21.15.16:0/4236090299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 msgr2=0x555bfdf23990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46dba9640 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf23990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46dba9640 1 -- 172.21.15.16:0/4236090299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 msgr2=0x555bfdf26480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46dba9640 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf26480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46dba9640 1 -- 172.21.15.16:0/4236090299 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bfdf26bf0 con 0x555bfdf15a10 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf26480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff46dba9640 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 0x555bfdf23450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff458009e70 tx=0x7ff458009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9a8caa06a7e75b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff451ffb640 1 -- 172.21.15.16:0/4236090299 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45802b040 con 0x555bfdf15a10 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff451ffb640 1 -- 172.21.15.16:0/4236090299 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff4580119b0 con 0x555bfdf15a10 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.956+0000 7ff4731fa740 1 -- 172.21.15.16:0/4236090299 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff464010430 con 0x555bfdf15a10 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.960+0000 7ff451ffb640 1 -- 172.21.15.16:0/4236090299 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff458044050 con 0x555bfdf15a10 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.960+0000 7ff4731fa740 10 client.25480 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.961+0000 7ff4731fa740 1 -- 172.21.15.16:0/4236090299 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bfdf95820 con 0x555bfdf15a10 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.961+0000 7ff451ffb640 1 -- 172.21.15.16:0/4236090299 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff464010430 con 0x555bfdf15a10 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.961+0000 7ff451ffb640 1 -- 172.21.15.16:0/4236090299 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff4580051d0 con 0x555bfdf15a10 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.961+0000 7ff4731fa740 10 client.25480 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.961+0000 7ff4731fa740 10 client.25480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.961+0000 7ff4731fa740 20 client.25480 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.961+0000 7ff4731fa740 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555bfdf95da0 0x555bfdf98190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.961+0000 7ff4731fa740 4 client.25480 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:44:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.961+0000 7ff4731fa740 1 -- 172.21.15.16:0/4236090299 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bfde806e0 con 0x555bfdf95da0 2024-03-27T06:44:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.961+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555bfdf95da0 0x555bfdf98190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.962+0000 7ff46e3aa640 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555bfdf95da0 0x555bfdf98190 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9802e6856cced75 server_cookie=7ca6f7ee79247e8c in_seq=0 out_seq=0 2024-03-27T06:44:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.962+0000 7ff451ffb640 10 client.25480 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff451ffb640 1 -- 172.21.15.16:0/4236090299 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bfde806e0 con 0x555bfdf95da0 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff451ffb640 10 client.25480 handle_command_reply: tid=0 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff4731fa740 1 client.25480 shutdown 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff4731fa740 1 -- 172.21.15.16:0/4236090299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 msgr2=0x555bfdf23450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff4731fa740 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 0x555bfdf23450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff458009e70 tx=0x7ff458009b50 comp rx=0 tx=0).stop 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff4731fa740 1 -- 172.21.15.16:0/4236090299 shutdown_connections 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff4731fa740 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555bfdf95da0 0x555bfdf98190 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff4731fa740 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555bfdf1e7e0 0x555bfdf23990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff4731fa740 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bfdf15a10 0x555bfdf23450 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff4731fa740 1 --2- 172.21.15.16:0/4236090299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555bfdf150f0 0x555bfdf26480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff451ffb640 0 client.25480 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff4731fa740 1 -- 172.21.15.16:0/4236090299 >> 172.21.15.16:0/4236090299 conn(0x555bfdf0fa10 msgr2=0x555bfdf1b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff4731fa740 1 -- 172.21.15.16:0/4236090299 shutdown_connections 2024-03-27T06:44:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.968+0000 7ff4731fa740 1 -- 172.21.15.16:0/4236090299 wait complete. 2024-03-27T06:44:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff4731fa740 20 client.25480 trim_cache size 0 max 16384 2024-03-27T06:44:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:44:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:44:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:44:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:44:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 -- 172.21.15.16:0/1150980845 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff4540a3a50 msgr2=0x7ff4540a5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff4540a3a50 0x7ff4540a5f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff45800fc90 tx=0x7ff45801e000 comp rx=0 tx=0).stop 2024-03-27T06:44:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 -- 172.21.15.16:0/1150980845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468106a90 msgr2=0x7ff468102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468106a90 0x7ff468102ef0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff46002c700 tx=0x7ff460034440 comp rx=0 tx=0).stop 2024-03-27T06:44:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 -- 172.21.15.16:0/1150980845 shutdown_connections 2024-03-27T06:44:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468106a90 0x7ff468102ef0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff4540a3a50 0x7ff4540a5f10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4681016d0 0x7ff4681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 --2- 172.21.15.16:0/1150980845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff468100d30 0x7ff468102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 -- 172.21.15.16:0/1150980845 >> 172.21.15.16:0/1150980845 conn(0x7ff468077770 msgr2=0x7ff468104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 -- 172.21.15.16:0/1150980845 shutdown_connections 2024-03-27T06:44:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:08.969+0000 7ff47060e640 1 -- 172.21.15.16:0/1150980845 wait complete. 2024-03-27T06:44:09.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:08 smithi156 ceph-mon[36261]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-03-27T06:44:09.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:08 smithi170 ceph-mon[36438]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-03-27T06:44:09.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:08 smithi016 ceph-mon[31908]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-03-27T06:44:11.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:10 smithi156 ceph-mon[36261]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-03-27T06:44:11.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:10 smithi170 ceph-mon[36438]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-03-27T06:44:11.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T06:44:11.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T06:44:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:10 smithi016 ceph-mon[31908]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-03-27T06:44:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.483+0000 7f545b313640 1 -- 172.21.15.16:0/1569698846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5454104610 msgr2=0x7f5454104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.483+0000 7f545b313640 1 --2- 172.21.15.16:0/1569698846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5454104610 0x7f5454104a90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5450009cd0 tx=0x7f545002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.483+0000 7f545b313640 1 -- 172.21.15.16:0/1569698846 shutdown_connections 2024-03-27T06:44:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.483+0000 7f545b313640 1 --2- 172.21.15.16:0/1569698846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5454104610 0x7f5454104a90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.483+0000 7f545b313640 1 --2- 172.21.15.16:0/1569698846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5454069290 0x7f54540696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.483+0000 7f545b313640 1 --2- 172.21.15.16:0/1569698846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5454106aa0 0x7f5454068d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.483+0000 7f545b313640 1 -- 172.21.15.16:0/1569698846 >> 172.21.15.16:0/1569698846 conn(0x7f5454077b50 msgr2=0x7f5454077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.483+0000 7f545b313640 1 -- 172.21.15.16:0/1569698846 shutdown_connections 2024-03-27T06:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.483+0000 7f545b313640 1 -- 172.21.15.16:0/1569698846 wait complete. 2024-03-27T06:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.483+0000 7f545b313640 1 Processor -- start 2024-03-27T06:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.483+0000 7f545b313640 1 -- start start 2024-03-27T06:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f545b313640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5454069290 0x7f54541a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f545b313640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5454104610 0x7f54541a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f545b313640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5454106aa0 0x7f545419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f545b313640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5454118610 con 0x7f5454069290 2024-03-27T06:44:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f545b313640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5454118490 con 0x7f5454106aa0 2024-03-27T06:44:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f545b313640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5454118790 con 0x7f5454104610 2024-03-27T06:44:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f54590af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5454069290 0x7f54541a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f54590af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5454069290 0x7f54541a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33260/0 (socket says 172.21.15.16:33260) 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f54590af640 1 -- 172.21.15.16:0/1128280448 learned_addr learned my addr 172.21.15.16:0/1128280448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f54598b0640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5454106aa0 0x7f545419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f54588ae640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5454104610 0x7f54541a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f54590af640 1 -- 172.21.15.16:0/1128280448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5454104610 msgr2=0x7f54541a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f54590af640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5454104610 0x7f54541a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f54590af640 1 -- 172.21.15.16:0/1128280448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5454106aa0 msgr2=0x7f545419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f54590af640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5454106aa0 0x7f545419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f54590af640 1 -- 172.21.15.16:0/1128280448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545419efe0 con 0x7f5454069290 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f54590af640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5454069290 0x7f54541a46a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f544c00e9f0 tx=0x7f544c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f544a7fc640 1 -- 172.21.15.16:0/1128280448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c00cd80 con 0x7f5454069290 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f545b313640 1 -- 172.21.15.16:0/1128280448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545419f2d0 con 0x7f5454069290 2024-03-27T06:44:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f544a7fc640 1 -- 172.21.15.16:0/1128280448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f544c010930 con 0x7f5454069290 2024-03-27T06:44:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f544a7fc640 1 -- 172.21.15.16:0/1128280448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c018b00 con 0x7f5454069290 2024-03-27T06:44:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.484+0000 7f545b313640 1 -- 172.21.15.16:0/1128280448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54541ab4f0 con 0x7f5454069290 2024-03-27T06:44:11.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.487+0000 7f544a7fc640 1 -- 172.21.15.16:0/1128280448 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7f544c018ca0 con 0x7f5454069290 2024-03-27T06:44:11.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.487+0000 7f545dfdc740 1 Processor -- start 2024-03-27T06:44:11.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.487+0000 7f545dfdc740 1 -- start start 2024-03-27T06:44:11.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.487+0000 7f544a7fc640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f54400a3930 0x7f54400a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.487+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356c250 0x55e36356c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.487+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356cb70 0x55e363575210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.487+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e363575750 0x55e363577b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.487+0000 7f545dfdc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e3634d0860 con 0x55e36356cb70 2024-03-27T06:44:11.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.487+0000 7f545dfdc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e3634788d0 con 0x55e36356c250 2024-03-27T06:44:11.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.487+0000 7f545dfdc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e3634cc810 con 0x55e363575750 2024-03-27T06:44:11.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54598b0640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f54400a3930 0x7f54400a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f544a7fc640 1 -- 172.21.15.16:0/1128280448 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f544c0d0730 con 0x7f5454069290 2024-03-27T06:44:11.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54588ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356cb70 0x55e363575210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54590af640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356c250 0x55e36356c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54598b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e363575750 0x55e363577b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54588ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356cb70 0x55e363575210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33262/0 (socket says 172.21.15.16:33262) 2024-03-27T06:44:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54588ae640 1 -- 172.21.15.16:0/2456074940 learned_addr learned my addr 172.21.15.16:0/2456074940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54590af640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356c250 0x55e36356c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60210/0 (socket says 172.21.15.16:60210) 2024-03-27T06:44:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54588ae640 1 -- 172.21.15.16:0/2456074940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e363575750 msgr2=0x55e363577b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54588ae640 1 --2- 172.21.15.16:0/2456074940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e363575750 0x55e363577b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54588ae640 1 -- 172.21.15.16:0/2456074940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356c250 msgr2=0x55e36356c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54588ae640 1 --2- 172.21.15.16:0/2456074940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356c250 0x55e36356c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54588ae640 1 -- 172.21.15.16:0/2456074940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e363483120 con 0x55e36356cb70 2024-03-27T06:44:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54590af640 1 --2- 172.21.15.16:0/2456074940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356c250 0x55e36356c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54598b0640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f54400a3930 0x7f54400a5df0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5450005cc0 tx=0x7f5450032d30 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f54588ae640 1 --2- 172.21.15.16:0/2456074940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356cb70 0x55e363575210 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5444009c60 tx=0x7f5444011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9928f6283c09bb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f543f7fe640 1 -- 172.21.15.16:0/2456074940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444004250 con 0x55e36356cb70 2024-03-27T06:44:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f543f7fe640 1 -- 172.21.15.16:0/2456074940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5444009e10 con 0x55e36356cb70 2024-03-27T06:44:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.488+0000 7f543f7fe640 1 -- 172.21.15.16:0/2456074940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444019830 con 0x55e36356cb70 2024-03-27T06:44:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.489+0000 7f545dfdc740 1 -- 172.21.15.16:0/2456074940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356cb70 msgr2=0x55e363575210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.489+0000 7f545dfdc740 1 --2- 172.21.15.16:0/2456074940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356cb70 0x55e363575210 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5444009c60 tx=0x7f5444011810 comp rx=0 tx=0).stop 2024-03-27T06:44:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.489+0000 7f545dfdc740 1 -- 172.21.15.16:0/2456074940 shutdown_connections 2024-03-27T06:44:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.489+0000 7f545dfdc740 1 --2- 172.21.15.16:0/2456074940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e363575750 0x55e363577b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.489+0000 7f545dfdc740 1 --2- 172.21.15.16:0/2456074940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356cb70 0x55e363575210 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.489+0000 7f545dfdc740 1 --2- 172.21.15.16:0/2456074940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356c250 0x55e36356c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.489+0000 7f545dfdc740 1 -- 172.21.15.16:0/2456074940 >> 172.21.15.16:0/2456074940 conn(0x55e363566980 msgr2=0x55e363566d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.489+0000 7f545dfdc740 1 -- 172.21.15.16:0/2456074940 shutdown_connections 2024-03-27T06:44:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.489+0000 7f545dfdc740 1 -- 172.21.15.16:0/2456074940 wait complete. 2024-03-27T06:44:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.489+0000 7f545dfdc740 1 -- start start 2024-03-27T06:44:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f545dfdc740 1 Processor -- start 2024-03-27T06:44:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363572750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356cb70 0x55e363572c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e363575750 0x55e3635731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f545dfdc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e363578540 con 0x55e36356c250 2024-03-27T06:44:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f545dfdc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e36348bcd0 con 0x55e36356cb70 2024-03-27T06:44:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f545dfdc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e3635786c0 con 0x55e363575750 2024-03-27T06:44:11.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f545dfdc740 10 client.0 _ll_register_callbacks cb 0x55e36347dee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:11.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f54590af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363572750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f54590af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363572750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33272/0 (socket says 172.21.15.16:33272) 2024-03-27T06:44:11.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f54590af640 1 -- 172.21.15.16:0/2281032143 learned_addr learned my addr 172.21.15.16:0/2281032143 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:11.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f54588ae640 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356cb70 0x55e363572c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f54598b0640 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e363575750 0x55e3635731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f54590af640 1 -- 172.21.15.16:0/2281032143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e363575750 msgr2=0x55e3635731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:11.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f54590af640 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e363575750 0x55e3635731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f54590af640 1 -- 172.21.15.16:0/2281032143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356cb70 msgr2=0x55e363572c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:11.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f54590af640 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356cb70 0x55e363572c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.490+0000 7f54590af640 1 -- 172.21.15.16:0/2281032143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3632ae7d0 con 0x55e36356c250 2024-03-27T06:44:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f54590af640 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363572750 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f544c00cfd0 tx=0x7f544c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2034a7c4c112b0a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f543cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f543cff9640 1 -- 172.21.15.16:0/2281032143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c087420 con 0x55e36356c250 2024-03-27T06:44:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f543cff9640 1 -- 172.21.15.16:0/2281032143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f544c010ae0 con 0x55e36356c250 2024-03-27T06:44:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f545dfdc740 1 -- 172.21.15.16:0/2281032143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5444009e10 con 0x55e36356c250 2024-03-27T06:44:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f543cff9640 1 -- 172.21.15.16:0/2281032143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f544c010ce0 con 0x55e36356c250 2024-03-27T06:44:11.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f545dfdc740 10 client.15654 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f545dfdc740 1 -- 172.21.15.16:0/2281032143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3631e43f0 con 0x55e36356c250 2024-03-27T06:44:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f543cff9640 1 -- 172.21.15.16:0/2281032143 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5444009e10 con 0x55e36356c250 2024-03-27T06:44:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f543cff9640 1 -- 172.21.15.16:0/2281032143 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f544c0984b0 con 0x55e36356c250 2024-03-27T06:44:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f545dfdc740 10 client.15654 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f545dfdc740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f545dfdc740 20 client.15654 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f545dfdc740 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e363585a40 0x55e363587ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f545dfdc740 4 client.15654 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f545dfdc740 1 -- 172.21.15.16:0/2281032143 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3634ddda0 con 0x55e363585a40 2024-03-27T06:44:11.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.491+0000 7f54588ae640 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e363585a40 0x55e363587ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.492+0000 7f543cff9640 10 client.15654 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.492+0000 7f54588ae640 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e363585a40 0x55e363587ed0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8332ab0eda6bae4 server_cookie=5395304600d52291 in_seq=0 out_seq=0 2024-03-27T06:44:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f543cff9640 1 -- 172.21.15.16:0/2281032143 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e3634ddda0 con 0x55e363585a40 2024-03-27T06:44:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f543cff9640 10 client.15654 handle_command_reply: tid=0 2024-03-27T06:44:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 1 client.15654 shutdown 2024-03-27T06:44:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 1 -- 172.21.15.16:0/2281032143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 msgr2=0x55e363572750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363572750 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f544c00cfd0 tx=0x7f544c000bc0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 1 -- 172.21.15.16:0/2281032143 shutdown_connections 2024-03-27T06:44:11.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e363585a40 0x55e363587ed0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e363575750 0x55e3635731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356cb70 0x55e363572c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 1 --2- 172.21.15.16:0/2281032143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363572750 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f543cff9640 0 client.15654 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 1 -- 172.21.15.16:0/2281032143 >> 172.21.15.16:0/2281032143 conn(0x55e363566980 msgr2=0x55e3635686d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 1 -- 172.21.15.16:0/2281032143 shutdown_connections 2024-03-27T06:44:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 1 -- 172.21.15.16:0/2281032143 wait complete. 2024-03-27T06:44:11.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.493+0000 7f545dfdc740 20 client.15654 trim_cache size 0 max 16384 2024-03-27T06:44:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f545dfdc740 1 Processor -- start 2024-03-27T06:44:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f545dfdc740 1 -- start start 2024-03-27T06:44:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363574e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356cb70 0x55e3635812e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e363575750 0x55e3635657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f545dfdc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54541b3e10 con 0x55e36356c250 2024-03-27T06:44:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f545dfdc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f54541b5950 con 0x55e363575750 2024-03-27T06:44:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f545dfdc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f54541b44e0 con 0x55e36356cb70 2024-03-27T06:44:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54590af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363574e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54590af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363574e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33282/0 (socket says 172.21.15.16:33282) 2024-03-27T06:44:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54590af640 1 -- 172.21.15.16:0/31486994 learned_addr learned my addr 172.21.15.16:0/31486994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54588ae640 1 --2- 172.21.15.16:0/31486994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356cb70 0x55e3635812e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54598b0640 1 --2- 172.21.15.16:0/31486994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e363575750 0x55e3635657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54588ae640 1 -- 172.21.15.16:0/31486994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e363575750 msgr2=0x55e3635657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54588ae640 1 --2- 172.21.15.16:0/31486994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e363575750 0x55e3635657a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54588ae640 1 -- 172.21.15.16:0/31486994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 msgr2=0x55e363574e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54588ae640 1 --2- 172.21.15.16:0/31486994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363574e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54588ae640 1 -- 172.21.15.16:0/31486994 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3631a40f0 con 0x55e36356cb70 2024-03-27T06:44:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54590af640 1 --2- 172.21.15.16:0/31486994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363574e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.511+0000 7f54588ae640 1 --2- 172.21.15.16:0/31486994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356cb70 0x55e3635812e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5444011d10 tx=0x7f5444009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e764e7d385f04ce server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f543effd640 1 -- 172.21.15.16:0/31486994 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544401f040 con 0x55e36356cb70 2024-03-27T06:44:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f543effd640 1 -- 172.21.15.16:0/31486994 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f54440035f0 con 0x55e36356cb70 2024-03-27T06:44:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f543effd640 1 -- 172.21.15.16:0/31486994 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444019720 con 0x55e36356cb70 2024-03-27T06:44:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f545dfdc740 1 -- 172.21.15.16:0/31486994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356cb70 msgr2=0x55e3635812e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f545dfdc740 1 --2- 172.21.15.16:0/31486994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356cb70 0x55e3635812e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5444011d10 tx=0x7f5444009f00 comp rx=0 tx=0).stop 2024-03-27T06:44:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f545dfdc740 1 -- 172.21.15.16:0/31486994 shutdown_connections 2024-03-27T06:44:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f545dfdc740 1 --2- 172.21.15.16:0/31486994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e363575750 0x55e3635657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f545dfdc740 1 --2- 172.21.15.16:0/31486994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356cb70 0x55e3635812e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f545dfdc740 1 --2- 172.21.15.16:0/31486994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e36356c250 0x55e363574e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f545dfdc740 1 -- 172.21.15.16:0/31486994 >> 172.21.15.16:0/31486994 conn(0x55e363566980 msgr2=0x55e363565090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f545dfdc740 1 -- 172.21.15.16:0/31486994 shutdown_connections 2024-03-27T06:44:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f545dfdc740 1 -- 172.21.15.16:0/31486994 wait complete. 2024-03-27T06:44:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f545dfdc740 1 -- start start 2024-03-27T06:44:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.512+0000 7f545dfdc740 1 Processor -- start 2024-03-27T06:44:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356c250 0x55e36357dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356cb70 0x55e363579010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f545dfdc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e363575750 0x55e363579590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f545dfdc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54541b5280 con 0x55e363575750 2024-03-27T06:44:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f545dfdc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f54541b4bb0 con 0x55e36356cb70 2024-03-27T06:44:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f545dfdc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f54541b1df0 con 0x55e36356c250 2024-03-27T06:44:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f545dfdc740 10 client.0 _ll_register_callbacks cb 0x7f544c098a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54590af640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356c250 0x55e36357dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54598b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e363575750 0x55e363579590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54590af640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356c250 0x55e36357dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:40554/0 (socket says 172.21.15.16:40554) 2024-03-27T06:44:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54590af640 1 -- 172.21.15.16:0/1578641937 learned_addr learned my addr 172.21.15.16:0/1578641937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54588ae640 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356cb70 0x55e363579010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54598b0640 1 -- 172.21.15.16:0/1578641937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356c250 msgr2=0x55e36357dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54598b0640 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356c250 0x55e36357dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54598b0640 1 -- 172.21.15.16:0/1578641937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356cb70 msgr2=0x55e363579010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54598b0640 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356cb70 0x55e363579010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54598b0640 1 -- 172.21.15.16:0/1578641937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e363579dc0 con 0x55e363575750 2024-03-27T06:44:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54590af640 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356c250 0x55e36357dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54588ae640 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356cb70 0x55e363579010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f54598b0640 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e363575750 0x55e363579590 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f545002c630 tx=0x7f545002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deefd6752c1865d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.513+0000 7f543cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f543cff9640 1 -- 172.21.15.16:0/1578641937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450039040 con 0x55e363575750 2024-03-27T06:44:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f543cff9640 1 -- 172.21.15.16:0/1578641937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5450004790 con 0x55e363575750 2024-03-27T06:44:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f545dfdc740 1 -- 172.21.15.16:0/1578641937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54440035f0 con 0x55e363575750 2024-03-27T06:44:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f543cff9640 1 -- 172.21.15.16:0/1578641937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f545005d050 con 0x55e363575750 2024-03-27T06:44:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f545dfdc740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f545dfdc740 1 -- 172.21.15.16:0/1578641937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3635ebea0 con 0x55e363575750 2024-03-27T06:44:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f543cff9640 1 -- 172.21.15.16:0/1578641937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f54440035f0 con 0x55e363575750 2024-03-27T06:44:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f543cff9640 1 -- 172.21.15.16:0/1578641937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f545003f050 con 0x55e363575750 2024-03-27T06:44:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f545dfdc740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f545dfdc740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f545dfdc740 20 client.15660 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f545dfdc740 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e3635ec640 0x55e3635eea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f545dfdc740 4 client.15660 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T06:44:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f545dfdc740 1 -- 172.21.15.16:0/1578641937 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e3634d80f0 con 0x55e3635ec640 2024-03-27T06:44:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.514+0000 7f54590af640 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e3635ec640 0x55e3635eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f543cff9640 10 client.15660 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f54590af640 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e3635ec640 0x55e3635eea60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ee3bc1b3729ad0 server_cookie=98862162b9ddb79d in_seq=0 out_seq=0 2024-03-27T06:44:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f543cff9640 1 -- 172.21.15.16:0/1578641937 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e3634d80f0 con 0x55e3635ec640 2024-03-27T06:44:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f543cff9640 10 client.15660 handle_command_reply: tid=0 2024-03-27T06:44:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 1 client.15660 shutdown 2024-03-27T06:44:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 1 -- 172.21.15.16:0/1578641937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e363575750 msgr2=0x55e363579590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e363575750 0x55e363579590 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f545002c630 tx=0x7f545002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 1 -- 172.21.15.16:0/1578641937 shutdown_connections 2024-03-27T06:44:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e3635ec640 0x55e3635eea60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e363575750 0x55e363579590 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e36356cb70 0x55e363579010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 1 --2- 172.21.15.16:0/1578641937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e36356c250 0x55e36357dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f543cff9640 0 client.15660 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 1 -- 172.21.15.16:0/1578641937 >> 172.21.15.16:0/1578641937 conn(0x55e363566980 msgr2=0x55e363565920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 1 -- 172.21.15.16:0/1578641937 shutdown_connections 2024-03-27T06:44:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 1 -- 172.21.15.16:0/1578641937 wait complete. 2024-03-27T06:44:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.515+0000 7f545dfdc740 20 client.15660 trim_cache size 0 max 16384 2024-03-27T06:44:11.523 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:44:11.523 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-03-27T06:44:11.523 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-03-27T06:44:11.523 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:44:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 -- 172.21.15.16:0/1128280448 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f54400a3930 msgr2=0x7f54400a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f54400a3930 0x7f54400a5df0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5450005cc0 tx=0x7f5450032d30 comp rx=0 tx=0).stop 2024-03-27T06:44:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 -- 172.21.15.16:0/1128280448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5454069290 msgr2=0x7f54541a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5454069290 0x7f54541a46a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f544c00e9f0 tx=0x7f544c00eeb0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 -- 172.21.15.16:0/1128280448 shutdown_connections 2024-03-27T06:44:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5454106aa0 0x7f545419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f54400a3930 0x7f54400a5df0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5454104610 0x7f54541a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 --2- 172.21.15.16:0/1128280448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5454069290 0x7f54541a46a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 -- 172.21.15.16:0/1128280448 >> 172.21.15.16:0/1128280448 conn(0x7f5454077b50 msgr2=0x7f5454101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 -- 172.21.15.16:0/1128280448 shutdown_connections 2024-03-27T06:44:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.516+0000 7f545b313640 1 -- 172.21.15.16:0/1128280448 wait complete. 2024-03-27T06:44:11.527 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-27T06:44:11.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:44:11.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.637+0000 7fc1093ac640 1 -- 172.21.15.16:0/176455117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1041013b0 msgr2=0x7fc10410d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.637+0000 7fc1093ac640 1 --2- 172.21.15.16:0/176455117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1041013b0 0x7fc10410d8c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009ca0 tx=0x7fc0f402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.637+0000 7fc1093ac640 1 -- 172.21.15.16:0/176455117 shutdown_connections 2024-03-27T06:44:11.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.637+0000 7fc1093ac640 1 --2- 172.21.15.16:0/176455117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1041013b0 0x7fc10410d8c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.637+0000 7fc1093ac640 1 --2- 172.21.15.16:0/176455117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1041009f0 0x7fc104100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.637+0000 7fc1093ac640 1 --2- 172.21.15.16:0/176455117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc104106a00 0x7fc104106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.637+0000 7fc1093ac640 1 -- 172.21.15.16:0/176455117 >> 172.21.15.16:0/176455117 conn(0x7fc104076dd0 msgr2=0x7fc1040771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.637+0000 7fc1093ac640 1 -- 172.21.15.16:0/176455117 shutdown_connections 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.637+0000 7fc1093ac640 1 -- 172.21.15.16:0/176455117 wait complete. 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1093ac640 1 Processor -- start 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1093ac640 1 -- start start 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1093ac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1041009f0 0x7fc104199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1093ac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1041013b0 0x7fc104199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1093ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc104106a00 0x7fc1041a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1093ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc104110f30 con 0x7fc104106a00 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1093ac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fc104110db0 con 0x7fc1041009f0 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1093ac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc1041110b0 con 0x7fc1041013b0 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc104106a00 0x7fc1041a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc104106a00 0x7fc1041a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33294/0 (socket says 172.21.15.16:33294) 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1037fe640 1 -- 172.21.15.16:0/3770381439 learned_addr learned my addr 172.21.15.16:0/3770381439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc102ffd640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1041009f0 0x7fc104199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.638+0000 7fc1027fc640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1041013b0 0x7fc104199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.639+0000 7fc1037fe640 1 -- 172.21.15.16:0/3770381439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1041013b0 msgr2=0x7fc104199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.639+0000 7fc1037fe640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1041013b0 0x7fc104199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.639+0000 7fc1037fe640 1 -- 172.21.15.16:0/3770381439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1041009f0 msgr2=0x7fc104199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.639+0000 7fc1037fe640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1041009f0 0x7fc104199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.639+0000 7fc1037fe640 1 -- 172.21.15.16:0/3770381439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1041a47f0 con 0x7fc104106a00 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.639+0000 7fc1037fe640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc104106a00 0x7fc1041a42b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005e30 tx=0x7fc0f4034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.639+0000 7fc0fbfff640 1 -- 172.21.15.16:0/3770381439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4039050 con 0x7fc104106a00 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.639+0000 7fc0fbfff640 1 -- 172.21.15.16:0/3770381439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc0f4045070 con 0x7fc104106a00 2024-03-27T06:44:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.639+0000 7fc0fbfff640 1 -- 172.21.15.16:0/3770381439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4004630 con 0x7fc104106a00 2024-03-27T06:44:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.639+0000 7fc1093ac640 1 -- 172.21.15.16:0/3770381439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1041a49c0 con 0x7fc104106a00 2024-03-27T06:44:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.639+0000 7fc1093ac640 1 -- 172.21.15.16:0/3770381439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1041a4e80 con 0x7fc104106a00 2024-03-27T06:44:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.640+0000 7fc1093ac640 1 -- 172.21.15.16:0/3770381439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc104102130 con 0x7fc104106a00 2024-03-27T06:44:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.642+0000 7fc0fbfff640 1 -- 172.21.15.16:0/3770381439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7fc0f4003970 con 0x7fc104106a00 2024-03-27T06:44:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.642+0000 7fc0fbfff640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc0f00a3930 0x7fc0f00a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.642+0000 7fc102ffd640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc0f00a3930 0x7fc0f00a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.642+0000 7fc0fbfff640 1 -- 172.21.15.16:0/3770381439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc0f40f3600 con 0x7fc104106a00 2024-03-27T06:44:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.643+0000 7fc102ffd640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc0f00a3930 0x7fc0f00a5df0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009b50 tx=0x7fc0fc005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.646+0000 7fc0fbfff640 1 -- 172.21.15.16:0/3770381439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fc0f40bfd90 con 0x7fc104106a00 2024-03-27T06:44:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.874+0000 7fc1093ac640 1 -- 172.21.15.16:0/3770381439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc104068d90 con 0x7fc104106a00 2024-03-27T06:44:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.875+0000 7fc0fbfff640 1 -- 172.21.15.16:0/3770381439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11932 (secure 0 0 0) 0x7fc104068d90 con 0x7fc104106a00 2024-03-27T06:44:11.870 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:44:11.870 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:43:32.824696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24467,24527,24500,24472,24493]},"id":1}]} 2024-03-27T06:44:11.870 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-03-27T06:44:11.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 -- 172.21.15.16:0/3770381439 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc0f00a3930 msgr2=0x7fc0f00a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc0f00a3930 0x7fc0f00a5df0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009b50 tx=0x7fc0fc005f90 comp rx=0 tx=0).stop 2024-03-27T06:44:11.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 -- 172.21.15.16:0/3770381439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc104106a00 msgr2=0x7fc1041a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc104106a00 0x7fc1041a42b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005e30 tx=0x7fc0f4034440 comp rx=0 tx=0).stop 2024-03-27T06:44:11.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 -- 172.21.15.16:0/3770381439 shutdown_connections 2024-03-27T06:44:11.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc104106a00 0x7fc1041a42b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc0f00a3930 0x7fc0f00a5df0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1041013b0 0x7fc104199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 --2- 172.21.15.16:0/3770381439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1041009f0 0x7fc104199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 -- 172.21.15.16:0/3770381439 >> 172.21.15.16:0/3770381439 conn(0x7fc104076dd0 msgr2=0x7fc1040fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:11.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 -- 172.21.15.16:0/3770381439 shutdown_connections 2024-03-27T06:44:11.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.877+0000 7fc1093ac640 1 -- 172.21.15.16:0/3770381439 wait complete. 2024-03-27T06:44:11.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-27T06:44:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.994+0000 7fad57c4d640 1 -- 172.21.15.16:0/3485841569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50101af0 msgr2=0x7fad50101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.994+0000 7fad57c4d640 1 --2- 172.21.15.16:0/3485841569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50101af0 0x7fad50101f70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fad44009930 tx=0x7fad4402bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.994+0000 7fad57c4d640 1 -- 172.21.15.16:0/3485841569 shutdown_connections 2024-03-27T06:44:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 --2- 172.21.15.16:0/3485841569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad501024b0 0x7fad5010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 --2- 172.21.15.16:0/3485841569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50101af0 0x7fad50101f70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 --2- 172.21.15.16:0/3485841569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad50107b90 0x7fad50107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 -- 172.21.15.16:0/3485841569 >> 172.21.15.16:0/3485841569 conn(0x7fad500fd580 msgr2=0x7fad500ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 -- 172.21.15.16:0/3485841569 shutdown_connections 2024-03-27T06:44:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 -- 172.21.15.16:0/3485841569 wait complete. 2024-03-27T06:44:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 Processor -- start 2024-03-27T06:44:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 -- start start 2024-03-27T06:44:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad50101af0 0x7fad501984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad501024b0 0x7fad50198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad50107b90 0x7fad5019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:11.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad50111fb0 con 0x7fad501024b0 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fad50111e30 con 0x7fad50107b90 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.995+0000 7fad57c4d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fad50112130 con 0x7fad50101af0 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad551e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad501024b0 0x7fad50198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad551e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad501024b0 0x7fad50198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33310/0 (socket says 172.21.15.16:33310) 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad551e8640 1 -- 172.21.15.16:0/3287808881 learned_addr learned my addr 172.21.15.16:0/3287808881 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad559e9640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad50101af0 0x7fad501984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad561ea640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad50107b90 0x7fad5019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad551e8640 1 -- 172.21.15.16:0/3287808881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad50101af0 msgr2=0x7fad501984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad551e8640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad50101af0 0x7fad501984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad551e8640 1 -- 172.21.15.16:0/3287808881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad50107b90 msgr2=0x7fad5019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad551e8640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad50107b90 0x7fad5019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad551e8640 1 -- 172.21.15.16:0/3287808881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5019d4e0 con 0x7fad501024b0 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad551e8640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad501024b0 0x7fad50198a30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fad44009230 tx=0x7fad4402dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad4effd640 1 -- 172.21.15.16:0/3287808881 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44043020 con 0x7fad501024b0 2024-03-27T06:44:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad4effd640 1 -- 172.21.15.16:0/3287808881 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad44004250 con 0x7fad501024b0 2024-03-27T06:44:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad57c4d640 1 -- 172.21.15.16:0/3287808881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5019d770 con 0x7fad501024b0 2024-03-27T06:44:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad4effd640 1 -- 172.21.15.16:0/3287808881 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44003970 con 0x7fad501024b0 2024-03-27T06:44:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.996+0000 7fad57c4d640 1 -- 172.21.15.16:0/3287808881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad501a5060 con 0x7fad501024b0 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.998+0000 7fad4effd640 1 -- 172.21.15.16:0/3287808881 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7fad44046020 con 0x7fad501024b0 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.999+0000 7fad5a830740 1 Processor -- start 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.999+0000 7fad5a830740 1 -- start start 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.999+0000 7fad5a830740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 0x556194832b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.999+0000 7fad4effd640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fad380a39b0 0x7fad380a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.999+0000 7fad5a830740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 0x55619483b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.999+0000 7fad5a830740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x55619483e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.999+0000 7fad5a830740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556194796710 con 0x5561948330d0 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.999+0000 7fad5a830740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55619473eaf0 con 0x5561948327b0 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.999+0000 7fad5a830740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5561947926c0 con 0x55619483bcb0 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:11.999+0000 7fad561ea640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fad380a39b0 0x7fad380a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad551e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 0x55619483b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad559e9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 0x556194832b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad559e9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 0x556194832b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60276/0 (socket says 172.21.15.16:60276) 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad559e9640 1 -- 172.21.15.16:0/4248998535 learned_addr learned my addr 172.21.15.16:0/4248998535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad561ea640 1 --2- 172.21.15.16:0/4248998535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x55619483e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad4effd640 1 -- 172.21.15.16:0/3287808881 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fad440f21c0 con 0x7fad501024b0 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad559e9640 1 -- 172.21.15.16:0/4248998535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 msgr2=0x55619483e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad559e9640 1 --2- 172.21.15.16:0/4248998535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x55619483e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad559e9640 1 -- 172.21.15.16:0/4248998535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 msgr2=0x55619483b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad559e9640 1 --2- 172.21.15.16:0/4248998535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 0x55619483b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad559e9640 1 -- 172.21.15.16:0/4248998535 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561944aa3f0 con 0x5561948327b0 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad561ea640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fad380a39b0 0x7fad380a5e70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad40005fd0 tx=0x7fad40009b80 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad559e9640 1 --2- 172.21.15.16:0/4248998535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 0x556194832b90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad48009c70 tx=0x7fad48011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb00871d534df409 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad37fff640 1 -- 172.21.15.16:0/4248998535 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4801e070 con 0x5561948327b0 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad37fff640 1 -- 172.21.15.16:0/4248998535 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad48009e20 con 0x5561948327b0 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.000+0000 7fad37fff640 1 -- 172.21.15.16:0/4248998535 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48019720 con 0x5561948327b0 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.001+0000 7fad5a830740 1 -- 172.21.15.16:0/4248998535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 msgr2=0x556194832b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.001+0000 7fad5a830740 1 --2- 172.21.15.16:0/4248998535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 0x556194832b90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad48009c70 tx=0x7fad48011810 comp rx=0 tx=0).stop 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.001+0000 7fad5a830740 1 -- 172.21.15.16:0/4248998535 shutdown_connections 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.001+0000 7fad5a830740 1 --2- 172.21.15.16:0/4248998535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x55619483e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.001+0000 7fad5a830740 1 --2- 172.21.15.16:0/4248998535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 0x55619483b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.001+0000 7fad5a830740 1 --2- 172.21.15.16:0/4248998535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 0x556194832b90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.001+0000 7fad5a830740 1 -- 172.21.15.16:0/4248998535 >> 172.21.15.16:0/4248998535 conn(0x55619482cee0 msgr2=0x55619482d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.001+0000 7fad5a830740 1 -- 172.21.15.16:0/4248998535 shutdown_connections 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.001+0000 7fad5a830740 1 -- 172.21.15.16:0/4248998535 wait complete. 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.001+0000 7fad5a830740 1 -- start start 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.001+0000 7fad5a830740 1 Processor -- start 2024-03-27T06:44:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad5a830740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 0x556194838cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad5a830740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 0x5561948391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad5a830740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x556194839730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad5a830740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55619483eaa0 con 0x5561948330d0 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad5a830740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x556194751eb0 con 0x5561948327b0 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad5a830740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55619483ec20 con 0x55619483bcb0 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad5a830740 10 client.0 _ll_register_callbacks cb 0x55619478e6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad551e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 0x5561948391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad551e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 0x5561948391f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33332/0 (socket says 172.21.15.16:33332) 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad551e8640 1 -- 172.21.15.16:0/1146087380 learned_addr learned my addr 172.21.15.16:0/1146087380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad559e9640 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 0x556194838cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad561ea640 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x556194839730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad551e8640 1 -- 172.21.15.16:0/1146087380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 msgr2=0x556194839730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad551e8640 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x556194839730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad551e8640 1 -- 172.21.15.16:0/1146087380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 msgr2=0x556194838cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad551e8640 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 0x556194838cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad551e8640 1 -- 172.21.15.16:0/1146087380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561945747d0 con 0x5561948330d0 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad551e8640 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 0x5561948391f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fad44009a50 tx=0x7fad4404fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02e1c7bc5eb15e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.002+0000 7fad357fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad357fa640 1 -- 172.21.15.16:0/1146087380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad440d2070 con 0x5561948330d0 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad357fa640 1 -- 172.21.15.16:0/1146087380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fad440bc590 con 0x5561948330d0 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad5a830740 1 -- 172.21.15.16:0/1146087380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad48009e20 con 0x5561948330d0 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad357fa640 1 -- 172.21.15.16:0/1146087380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad440be990 con 0x5561948330d0 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad5a830740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad5a830740 1 -- 172.21.15.16:0/1146087380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55619484ad60 con 0x5561948330d0 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad357fa640 1 -- 172.21.15.16:0/1146087380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad48009e20 con 0x5561948330d0 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad357fa640 1 -- 172.21.15.16:0/1146087380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fad4403ac80 con 0x5561948330d0 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad5a830740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:12.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad5a830740 10 client.15678 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad5a830740 20 client.15678 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad5a830740 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55619484c210 0x55619484e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad5a830740 4 client.15678 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad5a830740 1 -- 172.21.15.16:0/1146087380 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561947a3c50 con 0x55619484c210 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.003+0000 7fad559e9640 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55619484c210 0x55619484e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.004+0000 7fad357fa640 10 client.15678 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.004+0000 7fad559e9640 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55619484c210 0x55619484e6a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3030b911dcca69ef server_cookie=3bab77d67de42802 in_seq=0 out_seq=0 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.004+0000 7fad357fa640 1 -- 172.21.15.16:0/1146087380 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5561947a3c50 con 0x55619484c210 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.004+0000 7fad357fa640 10 client.15678 handle_command_reply: tid=0 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 1 client.15678 shutdown 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 1 -- 172.21.15.16:0/1146087380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 msgr2=0x5561948391f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 0x5561948391f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fad44009a50 tx=0x7fad4404fd50 comp rx=0 tx=0).stop 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 1 -- 172.21.15.16:0/1146087380 shutdown_connections 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55619484c210 0x55619484e6a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x556194839730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948330d0 0x5561948391f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 1 --2- 172.21.15.16:0/1146087380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948327b0 0x556194838cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad357fa640 0 client.15678 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:44:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 1 -- 172.21.15.16:0/1146087380 >> 172.21.15.16:0/1146087380 conn(0x55619482cee0 msgr2=0x55619482ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 1 -- 172.21.15.16:0/1146087380 shutdown_connections 2024-03-27T06:44:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 1 -- 172.21.15.16:0/1146087380 wait complete. 2024-03-27T06:44:12.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.005+0000 7fad5a830740 20 client.15678 trim_cache size 0 max 16384 2024-03-27T06:44:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.018+0000 7fad5a830740 1 Processor -- start 2024-03-27T06:44:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.018+0000 7fad5a830740 1 -- start start 2024-03-27T06:44:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.018+0000 7fad5a830740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948327b0 0x55619483b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.018+0000 7fad5a830740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948330d0 0x5561948478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.018+0000 7fad5a830740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x55619482ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad5a830740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad501ad750 con 0x5561948327b0 2024-03-27T06:44:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad5a830740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fad501af290 con 0x5561948330d0 2024-03-27T06:44:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad5a830740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fad501ade20 con 0x55619483bcb0 2024-03-27T06:44:12.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad559e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948327b0 0x55619483b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad559e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948327b0 0x55619483b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33340/0 (socket says 172.21.15.16:33340) 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad559e9640 1 -- 172.21.15.16:0/4150916505 learned_addr learned my addr 172.21.15.16:0/4150916505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad561ea640 1 --2- 172.21.15.16:0/4150916505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x55619482ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad551e8640 1 --2- 172.21.15.16:0/4150916505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948330d0 0x5561948478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad559e9640 1 -- 172.21.15.16:0/4150916505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 msgr2=0x55619482ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad559e9640 1 --2- 172.21.15.16:0/4150916505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x55619482ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad559e9640 1 -- 172.21.15.16:0/4150916505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948330d0 msgr2=0x5561948478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad559e9640 1 --2- 172.21.15.16:0/4150916505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948330d0 0x5561948478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad559e9640 1 -- 172.21.15.16:0/4150916505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561948287a0 con 0x5561948327b0 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad551e8640 1 --2- 172.21.15.16:0/4150916505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948330d0 0x5561948478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad559e9640 1 --2- 172.21.15.16:0/4150916505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948327b0 0x55619483b3e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fad48012fd0 tx=0x7fad48003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98aa01f620281ce5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad377fe640 1 -- 172.21.15.16:0/4150916505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4801e040 con 0x5561948327b0 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad377fe640 1 -- 172.21.15.16:0/4150916505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad4802b680 con 0x5561948327b0 2024-03-27T06:44:12.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.019+0000 7fad377fe640 1 -- 172.21.15.16:0/4150916505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad480035f0 con 0x5561948327b0 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.020+0000 7fad5a830740 1 -- 172.21.15.16:0/4150916505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948327b0 msgr2=0x55619483b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.020+0000 7fad5a830740 1 --2- 172.21.15.16:0/4150916505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948327b0 0x55619483b3e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fad48012fd0 tx=0x7fad48003e50 comp rx=0 tx=0).stop 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.020+0000 7fad5a830740 1 -- 172.21.15.16:0/4150916505 shutdown_connections 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.020+0000 7fad5a830740 1 --2- 172.21.15.16:0/4150916505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55619483bcb0 0x55619482ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.020+0000 7fad5a830740 1 --2- 172.21.15.16:0/4150916505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948330d0 0x5561948478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.020+0000 7fad5a830740 1 --2- 172.21.15.16:0/4150916505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561948327b0 0x55619483b3e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.020+0000 7fad5a830740 1 -- 172.21.15.16:0/4150916505 >> 172.21.15.16:0/4150916505 conn(0x55619482cee0 msgr2=0x55619482e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.020+0000 7fad5a830740 1 -- 172.21.15.16:0/4150916505 shutdown_connections 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.020+0000 7fad5a830740 1 -- 172.21.15.16:0/4150916505 wait complete. 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.020+0000 7fad5a830740 1 -- start start 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.020+0000 7fad5a830740 1 Processor -- start 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad5a830740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561948327b0 0x55619483fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad5a830740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948330d0 0x556194840400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad5a830740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55619483bcb0 0x5561948b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad5a830740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad501aebc0 con 0x55619483bcb0 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad5a830740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fad501ae4f0 con 0x5561948330d0 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad5a830740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fad501ab730 con 0x5561948327b0 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad5a830740 10 client.0 _ll_register_callbacks cb 0x7fad44009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad561ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55619483bcb0 0x5561948b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad561ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55619483bcb0 0x5561948b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33352/0 (socket says 172.21.15.16:33352) 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad561ea640 1 -- 172.21.15.16:0/4028612749 learned_addr learned my addr 172.21.15.16:0/4028612749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad559e9640 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561948327b0 0x55619483fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad551e8640 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948330d0 0x556194840400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad561ea640 1 -- 172.21.15.16:0/4028612749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561948327b0 msgr2=0x55619483fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad561ea640 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561948327b0 0x55619483fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad561ea640 1 -- 172.21.15.16:0/4028612749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948330d0 msgr2=0x556194840400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad561ea640 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948330d0 0x556194840400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad561ea640 1 -- 172.21.15.16:0/4028612749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561948b1b20 con 0x55619483bcb0 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad561ea640 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55619483bcb0 0x5561948b15e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fad40013340 tx=0x7fad40004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf2cfc62a88734 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.021+0000 7fad357fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad357fa640 1 -- 172.21.15.16:0/4028612749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40030060 con 0x55619483bcb0 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad357fa640 1 -- 172.21.15.16:0/4028612749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fad4001c040 con 0x55619483bcb0 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad5a830740 1 -- 172.21.15.16:0/4028612749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad4802b680 con 0x55619483bcb0 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad357fa640 1 -- 172.21.15.16:0/4028612749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad40035480 con 0x55619483bcb0 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad5a830740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad5a830740 1 -- 172.21.15.16:0/4028612749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5561948b2dd0 con 0x55619483bcb0 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad357fa640 1 -- 172.21.15.16:0/4028612749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fad4802b680 con 0x55619483bcb0 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad357fa640 1 -- 172.21.15.16:0/4028612749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fad4002a070 con 0x55619483bcb0 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad5a830740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad5a830740 10 client.15684 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad5a830740 20 client.15684 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad5a830740 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x5561948b3580 0x5561948b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad5a830740 4 client.15684 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:44:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad5a830740 1 -- 172.21.15.16:0/4028612749 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55619479dfa0 con 0x5561948b3580 2024-03-27T06:44:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.022+0000 7fad559e9640 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x5561948b3580 0x5561948b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad357fa640 10 client.15684 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:44:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad559e9640 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x5561948b3580 0x5561948b59a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cfc9cca0540ead86 server_cookie=5c2ac7b90892cb9d in_seq=0 out_seq=0 2024-03-27T06:44:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad357fa640 1 -- 172.21.15.16:0/4028612749 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55619479dfa0 con 0x5561948b3580 2024-03-27T06:44:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad357fa640 10 client.15684 handle_command_reply: tid=0 2024-03-27T06:44:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 1 client.15684 shutdown 2024-03-27T06:44:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 1 -- 172.21.15.16:0/4028612749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55619483bcb0 msgr2=0x5561948b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55619483bcb0 0x5561948b15e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fad40013340 tx=0x7fad40004ab0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 1 -- 172.21.15.16:0/4028612749 shutdown_connections 2024-03-27T06:44:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x5561948b3580 0x5561948b59a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55619483bcb0 0x5561948b15e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561948330d0 0x556194840400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad357fa640 0 client.15684 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:44:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 1 --2- 172.21.15.16:0/4028612749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561948327b0 0x55619483fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 1 -- 172.21.15.16:0/4028612749 >> 172.21.15.16:0/4028612749 conn(0x55619482cee0 msgr2=0x55619482eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 1 -- 172.21.15.16:0/4028612749 shutdown_connections 2024-03-27T06:44:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 1 -- 172.21.15.16:0/4028612749 wait complete. 2024-03-27T06:44:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.023+0000 7fad5a830740 20 client.15684 trim_cache size 0 max 16384 2024-03-27T06:44:12.025 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:44:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 -- 172.21.15.16:0/3287808881 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fad380a39b0 msgr2=0x7fad380a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fad380a39b0 0x7fad380a5e70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad40005fd0 tx=0x7fad40009b80 comp rx=0 tx=0).stop 2024-03-27T06:44:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 -- 172.21.15.16:0/3287808881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad501024b0 msgr2=0x7fad50198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad501024b0 0x7fad50198a30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fad44009230 tx=0x7fad4402dc90 comp rx=0 tx=0).stop 2024-03-27T06:44:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 -- 172.21.15.16:0/3287808881 shutdown_connections 2024-03-27T06:44:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad50107b90 0x7fad5019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad501024b0 0x7fad50198a30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fad380a39b0 0x7fad380a5e70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 --2- 172.21.15.16:0/3287808881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad50101af0 0x7fad501984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 -- 172.21.15.16:0/3287808881 >> 172.21.15.16:0/3287808881 conn(0x7fad500fd580 msgr2=0x7fad500fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 -- 172.21.15.16:0/3287808881 shutdown_connections 2024-03-27T06:44:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.024+0000 7fad57c4d640 1 -- 172.21.15.16:0/3287808881 wait complete. 2024-03-27T06:44:12.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-27T06:44:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.141+0000 7fae2fbe2640 1 -- 172.21.15.16:0/686352199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae281009c0 msgr2=0x7fae28100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.141+0000 7fae2fbe2640 1 --2- 172.21.15.16:0/686352199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae281009c0 0x7fae28100e40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fae18009930 tx=0x7fae1802bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.141+0000 7fae2fbe2640 1 -- 172.21.15.16:0/686352199 shutdown_connections 2024-03-27T06:44:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.141+0000 7fae2fbe2640 1 --2- 172.21.15.16:0/686352199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae28101380 0x7fae2810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.141+0000 7fae2fbe2640 1 --2- 172.21.15.16:0/686352199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae281009c0 0x7fae28100e40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.141+0000 7fae2fbe2640 1 --2- 172.21.15.16:0/686352199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae281069d0 0x7fae28106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.141+0000 7fae2fbe2640 1 -- 172.21.15.16:0/686352199 >> 172.21.15.16:0/686352199 conn(0x7fae28077b50 msgr2=0x7fae28077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.141+0000 7fae2fbe2640 1 -- 172.21.15.16:0/686352199 shutdown_connections 2024-03-27T06:44:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.141+0000 7fae2fbe2640 1 -- 172.21.15.16:0/686352199 wait complete. 2024-03-27T06:44:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.141+0000 7fae2fbe2640 1 Processor -- start 2024-03-27T06:44:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.141+0000 7fae2fbe2640 1 -- start start 2024-03-27T06:44:12.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2fbe2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae281009c0 0x7fae28197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2fbe2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae28101380 0x7fae28197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2fbe2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae281069d0 0x7fae2819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2fbe2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae28110eb0 con 0x7fae28101380 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2fbe2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fae28110d30 con 0x7fae281069d0 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2fbe2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fae28111030 con 0x7fae281009c0 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2d17d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae28101380 0x7fae28197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2d17d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae28101380 0x7fae28197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33364/0 (socket says 172.21.15.16:33364) 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2d17d640 1 -- 172.21.15.16:0/4098711654 learned_addr learned my addr 172.21.15.16:0/4098711654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2d97e640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae281009c0 0x7fae28197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2e17f640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae281069d0 0x7fae2819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2d17d640 1 -- 172.21.15.16:0/4098711654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae281009c0 msgr2=0x7fae28197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2d17d640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae281009c0 0x7fae28197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2d17d640 1 -- 172.21.15.16:0/4098711654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae281069d0 msgr2=0x7fae2819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2d17d640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae281069d0 0x7fae2819bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2d17d640 1 -- 172.21.15.16:0/4098711654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2819c440 con 0x7fae28101380 2024-03-27T06:44:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2d17d640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae28101380 0x7fae28197990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fae18009230 tx=0x7fae1802dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae1effd640 1 -- 172.21.15.16:0/4098711654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18043020 con 0x7fae28101380 2024-03-27T06:44:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae1effd640 1 -- 172.21.15.16:0/4098711654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae18004250 con 0x7fae28101380 2024-03-27T06:44:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae1effd640 1 -- 172.21.15.16:0/4098711654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18003970 con 0x7fae28101380 2024-03-27T06:44:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2fbe2640 1 -- 172.21.15.16:0/4098711654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2819c6d0 con 0x7fae28101380 2024-03-27T06:44:12.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.142+0000 7fae2fbe2640 1 -- 172.21.15.16:0/4098711654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae281a3fc0 con 0x7fae28101380 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.145+0000 7fae1effd640 1 -- 172.21.15.16:0/4098711654 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7fae18046020 con 0x7fae28101380 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.145+0000 7fae327c6740 1 Processor -- start 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae327c6740 1 -- start start 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae327c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 0x555cb0ab0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae327c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0d90 0x555cb0abb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae327c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0abbec0 0x555cb0abe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae327c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555cb0a14710 con 0x555cb0abbec0 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae327c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555cb09bcaf0 con 0x555cb0ab0470 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae327c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555cb0a106c0 con 0x555cb0ab0d90 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2d17d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0abbec0 0x555cb0abe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2e17f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 0x555cb0ab0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2d97e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0d90 0x555cb0abb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2e17f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 0x555cb0ab0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60322/0 (socket says 172.21.15.16:60322) 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae1effd640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae140a3930 0x7fae140a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2d17d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0abbec0 0x555cb0abe2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33374/0 (socket says 172.21.15.16:33374) 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2e17f640 1 -- 172.21.15.16:0/1126840288 learned_addr learned my addr 172.21.15.16:0/1126840288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2e17f640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae140a3930 0x7fae140a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae1effd640 1 -- 172.21.15.16:0/4098711654 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fae180f2200 con 0x7fae28101380 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2d97e640 1 -- 172.21.15.16:0/1126840288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 msgr2=0x555cb0ab0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2d97e640 1 --2- 172.21.15.16:0/1126840288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 0x555cb0ab0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2d97e640 1 -- 172.21.15.16:0/1126840288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0abbec0 msgr2=0x555cb0abe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2d97e640 1 --2- 172.21.15.16:0/1126840288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0abbec0 0x555cb0abe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2d97e640 1 -- 172.21.15.16:0/1126840288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cb07283f0 con 0x555cb0ab0d90 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.146+0000 7fae2e17f640 1 --2- 172.21.15.16:0/1126840288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 0x555cb0ab0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.147+0000 7fae2d97e640 1 --2- 172.21.15.16:0/1126840288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0d90 0x555cb0abb980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae24009c00 tx=0x7fae24011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510b41de2ff6fde6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.147+0000 7fae2e17f640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae140a3930 0x7fae140a5df0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae20009b80 tx=0x7fae20004ab0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.147+0000 7fae13fff640 1 -- 172.21.15.16:0/1126840288 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24004050 con 0x555cb0ab0d90 2024-03-27T06:44:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.147+0000 7fae13fff640 1 -- 172.21.15.16:0/1126840288 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae24005520 con 0x555cb0ab0d90 2024-03-27T06:44:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.147+0000 7fae13fff640 1 -- 172.21.15.16:0/1126840288 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24004de0 con 0x555cb0ab0d90 2024-03-27T06:44:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 -- 172.21.15.16:0/1126840288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0d90 msgr2=0x555cb0abb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 --2- 172.21.15.16:0/1126840288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0d90 0x555cb0abb980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae24009c00 tx=0x7fae24011810 comp rx=0 tx=0).stop 2024-03-27T06:44:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 -- 172.21.15.16:0/1126840288 shutdown_connections 2024-03-27T06:44:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 --2- 172.21.15.16:0/1126840288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0abbec0 0x555cb0abe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 --2- 172.21.15.16:0/1126840288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0d90 0x555cb0abb980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 --2- 172.21.15.16:0/1126840288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 0x555cb0ab0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 -- 172.21.15.16:0/1126840288 >> 172.21.15.16:0/1126840288 conn(0x555cb0aaaee0 msgr2=0x555cb0aab2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 -- 172.21.15.16:0/1126840288 shutdown_connections 2024-03-27T06:44:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 -- 172.21.15.16:0/1126840288 wait complete. 2024-03-27T06:44:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 -- start start 2024-03-27T06:44:12.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 Processor -- start 2024-03-27T06:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0470 0x555cb0ab8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0d90 0x555cb0ab9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0abbec0 0x555cb0ab9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555cb0abecb0 con 0x555cb0ab0470 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555cb09cfeb0 con 0x555cb0abbec0 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.148+0000 7fae327c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555cb0abee30 con 0x555cb0ab0d90 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae327c6740 10 client.0 _ll_register_callbacks cb 0x555cb0a0c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae2d97e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0470 0x555cb0ab8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae2d97e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0470 0x555cb0ab8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33386/0 (socket says 172.21.15.16:33386) 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae2d97e640 1 -- 172.21.15.16:0/2883967686 learned_addr learned my addr 172.21.15.16:0/2883967686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae2e17f640 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0abbec0 0x555cb0ab9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae2d17d640 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0d90 0x555cb0ab9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae2d97e640 1 -- 172.21.15.16:0/2883967686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0d90 msgr2=0x555cb0ab9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae2d97e640 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0d90 0x555cb0ab9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae2d97e640 1 -- 172.21.15.16:0/2883967686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0abbec0 msgr2=0x555cb0ab9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae2d97e640 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0abbec0 0x555cb0ab9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae2d97e640 1 -- 172.21.15.16:0/2883967686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cb07f27d0 con 0x555cb0ab0470 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae2d97e640 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0470 0x555cb0ab8dd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fae24009bd0 tx=0x7fae240056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e0d0f2bfec8a83 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae117fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae117fa640 1 -- 172.21.15.16:0/2883967686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24029070 con 0x555cb0ab0470 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae117fa640 1 -- 172.21.15.16:0/2883967686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fae24004d70 con 0x555cb0ab0470 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae327c6740 1 -- 172.21.15.16:0/2883967686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae24005520 con 0x555cb0ab0470 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae117fa640 1 -- 172.21.15.16:0/2883967686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fae24035050 con 0x555cb0ab0470 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae327c6740 10 client.15696 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.149+0000 7fae327c6740 1 -- 172.21.15.16:0/2883967686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555cb0acae80 con 0x555cb0ab0470 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.150+0000 7fae117fa640 1 -- 172.21.15.16:0/2883967686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae24005520 con 0x555cb0ab0470 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.150+0000 7fae117fa640 1 -- 172.21.15.16:0/2883967686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fae24023430 con 0x555cb0ab0470 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.150+0000 7fae327c6740 10 client.15696 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.150+0000 7fae327c6740 10 client.15696 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-27T06:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.150+0000 7fae327c6740 20 client.15696 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.150+0000 7fae327c6740 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555cb0acc230 0x555cb0ace620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.150+0000 7fae327c6740 4 client.15696 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.150+0000 7fae327c6740 1 -- 172.21.15.16:0/2883967686 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cb0a21c50 con 0x555cb0acc230 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.150+0000 7fae2d17d640 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555cb0acc230 0x555cb0ace620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.150+0000 7fae2d17d640 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555cb0acc230 0x555cb0ace620 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bed6ea63a413bdda server_cookie=8713e28538abdf8b in_seq=0 out_seq=0 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.150+0000 7fae117fa640 10 client.15696 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae117fa640 1 -- 172.21.15.16:0/2883967686 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555cb0a21c50 con 0x555cb0acc230 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae117fa640 10 client.15696 handle_command_reply: tid=0 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae327c6740 1 client.15696 shutdown 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae327c6740 1 -- 172.21.15.16:0/2883967686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0470 msgr2=0x555cb0ab8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae327c6740 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0470 0x555cb0ab8dd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fae24009bd0 tx=0x7fae240056d0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae327c6740 1 -- 172.21.15.16:0/2883967686 shutdown_connections 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae327c6740 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555cb0acc230 0x555cb0ace620 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae327c6740 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0abbec0 0x555cb0ab9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae327c6740 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0d90 0x555cb0ab9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae117fa640 0 client.15696 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae327c6740 1 --2- 172.21.15.16:0/2883967686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0470 0x555cb0ab8dd0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae327c6740 1 -- 172.21.15.16:0/2883967686 >> 172.21.15.16:0/2883967686 conn(0x555cb0aaaee0 msgr2=0x555cb0aacc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae327c6740 1 -- 172.21.15.16:0/2883967686 shutdown_connections 2024-03-27T06:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.151+0000 7fae327c6740 1 -- 172.21.15.16:0/2883967686 wait complete. 2024-03-27T06:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.152+0000 7fae327c6740 20 client.15696 trim_cache size 0 max 16384 2024-03-27T06:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae327c6740 1 Processor -- start 2024-03-27T06:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae327c6740 1 -- start start 2024-03-27T06:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae327c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0470 0x555cb0abb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae327c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0aace00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae327c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0abbec0 0x555cb0ac7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae327c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ab690 con 0x555cb0ab0d90 2024-03-27T06:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae327c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ad1d0 con 0x555cb0abbec0 2024-03-27T06:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae327c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fae281abd60 con 0x555cb0ab0470 2024-03-27T06:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae2d17d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0aace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae2d17d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0aace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33392/0 (socket says 172.21.15.16:33392) 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae2d17d640 1 -- 172.21.15.16:0/576461653 learned_addr learned my addr 172.21.15.16:0/576461653 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae2d97e640 1 --2- 172.21.15.16:0/576461653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0470 0x555cb0abb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae2e17f640 1 --2- 172.21.15.16:0/576461653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0abbec0 0x555cb0ac7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae2d17d640 1 -- 172.21.15.16:0/576461653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0470 msgr2=0x555cb0abb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae2d17d640 1 --2- 172.21.15.16:0/576461653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0470 0x555cb0abb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae2d17d640 1 -- 172.21.15.16:0/576461653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0abbec0 msgr2=0x555cb0ac7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae2d17d640 1 --2- 172.21.15.16:0/576461653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0abbec0 0x555cb0ac7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.165+0000 7fae2d17d640 1 -- 172.21.15.16:0/576461653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cb0ace990 con 0x555cb0ab0d90 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae2d17d640 1 --2- 172.21.15.16:0/576461653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0aace00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fae18005c40 tx=0x7fae180055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bcecc301c7f51b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae137fe640 1 -- 172.21.15.16:0/576461653 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180be990 con 0x555cb0ab0d90 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae137fe640 1 -- 172.21.15.16:0/576461653 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae180be290 con 0x555cb0ab0d90 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae137fe640 1 -- 172.21.15.16:0/576461653 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180ba900 con 0x555cb0ab0d90 2024-03-27T06:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae327c6740 1 -- 172.21.15.16:0/576461653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 msgr2=0x555cb0aace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae327c6740 1 --2- 172.21.15.16:0/576461653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0aace00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fae18005c40 tx=0x7fae180055a0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae327c6740 1 -- 172.21.15.16:0/576461653 shutdown_connections 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae327c6740 1 --2- 172.21.15.16:0/576461653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0abbec0 0x555cb0ac7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae327c6740 1 --2- 172.21.15.16:0/576461653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0aace00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae327c6740 1 --2- 172.21.15.16:0/576461653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0ab0470 0x555cb0abb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae327c6740 1 -- 172.21.15.16:0/576461653 >> 172.21.15.16:0/576461653 conn(0x555cb0aaaee0 msgr2=0x555cb0ab9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae327c6740 1 -- 172.21.15.16:0/576461653 shutdown_connections 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.166+0000 7fae327c6740 1 -- 172.21.15.16:0/576461653 wait complete. 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae327c6740 1 -- start start 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae327c6740 1 Processor -- start 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae327c6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 0x555cb0ab5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae327c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0ab6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae327c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0abbec0 0x555cb0b31430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae327c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae281acb00 con 0x555cb0ab0d90 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae327c6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ac430 con 0x555cb0ab0470 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae327c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fae281a9670 con 0x555cb0abbec0 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae327c6740 10 client.0 _ll_register_callbacks cb 0x7fae180be060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae2d17d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0ab6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae2d17d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0ab6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33406/0 (socket says 172.21.15.16:33406) 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae2d17d640 1 -- 172.21.15.16:0/1104911509 learned_addr learned my addr 172.21.15.16:0/1104911509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae2d97e640 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 0x555cb0ab5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae2e17f640 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0abbec0 0x555cb0b31430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae2d17d640 1 -- 172.21.15.16:0/1104911509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0abbec0 msgr2=0x555cb0b31430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae2d17d640 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0abbec0 0x555cb0b31430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae2d17d640 1 -- 172.21.15.16:0/1104911509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 msgr2=0x555cb0ab5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae2d17d640 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 0x555cb0ab5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.167+0000 7fae2d17d640 1 -- 172.21.15.16:0/1104911509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cb0ab0bc0 con 0x555cb0ab0d90 2024-03-27T06:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae2d17d640 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0ab6460 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fae180be220 tx=0x7fae18005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cc3f8b398f7f69 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae117fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae117fa640 1 -- 172.21.15.16:0/1104911509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180be990 con 0x555cb0ab0d90 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae117fa640 1 -- 172.21.15.16:0/1104911509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fae180ba900 con 0x555cb0ab0d90 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae327c6740 1 -- 172.21.15.16:0/1104911509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae180be290 con 0x555cb0ab0d90 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae117fa640 1 -- 172.21.15.16:0/1104911509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fae180c2640 con 0x555cb0ab0d90 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae327c6740 10 client.15702 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae327c6740 1 -- 172.21.15.16:0/1104911509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555cb0b32940 con 0x555cb0ab0d90 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae117fa640 1 -- 172.21.15.16:0/1104911509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae180be290 con 0x555cb0ab0d90 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae117fa640 1 -- 172.21.15.16:0/1104911509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fae180d2cd0 con 0x555cb0ab0d90 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae327c6740 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae327c6740 10 client.15702 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae327c6740 20 client.15702 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae327c6740 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555cb0b32e50 0x555cb0b35270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae327c6740 4 client.15702 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.168+0000 7fae327c6740 1 -- 172.21.15.16:0/1104911509 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cb0a1bfa0 con 0x555cb0b32e50 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae2d97e640 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555cb0b32e50 0x555cb0b35270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae2d97e640 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555cb0b32e50 0x555cb0b35270 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3124d6560e8b8d6 server_cookie=a1b2fa6c53e45a45 in_seq=0 out_seq=0 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae117fa640 10 client.15702 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae117fa640 1 -- 172.21.15.16:0/1104911509 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cb0a1bfa0 con 0x555cb0b32e50 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae117fa640 10 client.15702 handle_command_reply: tid=0 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 1 client.15702 shutdown 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 1 -- 172.21.15.16:0/1104911509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 msgr2=0x555cb0ab6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0ab6460 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fae180be220 tx=0x7fae18005fd0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 1 -- 172.21.15.16:0/1104911509 shutdown_connections 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x555cb0b32e50 0x555cb0b35270 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cb0abbec0 0x555cb0b31430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555cb0ab0d90 0x555cb0ab6460 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae117fa640 0 client.15702 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 1 --2- 172.21.15.16:0/1104911509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555cb0ab0470 0x555cb0ab5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 1 -- 172.21.15.16:0/1104911509 >> 172.21.15.16:0/1104911509 conn(0x555cb0aaaee0 msgr2=0x555cb0ab8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 1 -- 172.21.15.16:0/1104911509 shutdown_connections 2024-03-27T06:44:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 1 -- 172.21.15.16:0/1104911509 wait complete. 2024-03-27T06:44:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.169+0000 7fae327c6740 20 client.15702 trim_cache size 0 max 16384 2024-03-27T06:44:12.171 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 -- 172.21.15.16:0/4098711654 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae140a3930 msgr2=0x7fae140a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae140a3930 0x7fae140a5df0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae20009b80 tx=0x7fae20004ab0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 -- 172.21.15.16:0/4098711654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae28101380 msgr2=0x7fae28197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae28101380 0x7fae28197990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fae18009230 tx=0x7fae1802dc90 comp rx=0 tx=0).stop 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 -- 172.21.15.16:0/4098711654 shutdown_connections 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae281069d0 0x7fae2819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae28101380 0x7fae28197990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae140a3930 0x7fae140a5df0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 --2- 172.21.15.16:0/4098711654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae281009c0 0x7fae28197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 -- 172.21.15.16:0/4098711654 >> 172.21.15.16:0/4098711654 conn(0x7fae28077b50 msgr2=0x7fae28103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 -- 172.21.15.16:0/4098711654 shutdown_connections 2024-03-27T06:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.170+0000 7fae2fbe2640 1 -- 172.21.15.16:0/4098711654 wait complete. 2024-03-27T06:44:12.174 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:11 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3770381439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:44:12.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-27T06:44:12.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:11 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3770381439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:44:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.286+0000 7f32a809a640 1 -- 172.21.15.16:0/3932854051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a0105a40 msgr2=0x7f32a0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.286+0000 7f32a809a640 1 --2- 172.21.15.16:0/3932854051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a0105a40 0x7f32a0105e20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f329c009900 tx=0x7f329c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.286+0000 7f32a809a640 1 -- 172.21.15.16:0/3932854051 shutdown_connections 2024-03-27T06:44:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.286+0000 7f32a809a640 1 --2- 172.21.15.16:0/3932854051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a0101b20 0x7f32a010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.286+0000 7f32a809a640 1 --2- 172.21.15.16:0/3932854051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a0101160 0x7f32a01015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.286+0000 7f32a809a640 1 --2- 172.21.15.16:0/3932854051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a0105a40 0x7f32a0105e20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.286+0000 7f32a809a640 1 -- 172.21.15.16:0/3932854051 >> 172.21.15.16:0/3932854051 conn(0x7f32a0077010 msgr2=0x7f32a00fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.286+0000 7f32a809a640 1 -- 172.21.15.16:0/3932854051 shutdown_connections 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.286+0000 7f32a809a640 1 -- 172.21.15.16:0/3932854051 wait complete. 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a809a640 1 Processor -- start 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a809a640 1 -- start start 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a809a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a0101160 0x7f32a0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a809a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a0101b20 0x7f32a0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a809a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a0105a40 0x7f32a019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a809a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0112000 con 0x7f32a0101b20 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a809a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0111e80 con 0x7f32a0105a40 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a809a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0112180 con 0x7f32a0101160 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a5635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a0101b20 0x7f32a0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a5635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a0101b20 0x7f32a0198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33432/0 (socket says 172.21.15.16:33432) 2024-03-27T06:44:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a5635640 1 -- 172.21.15.16:0/2330193016 learned_addr learned my addr 172.21.15.16:0/2330193016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a5e36640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a0101160 0x7f32a0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a6637640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a0105a40 0x7f32a019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a5635640 1 -- 172.21.15.16:0/2330193016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a0101160 msgr2=0x7f32a0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a5635640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a0101160 0x7f32a0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a5635640 1 -- 172.21.15.16:0/2330193016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a0105a40 msgr2=0x7f32a019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a5635640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a0105a40 0x7f32a019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.287+0000 7f32a5635640 1 -- 172.21.15.16:0/2330193016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a019d3c0 con 0x7f32a0101b20 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.288+0000 7f32a5635640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a0101b20 0x7f32a0198960 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f329000ea60 tx=0x7f329000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.288+0000 7f3296ffd640 1 -- 172.21.15.16:0/2330193016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329000cd20 con 0x7f32a0101b20 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.288+0000 7f3296ffd640 1 -- 172.21.15.16:0/2330193016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32900035f0 con 0x7f32a0101b20 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.288+0000 7f32a809a640 1 -- 172.21.15.16:0/2330193016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a019d6b0 con 0x7f32a0101b20 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.288+0000 7f3296ffd640 1 -- 172.21.15.16:0/2330193016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290010690 con 0x7f32a0101b20 2024-03-27T06:44:12.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.288+0000 7f32a809a640 1 -- 172.21.15.16:0/2330193016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a01a4f40 con 0x7f32a0101b20 2024-03-27T06:44:12.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.290+0000 7f3296ffd640 1 -- 172.21.15.16:0/2330193016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7f32900037a0 con 0x7f32a0101b20 2024-03-27T06:44:12.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.290+0000 7f32aa922740 1 Processor -- start 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.290+0000 7f32aa922740 1 -- start start 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.290+0000 7f32aa922740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de137b0 0x55712de13b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.290+0000 7f3296ffd640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f328c0a3930 0x7f328c0a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.290+0000 7f32aa922740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de1c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.290+0000 7f32aa922740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 0x55712de1f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.290+0000 7f32aa922740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55712dd77710 con 0x55712de1ccb0 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.290+0000 7f32aa922740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55712dd1faf0 con 0x55712de140d0 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.290+0000 7f32aa922740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55712dd736c0 con 0x55712de137b0 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a6637640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 0x55712de1f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f3296ffd640 1 -- 172.21.15.16:0/2330193016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f32900d0750 con 0x7f32a0101b20 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a5635640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de1c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a5e36640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f328c0a3930 0x7f328c0a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a6637640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 0x55712de1f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33438/0 (socket says 172.21.15.16:33438) 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a6637640 1 -- 172.21.15.16:0/989651385 learned_addr learned my addr 172.21.15.16:0/989651385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a6637640 1 --2- 172.21.15.16:0/989651385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de137b0 0x55712de13b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a5635640 1 -- 172.21.15.16:0/989651385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de137b0 msgr2=0x55712de13b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a5635640 1 --2- 172.21.15.16:0/989651385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de137b0 0x55712de13b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a5635640 1 -- 172.21.15.16:0/989651385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 msgr2=0x55712de1f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a5635640 1 --2- 172.21.15.16:0/989651385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 0x55712de1f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a5635640 1 -- 172.21.15.16:0/989651385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55712da8b3f0 con 0x55712de140d0 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a5635640 1 --2- 172.21.15.16:0/989651385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de1c770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32900cdc90 tx=0x7f329009c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e790480944952d43 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f32a5e36640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f328c0a3930 0x7f328c0a5df0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f329c006010 tx=0x7f329c005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f328bfff640 1 -- 172.21.15.16:0/989651385 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32900a9f40 con 0x55712de140d0 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f328bfff640 1 -- 172.21.15.16:0/989651385 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3290003bc0 con 0x55712de140d0 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.291+0000 7f328bfff640 1 -- 172.21.15.16:0/989651385 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32900870d0 con 0x55712de140d0 2024-03-27T06:44:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.292+0000 7f32aa922740 1 -- 172.21.15.16:0/989651385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 msgr2=0x55712de1c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.292+0000 7f32aa922740 1 --2- 172.21.15.16:0/989651385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de1c770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32900cdc90 tx=0x7f329009c560 comp rx=0 tx=0).stop 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.292+0000 7f32aa922740 1 -- 172.21.15.16:0/989651385 shutdown_connections 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.292+0000 7f32aa922740 1 --2- 172.21.15.16:0/989651385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 0x55712de1f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.292+0000 7f32aa922740 1 --2- 172.21.15.16:0/989651385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de1c770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.292+0000 7f32aa922740 1 --2- 172.21.15.16:0/989651385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de137b0 0x55712de13b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.292+0000 7f32aa922740 1 -- 172.21.15.16:0/989651385 >> 172.21.15.16:0/989651385 conn(0x55712de0dee0 msgr2=0x55712de0e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.292+0000 7f32aa922740 1 -- 172.21.15.16:0/989651385 shutdown_connections 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.292+0000 7f32aa922740 1 -- 172.21.15.16:0/989651385 wait complete. 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.292+0000 7f32aa922740 1 -- start start 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.292+0000 7f32aa922740 1 Processor -- start 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32aa922740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de137b0 0x55712de19d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32aa922740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de140d0 0x55712de1a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32aa922740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de1ccb0 0x55712de1a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32aa922740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55712de1fa40 con 0x55712de140d0 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32aa922740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55712dd32eb0 con 0x55712de137b0 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32aa922740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55712de1fbc0 con 0x55712de1ccb0 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32aa922740 10 client.0 _ll_register_callbacks cb 0x55712dd6f6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32a5635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de140d0 0x55712de1a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32a5635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de140d0 0x55712de1a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33452/0 (socket says 172.21.15.16:33452) 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32a5635640 1 -- 172.21.15.16:0/1719740746 learned_addr learned my addr 172.21.15.16:0/1719740746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32a6637640 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de1ccb0 0x55712de1a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32a5e36640 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de137b0 0x55712de19d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32a5635640 1 -- 172.21.15.16:0/1719740746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de1ccb0 msgr2=0x55712de1a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32a5635640 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de1ccb0 0x55712de1a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32a5635640 1 -- 172.21.15.16:0/1719740746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de137b0 msgr2=0x55712de19d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32a5635640 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de137b0 0x55712de19d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32a5635640 1 -- 172.21.15.16:0/1719740746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55712db557d0 con 0x55712de140d0 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32a5635640 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de140d0 0x55712de1a270 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f32900cdcf0 tx=0x7f3290003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1f9aa0eb45e6c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.293+0000 7f32897fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32897fa640 1 -- 172.21.15.16:0/1719740746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32900af5a0 con 0x55712de140d0 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32897fa640 1 -- 172.21.15.16:0/1719740746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f329009bb20 con 0x55712de140d0 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32aa922740 1 -- 172.21.15.16:0/1719740746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55712dd84c50 con 0x55712de140d0 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32897fa640 1 -- 172.21.15.16:0/1719740746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f32900af780 con 0x55712de140d0 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32aa922740 10 client.15714 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32aa922740 1 -- 172.21.15.16:0/1719740746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55712de2bd30 con 0x55712de140d0 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32897fa640 1 -- 172.21.15.16:0/1719740746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55712dd84c50 con 0x55712de140d0 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32897fa640 1 -- 172.21.15.16:0/1719740746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f32900acce0 con 0x55712de140d0 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32aa922740 10 client.15714 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32aa922740 10 client.15714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-27T06:44:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32aa922740 20 client.15714 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32aa922740 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55712de2d080 0x55712de2f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32aa922740 4 client.15714 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32aa922740 1 -- 172.21.15.16:0/1719740746 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55712dd7efa0 con 0x55712de2d080 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.294+0000 7f32a5e36640 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55712de2d080 0x55712de2f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.295+0000 7f32897fa640 10 client.15714 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.295+0000 7f32a5e36640 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55712de2d080 0x55712de2f470 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2132e66ca4a0c637 server_cookie=93403bd06f2f5e5b in_seq=0 out_seq=0 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.295+0000 7f32897fa640 1 -- 172.21.15.16:0/1719740746 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55712dd7efa0 con 0x55712de2d080 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.295+0000 7f32897fa640 10 client.15714 handle_command_reply: tid=0 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.295+0000 7f32aa922740 1 client.15714 shutdown 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.295+0000 7f32aa922740 1 -- 172.21.15.16:0/1719740746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de140d0 msgr2=0x55712de1a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.295+0000 7f32aa922740 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de140d0 0x55712de1a270 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f32900cdcf0 tx=0x7f3290003f30 comp rx=0 tx=0).stop 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.296+0000 7f32aa922740 1 -- 172.21.15.16:0/1719740746 shutdown_connections 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.296+0000 7f32aa922740 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55712de2d080 0x55712de2f470 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.296+0000 7f32aa922740 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de1ccb0 0x55712de1a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.296+0000 7f32aa922740 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de140d0 0x55712de1a270 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.296+0000 7f32aa922740 1 --2- 172.21.15.16:0/1719740746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de137b0 0x55712de19d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.296+0000 7f32897fa640 0 client.15714 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:44:12.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.296+0000 7f32aa922740 1 -- 172.21.15.16:0/1719740746 >> 172.21.15.16:0/1719740746 conn(0x55712de0dee0 msgr2=0x55712de0fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.296+0000 7f32aa922740 1 -- 172.21.15.16:0/1719740746 shutdown_connections 2024-03-27T06:44:12.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.296+0000 7f32aa922740 1 -- 172.21.15.16:0/1719740746 wait complete. 2024-03-27T06:44:12.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.296+0000 7f32aa922740 20 client.15714 trim_cache size 0 max 16384 2024-03-27T06:44:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32aa922740 1 Processor -- start 2024-03-27T06:44:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32aa922740 1 -- start start 2024-03-27T06:44:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32aa922740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de137b0 0x55712de1c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32aa922740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de0ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32aa922740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 0x55712de32490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32aa922740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01ad8a0 con 0x55712de1ccb0 2024-03-27T06:44:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32aa922740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01af3e0 con 0x55712de140d0 2024-03-27T06:44:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32aa922740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01adf70 con 0x55712de137b0 2024-03-27T06:44:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32a6637640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 0x55712de32490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32a6637640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 0x55712de32490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33458/0 (socket says 172.21.15.16:33458) 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32a5e36640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de137b0 0x55712de1c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32a6637640 1 -- 172.21.15.16:0/2494513268 learned_addr learned my addr 172.21.15.16:0/2494513268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32a5635640 1 --2- 172.21.15.16:0/2494513268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de0ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32a6637640 1 -- 172.21.15.16:0/2494513268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de137b0 msgr2=0x55712de1c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32a6637640 1 --2- 172.21.15.16:0/2494513268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de137b0 0x55712de1c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32a6637640 1 -- 172.21.15.16:0/2494513268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 msgr2=0x55712de0ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.309+0000 7f32a6637640 1 --2- 172.21.15.16:0/2494513268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de0ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f32a6637640 1 -- 172.21.15.16:0/2494513268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55712de1f5c0 con 0x55712de1ccb0 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f32a6637640 1 --2- 172.21.15.16:0/2494513268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 0x55712de32490 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3298009b80 tx=0x7f3298009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0991386017459d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f328b7fe640 1 -- 172.21.15.16:0/2494513268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298023070 con 0x55712de1ccb0 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f328b7fe640 1 -- 172.21.15.16:0/2494513268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3298004380 con 0x55712de1ccb0 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f328b7fe640 1 -- 172.21.15.16:0/2494513268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329801e9c0 con 0x55712de1ccb0 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f32aa922740 1 -- 172.21.15.16:0/2494513268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 msgr2=0x55712de32490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f32aa922740 1 --2- 172.21.15.16:0/2494513268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 0x55712de32490 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3298009b80 tx=0x7f3298009af0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f32aa922740 1 -- 172.21.15.16:0/2494513268 shutdown_connections 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f32aa922740 1 --2- 172.21.15.16:0/2494513268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de1ccb0 0x55712de32490 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f32aa922740 1 --2- 172.21.15.16:0/2494513268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de0ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f32aa922740 1 --2- 172.21.15.16:0/2494513268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de137b0 0x55712de1c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f32aa922740 1 -- 172.21.15.16:0/2494513268 >> 172.21.15.16:0/2494513268 conn(0x55712de0dee0 msgr2=0x55712de0c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f32aa922740 1 -- 172.21.15.16:0/2494513268 shutdown_connections 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.310+0000 7f32aa922740 1 -- 172.21.15.16:0/2494513268 wait complete. 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32aa922740 1 -- start start 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32aa922740 1 Processor -- start 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32aa922740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de137b0 0x55712de2fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32aa922740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de301f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32aa922740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de1ccb0 0x55712de30730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32aa922740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01aed10 con 0x55712de137b0 2024-03-27T06:44:12.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32aa922740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01ae640 con 0x55712de140d0 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32aa922740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01ab880 con 0x55712de1ccb0 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32aa922740 10 client.0 _ll_register_callbacks cb 0x7f3298009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32a5e36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de137b0 0x55712de2fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32a5e36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de137b0 0x55712de2fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33468/0 (socket says 172.21.15.16:33468) 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32a5e36640 1 -- 172.21.15.16:0/66297225 learned_addr learned my addr 172.21.15.16:0/66297225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32a5635640 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32a6637640 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de1ccb0 0x55712de30730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.311+0000 7f32a5e36640 1 -- 172.21.15.16:0/66297225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de1ccb0 msgr2=0x55712de30730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32a5e36640 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de1ccb0 0x55712de30730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32a5e36640 1 -- 172.21.15.16:0/66297225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 msgr2=0x55712de301f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32a5e36640 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de301f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32a5e36640 1 -- 172.21.15.16:0/66297225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55712de96430 con 0x55712de137b0 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32a5e36640 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de137b0 0x55712de2fcb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f329c0096c0 tx=0x7f329c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feae2d85bfa8921c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32897fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32897fa640 1 -- 172.21.15.16:0/66297225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c056020 con 0x55712de137b0 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32897fa640 1 -- 172.21.15.16:0/66297225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f329c005100 con 0x55712de137b0 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32aa922740 1 -- 172.21.15.16:0/66297225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3298004380 con 0x55712de137b0 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32897fa640 1 -- 172.21.15.16:0/66297225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f329c061050 con 0x55712de137b0 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32aa922740 10 client.15720 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32aa922740 1 -- 172.21.15.16:0/66297225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55712de97610 con 0x55712de137b0 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32897fa640 1 -- 172.21.15.16:0/66297225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3298004380 con 0x55712de137b0 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32897fa640 1 -- 172.21.15.16:0/66297225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f329c0048f0 con 0x55712de137b0 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32aa922740 10 client.15720 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32aa922740 10 client.15720 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32aa922740 20 client.15720 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32aa922740 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55712de97d30 0x55712de9a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32aa922740 4 client.15720 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.312+0000 7f32aa922740 1 -- 172.21.15.16:0/66297225 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55712dcfc6a0 con 0x55712de97d30 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32a5635640 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55712de97d30 0x55712de9a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32a5635640 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55712de97d30 0x55712de9a120 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcea50eec329252e server_cookie=2d74a03fd1b7c46d in_seq=0 out_seq=0 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32897fa640 10 client.15720 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32897fa640 1 -- 172.21.15.16:0/66297225 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55712dcfc6a0 con 0x55712de97d30 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32897fa640 10 client.15720 handle_command_reply: tid=0 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 1 client.15720 shutdown 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 1 -- 172.21.15.16:0/66297225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de137b0 msgr2=0x55712de2fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de137b0 0x55712de2fcb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f329c0096c0 tx=0x7f329c009260 comp rx=0 tx=0).stop 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 1 -- 172.21.15.16:0/66297225 shutdown_connections 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55712de97d30 0x55712de9a120 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712de1ccb0 0x55712de30730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55712de140d0 0x55712de301f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 1 --2- 172.21.15.16:0/66297225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55712de137b0 0x55712de2fcb0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32897fa640 0 client.15720 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:44:12.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 1 -- 172.21.15.16:0/66297225 >> 172.21.15.16:0/66297225 conn(0x55712de0dee0 msgr2=0x55712de286a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 1 -- 172.21.15.16:0/66297225 shutdown_connections 2024-03-27T06:44:12.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 1 -- 172.21.15.16:0/66297225 wait complete. 2024-03-27T06:44:12.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.313+0000 7f32aa922740 20 client.15720 trim_cache size 0 max 16384 2024-03-27T06:44:12.316 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:44:12.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 -- 172.21.15.16:0/2330193016 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f328c0a3930 msgr2=0x7f328c0a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f328c0a3930 0x7f328c0a5df0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f329c006010 tx=0x7f329c005e40 comp rx=0 tx=0).stop 2024-03-27T06:44:12.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 -- 172.21.15.16:0/2330193016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a0101b20 msgr2=0x7f32a0198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a0101b20 0x7f32a0198960 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f329000ea60 tx=0x7f329000ee60 comp rx=0 tx=0).stop 2024-03-27T06:44:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 -- 172.21.15.16:0/2330193016 shutdown_connections 2024-03-27T06:44:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a0105a40 0x7f32a019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32a0101b20 0x7f32a0198960 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f328c0a3930 0x7f328c0a5df0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 --2- 172.21.15.16:0/2330193016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a0101160 0x7f32a0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 -- 172.21.15.16:0/2330193016 >> 172.21.15.16:0/2330193016 conn(0x7f32a0077010 msgr2=0x7f32a0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 -- 172.21.15.16:0/2330193016 shutdown_connections 2024-03-27T06:44:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.314+0000 7f32a809a640 1 -- 172.21.15.16:0/2330193016 wait complete. 2024-03-27T06:44:12.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-27T06:44:12.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:11 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3770381439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:44:12.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.430+0000 7f90966e0640 1 -- 172.21.15.16:0/1275113693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9090101160 msgr2=0x7f90901015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.430+0000 7f90966e0640 1 --2- 172.21.15.16:0/1275113693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9090101160 0x7f90901015e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9080009900 tx=0x7f908002bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:12.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.430+0000 7f90966e0640 1 -- 172.21.15.16:0/1275113693 shutdown_connections 2024-03-27T06:44:12.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.430+0000 7f90966e0640 1 --2- 172.21.15.16:0/1275113693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9090101b20 0x7f909010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.430+0000 7f90966e0640 1 --2- 172.21.15.16:0/1275113693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9090101160 0x7f90901015e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.430+0000 7f90966e0640 1 --2- 172.21.15.16:0/1275113693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9090105a40 0x7f9090105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.430+0000 7f90966e0640 1 -- 172.21.15.16:0/1275113693 >> 172.21.15.16:0/1275113693 conn(0x7f9090077010 msgr2=0x7f90900fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.430+0000 7f90966e0640 1 -- 172.21.15.16:0/1275113693 shutdown_connections 2024-03-27T06:44:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.430+0000 7f90966e0640 1 -- 172.21.15.16:0/1275113693 wait complete. 2024-03-27T06:44:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f90966e0640 1 Processor -- start 2024-03-27T06:44:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f90966e0640 1 -- start start 2024-03-27T06:44:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f90966e0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9090101160 0x7f9090198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f90966e0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9090101b20 0x7f9090198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f90966e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9090105a40 0x7f909019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f90966e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9090110f40 con 0x7f9090105a40 2024-03-27T06:44:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f90966e0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f9090110dc0 con 0x7f9090101160 2024-03-27T06:44:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f90966e0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90901110c0 con 0x7f9090101b20 2024-03-27T06:44:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f9094c7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9090105a40 0x7f909019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f9094c7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9090105a40 0x7f909019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33480/0 (socket says 172.21.15.16:33480) 2024-03-27T06:44:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f9094c7d640 1 -- 172.21.15.16:0/3222264548 learned_addr learned my addr 172.21.15.16:0/3222264548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f908f7fe640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9090101b20 0x7f9090198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.431+0000 7f908ffff640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9090101160 0x7f9090198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f908f7fe640 1 -- 172.21.15.16:0/3222264548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9090101160 msgr2=0x7f9090198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f908f7fe640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9090101160 0x7f9090198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f908f7fe640 1 -- 172.21.15.16:0/3222264548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9090105a40 msgr2=0x7f909019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f908f7fe640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9090105a40 0x7f909019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f908f7fe640 1 -- 172.21.15.16:0/3222264548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909019d420 con 0x7f9090101b20 2024-03-27T06:44:12.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f9094c7d640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9090105a40 0x7f909019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:12.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f908f7fe640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9090101b20 0x7f9090198970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9080036040 tx=0x7f908002dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f908d7fa640 1 -- 172.21.15.16:0/3222264548 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080043020 con 0x7f9090101b20 2024-03-27T06:44:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f90966e0640 1 -- 172.21.15.16:0/3222264548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909019d6b0 con 0x7f9090101b20 2024-03-27T06:44:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f908d7fa640 1 -- 172.21.15.16:0/3222264548 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9080004e50 con 0x7f9090101b20 2024-03-27T06:44:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f908d7fa640 1 -- 172.21.15.16:0/3222264548 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080003970 con 0x7f9090101b20 2024-03-27T06:44:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.432+0000 7f90966e0640 1 -- 172.21.15.16:0/3222264548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90901a4fa0 con 0x7f9090101b20 2024-03-27T06:44:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.434+0000 7f908d7fa640 1 -- 172.21.15.16:0/3222264548 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7f9080046020 con 0x7f9090101b20 2024-03-27T06:44:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.434+0000 7f909939d740 1 Processor -- start 2024-03-27T06:44:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f909939d740 1 -- start start 2024-03-27T06:44:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f909939d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f84e0 0x55dc9f6f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f908d7fa640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f907c0a3930 0x7f907c0a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f909939d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f8e00 0x55dc9f705b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f909939d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 0x55dc9f7084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f909939d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc9f65c710 con 0x55dc9f7060c0 2024-03-27T06:44:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f909939d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55dc9f604af0 con 0x55dc9f6f84e0 2024-03-27T06:44:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f909939d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dc9f6586c0 con 0x55dc9f6f8e00 2024-03-27T06:44:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f908ffff640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f907c0a3930 0x7f907c0a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f9094c7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 0x55dc9f7084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f9094c7d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f84e0 0x55dc9f6f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f9094c7d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f84e0 0x55dc9f6f88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60444/0 (socket says 172.21.15.16:60444) 2024-03-27T06:44:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f9094c7d640 1 -- 172.21.15.16:0/145213850 learned_addr learned my addr 172.21.15.16:0/145213850 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f908f7fe640 1 --2- 172.21.15.16:0/145213850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f8e00 0x55dc9f705b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f908d7fa640 1 -- 172.21.15.16:0/3222264548 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f90800f20e0 con 0x7f9090101b20 2024-03-27T06:44:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f9094c7d640 1 -- 172.21.15.16:0/145213850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f8e00 msgr2=0x55dc9f705b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f9094c7d640 1 --2- 172.21.15.16:0/145213850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f8e00 0x55dc9f705b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f9094c7d640 1 -- 172.21.15.16:0/145213850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 msgr2=0x55dc9f7084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f9094c7d640 1 --2- 172.21.15.16:0/145213850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 0x55dc9f7084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f9094c7d640 1 -- 172.21.15.16:0/145213850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9f3703f0 con 0x55dc9f6f84e0 2024-03-27T06:44:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f908ffff640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f907c0a3930 0x7f907c0a5df0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9078009bd0 tx=0x7f9078005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.435+0000 7f9094c7d640 1 --2- 172.21.15.16:0/145213850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f84e0 0x55dc9f6f88c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f908800eef0 tx=0x7f9088009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90e023d29321727f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f90867fc640 1 -- 172.21.15.16:0/145213850 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90880130d0 con 0x55dc9f6f84e0 2024-03-27T06:44:12.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f90867fc640 1 -- 172.21.15.16:0/145213850 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9088013270 con 0x55dc9f6f84e0 2024-03-27T06:44:12.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f90867fc640 1 -- 172.21.15.16:0/145213850 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088004210 con 0x55dc9f6f84e0 2024-03-27T06:44:12.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f909939d740 1 -- 172.21.15.16:0/145213850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f84e0 msgr2=0x55dc9f6f88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f909939d740 1 --2- 172.21.15.16:0/145213850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f84e0 0x55dc9f6f88c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f908800eef0 tx=0x7f9088009500 comp rx=0 tx=0).stop 2024-03-27T06:44:12.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f909939d740 1 -- 172.21.15.16:0/145213850 shutdown_connections 2024-03-27T06:44:12.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f909939d740 1 --2- 172.21.15.16:0/145213850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 0x55dc9f7084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f909939d740 1 --2- 172.21.15.16:0/145213850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f8e00 0x55dc9f705b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f909939d740 1 --2- 172.21.15.16:0/145213850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f84e0 0x55dc9f6f88c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f909939d740 1 -- 172.21.15.16:0/145213850 >> 172.21.15.16:0/145213850 conn(0x55dc9f6f2ee0 msgr2=0x55dc9f6f32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f909939d740 1 -- 172.21.15.16:0/145213850 shutdown_connections 2024-03-27T06:44:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.436+0000 7f909939d740 1 -- 172.21.15.16:0/145213850 wait complete. 2024-03-27T06:44:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f909939d740 1 -- start start 2024-03-27T06:44:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f909939d740 1 Processor -- start 2024-03-27T06:44:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f909939d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f703140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f909939d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f8e00 0x55dc9f703680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f909939d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f7060c0 0x55dc9f703bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f909939d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc9f708e20 con 0x55dc9f6f84e0 2024-03-27T06:44:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f909939d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55dc9f617eb0 con 0x55dc9f7060c0 2024-03-27T06:44:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f909939d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dc9f708fa0 con 0x55dc9f6f8e00 2024-03-27T06:44:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f908ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f703140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f909939d740 10 client.0 _ll_register_callbacks cb 0x55dc9f6546a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f908ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f703140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33492/0 (socket says 172.21.15.16:33492) 2024-03-27T06:44:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f908ffff640 1 -- 172.21.15.16:0/870204574 learned_addr learned my addr 172.21.15.16:0/870204574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f908f7fe640 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f8e00 0x55dc9f703680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.437+0000 7f9094c7d640 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f7060c0 0x55dc9f703bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f908ffff640 1 -- 172.21.15.16:0/870204574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f8e00 msgr2=0x55dc9f703680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f908ffff640 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f8e00 0x55dc9f703680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f908ffff640 1 -- 172.21.15.16:0/870204574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f7060c0 msgr2=0x55dc9f703bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f908ffff640 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f7060c0 0x55dc9f703bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f908ffff640 1 -- 172.21.15.16:0/870204574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc9f43a7d0 con 0x55dc9f6f84e0 2024-03-27T06:44:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f908ffff640 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f703140 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9078003940 tx=0x7f9078004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13277769bb8417f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f906f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f906f7fe640 1 -- 172.21.15.16:0/870204574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907800ecc0 con 0x55dc9f6f84e0 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f906f7fe640 1 -- 172.21.15.16:0/870204574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f9078005020 con 0x55dc9f6f84e0 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f909939d740 1 -- 172.21.15.16:0/870204574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9088013270 con 0x55dc9f6f84e0 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f906f7fe640 1 -- 172.21.15.16:0/870204574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f907800ee60 con 0x55dc9f6f84e0 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f909939d740 10 client.15732 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f909939d740 1 -- 172.21.15.16:0/870204574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc9f7151b0 con 0x55dc9f6f84e0 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f906f7fe640 1 -- 172.21.15.16:0/870204574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9088013270 con 0x55dc9f6f84e0 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f906f7fe640 1 -- 172.21.15.16:0/870204574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f907802f4e0 con 0x55dc9f6f84e0 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f909939d740 10 client.15732 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f909939d740 10 client.15732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f909939d740 20 client.15732 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f909939d740 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55dc9f716610 0x55dc9f718a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f909939d740 4 client.15732 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.438+0000 7f909939d740 1 -- 172.21.15.16:0/870204574 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc9f669c50 con 0x55dc9f716610 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.439+0000 7f908f7fe640 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55dc9f716610 0x55dc9f718a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.439+0000 7f908f7fe640 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55dc9f716610 0x55dc9f718a70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=acf06b65047678c7 server_cookie=70f6ce92f30eab9c in_seq=0 out_seq=0 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.439+0000 7f906f7fe640 10 client.15732 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f906f7fe640 1 -- 172.21.15.16:0/870204574 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dc9f669c50 con 0x55dc9f716610 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f906f7fe640 10 client.15732 handle_command_reply: tid=0 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 1 client.15732 shutdown 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 1 -- 172.21.15.16:0/870204574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 msgr2=0x55dc9f703140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f703140 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9078003940 tx=0x7f9078004730 comp rx=0 tx=0).stop 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 1 -- 172.21.15.16:0/870204574 shutdown_connections 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55dc9f716610 0x55dc9f718a70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f7060c0 0x55dc9f703bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f906f7fe640 0 client.15732 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f8e00 0x55dc9f703680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 1 --2- 172.21.15.16:0/870204574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f703140 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 1 -- 172.21.15.16:0/870204574 >> 172.21.15.16:0/870204574 conn(0x55dc9f6f2ee0 msgr2=0x55dc9f6f4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 1 -- 172.21.15.16:0/870204574 shutdown_connections 2024-03-27T06:44:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 1 -- 172.21.15.16:0/870204574 wait complete. 2024-03-27T06:44:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.440+0000 7f909939d740 20 client.15732 trim_cache size 0 max 16384 2024-03-27T06:44:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.453+0000 7f909939d740 1 Processor -- start 2024-03-27T06:44:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.453+0000 7f909939d740 1 -- start start 2024-03-27T06:44:12.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.453+0000 7f909939d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f84e0 0x55dc9f702a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.453+0000 7f909939d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f8e00 0x55dc9f6f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.453+0000 7f909939d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 0x55dc9f711c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.453+0000 7f909939d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90901ab5d0 con 0x55dc9f7060c0 2024-03-27T06:44:12.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.453+0000 7f909939d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f90901ad110 con 0x55dc9f6f8e00 2024-03-27T06:44:12.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.453+0000 7f909939d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90901abca0 con 0x55dc9f6f84e0 2024-03-27T06:44:12.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.453+0000 7f9094c7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 0x55dc9f711c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f9094c7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 0x55dc9f711c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33508/0 (socket says 172.21.15.16:33508) 2024-03-27T06:44:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f9094c7d640 1 -- 172.21.15.16:0/3545479 learned_addr learned my addr 172.21.15.16:0/3545479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f908ffff640 1 --2- 172.21.15.16:0/3545479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f84e0 0x55dc9f702a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f908f7fe640 1 --2- 172.21.15.16:0/3545479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f8e00 0x55dc9f6f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f9094c7d640 1 -- 172.21.15.16:0/3545479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f84e0 msgr2=0x55dc9f702a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f9094c7d640 1 --2- 172.21.15.16:0/3545479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f84e0 0x55dc9f702a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f9094c7d640 1 -- 172.21.15.16:0/3545479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f8e00 msgr2=0x55dc9f6f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f9094c7d640 1 --2- 172.21.15.16:0/3545479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f8e00 0x55dc9f6f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f9094c7d640 1 -- 172.21.15.16:0/3545479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9f6f8c30 con 0x55dc9f7060c0 2024-03-27T06:44:12.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f9094c7d640 1 --2- 172.21.15.16:0/3545479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 0x55dc9f711c00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9088013260 tx=0x7f9088013290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4fb803fa359bd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f9085ffb640 1 -- 172.21.15.16:0/3545479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088004c00 con 0x55dc9f7060c0 2024-03-27T06:44:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f9085ffb640 1 -- 172.21.15.16:0/3545479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9088024030 con 0x55dc9f7060c0 2024-03-27T06:44:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.454+0000 7f9085ffb640 1 -- 172.21.15.16:0/3545479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088004420 con 0x55dc9f7060c0 2024-03-27T06:44:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 -- 172.21.15.16:0/3545479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 msgr2=0x55dc9f711c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 --2- 172.21.15.16:0/3545479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 0x55dc9f711c00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9088013260 tx=0x7f9088013290 comp rx=0 tx=0).stop 2024-03-27T06:44:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 -- 172.21.15.16:0/3545479 shutdown_connections 2024-03-27T06:44:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 --2- 172.21.15.16:0/3545479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f7060c0 0x55dc9f711c00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 --2- 172.21.15.16:0/3545479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f8e00 0x55dc9f6f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 --2- 172.21.15.16:0/3545479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f6f84e0 0x55dc9f702a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 -- 172.21.15.16:0/3545479 >> 172.21.15.16:0/3545479 conn(0x55dc9f6f2ee0 msgr2=0x55dc9f703100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 -- 172.21.15.16:0/3545479 shutdown_connections 2024-03-27T06:44:12.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 -- 172.21.15.16:0/3545479 wait complete. 2024-03-27T06:44:12.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 -- start start 2024-03-27T06:44:12.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 Processor -- start 2024-03-27T06:44:12.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f70e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f8e00 0x55dc9f6ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f7060c0 0x55dc9f6fff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90901aca40 con 0x55dc9f6f84e0 2024-03-27T06:44:12.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f90901ac370 con 0x55dc9f6f8e00 2024-03-27T06:44:12.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.455+0000 7f909939d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90901a95d0 con 0x55dc9f7060c0 2024-03-27T06:44:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f909939d740 10 client.0 _ll_register_callbacks cb 0x7f9078004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f908ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f70e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f908ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f70e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33522/0 (socket says 172.21.15.16:33522) 2024-03-27T06:44:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f908ffff640 1 -- 172.21.15.16:0/2981344405 learned_addr learned my addr 172.21.15.16:0/2981344405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f908f7fe640 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f8e00 0x55dc9f6ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f9094c7d640 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f7060c0 0x55dc9f6fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f908ffff640 1 -- 172.21.15.16:0/2981344405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f7060c0 msgr2=0x55dc9f6fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f908ffff640 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f7060c0 0x55dc9f6fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f908ffff640 1 -- 172.21.15.16:0/2981344405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f8e00 msgr2=0x55dc9f6ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f908ffff640 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f8e00 0x55dc9f6ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f908ffff640 1 -- 172.21.15.16:0/2981344405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc9f700550 con 0x55dc9f6f84e0 2024-03-27T06:44:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f908ffff640 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f70e420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f907800a010 tx=0x7f9078011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca95e78f44d67ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f906f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f906f7fe640 1 -- 172.21.15.16:0/2981344405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078035070 con 0x55dc9f6f84e0 2024-03-27T06:44:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f906f7fe640 1 -- 172.21.15.16:0/2981344405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f907802f450 con 0x55dc9f6f84e0 2024-03-27T06:44:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f909939d740 1 -- 172.21.15.16:0/2981344405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9088024030 con 0x55dc9f6f84e0 2024-03-27T06:44:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f906f7fe640 1 -- 172.21.15.16:0/2981344405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f907802f650 con 0x55dc9f6f84e0 2024-03-27T06:44:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f909939d740 10 client.15738 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:12.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.456+0000 7f909939d740 1 -- 172.21.15.16:0/2981344405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc9f77c7a0 con 0x55dc9f6f84e0 2024-03-27T06:44:12.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f906f7fe640 1 -- 172.21.15.16:0/2981344405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9088024030 con 0x55dc9f6f84e0 2024-03-27T06:44:12.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f906f7fe640 1 -- 172.21.15.16:0/2981344405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f90780047a0 con 0x55dc9f6f84e0 2024-03-27T06:44:12.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f909939d740 10 client.15738 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:12.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f909939d740 10 client.15738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T06:44:12.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f909939d740 20 client.15738 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:12.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f909939d740 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55dc9f77cee0 0x55dc9f77f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f909939d740 4 client.15738 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:44:12.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f909939d740 1 -- 172.21.15.16:0/2981344405 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc9f663fa0 con 0x55dc9f77cee0 2024-03-27T06:44:12.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f908f7fe640 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55dc9f77cee0 0x55dc9f77f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f908f7fe640 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55dc9f77cee0 0x55dc9f77f300 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4deaef3dbe57388 server_cookie=a63b430be072fb86 in_seq=0 out_seq=0 2024-03-27T06:44:12.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f906f7fe640 10 client.15738 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:44:12.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f906f7fe640 1 -- 172.21.15.16:0/2981344405 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc9f663fa0 con 0x55dc9f77cee0 2024-03-27T06:44:12.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f906f7fe640 10 client.15738 handle_command_reply: tid=0 2024-03-27T06:44:12.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.457+0000 7f909939d740 1 client.15738 shutdown 2024-03-27T06:44:12.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f909939d740 1 -- 172.21.15.16:0/2981344405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 msgr2=0x55dc9f70e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f909939d740 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f70e420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f907800a010 tx=0x7f9078011920 comp rx=0 tx=0).stop 2024-03-27T06:44:12.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f909939d740 1 -- 172.21.15.16:0/2981344405 shutdown_connections 2024-03-27T06:44:12.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f909939d740 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55dc9f77cee0 0x55dc9f77f300 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f909939d740 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc9f7060c0 0x55dc9f6fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f909939d740 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dc9f6f8e00 0x55dc9f6ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f906f7fe640 0 client.15738 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:44:12.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f909939d740 1 --2- 172.21.15.16:0/2981344405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc9f6f84e0 0x55dc9f70e420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f909939d740 1 -- 172.21.15.16:0/2981344405 >> 172.21.15.16:0/2981344405 conn(0x55dc9f6f2ee0 msgr2=0x55dc9f6f49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f909939d740 1 -- 172.21.15.16:0/2981344405 shutdown_connections 2024-03-27T06:44:12.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f909939d740 1 -- 172.21.15.16:0/2981344405 wait complete. 2024-03-27T06:44:12.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f909939d740 20 client.15738 trim_cache size 0 max 16384 2024-03-27T06:44:12.482 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:44:12.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 -- 172.21.15.16:0/3222264548 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f907c0a3930 msgr2=0x7f907c0a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f907c0a3930 0x7f907c0a5df0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9078009bd0 tx=0x7f9078005f90 comp rx=0 tx=0).stop 2024-03-27T06:44:12.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 -- 172.21.15.16:0/3222264548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9090101b20 msgr2=0x7f9090198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9090101b20 0x7f9090198970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9080036040 tx=0x7f908002dc00 comp rx=0 tx=0).stop 2024-03-27T06:44:12.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 -- 172.21.15.16:0/3222264548 shutdown_connections 2024-03-27T06:44:12.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9090105a40 0x7f909019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f907c0a3930 0x7f907c0a5df0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9090101b20 0x7f9090198970 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 --2- 172.21.15.16:0/3222264548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9090101160 0x7f9090198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 -- 172.21.15.16:0/3222264548 >> 172.21.15.16:0/3222264548 conn(0x7f9090077010 msgr2=0x7f90900fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 -- 172.21.15.16:0/3222264548 shutdown_connections 2024-03-27T06:44:12.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.458+0000 7f90966e0640 1 -- 172.21.15.16:0/3222264548 wait complete. 2024-03-27T06:44:12.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-27T06:44:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.598+0000 7f3565839640 1 -- 172.21.15.16:0/958799596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3560106a80 msgr2=0x7f3560106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.598+0000 7f3565839640 1 --2- 172.21.15.16:0/958799596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3560106a80 0x7f3560106e60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3558009930 tx=0x7f355802bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.598+0000 7f3565839640 1 -- 172.21.15.16:0/958799596 shutdown_connections 2024-03-27T06:44:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.598+0000 7f3565839640 1 --2- 172.21.15.16:0/958799596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3560101430 0x7f356010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.598+0000 7f3565839640 1 --2- 172.21.15.16:0/958799596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3560100a70 0x7f3560100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.598+0000 7f3565839640 1 --2- 172.21.15.16:0/958799596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3560106a80 0x7f3560106e60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.598+0000 7f3565839640 1 -- 172.21.15.16:0/958799596 >> 172.21.15.16:0/958799596 conn(0x7f3560077bb0 msgr2=0x7f3560077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.598+0000 7f3565839640 1 -- 172.21.15.16:0/958799596 shutdown_connections 2024-03-27T06:44:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.598+0000 7f3565839640 1 -- 172.21.15.16:0/958799596 wait complete. 2024-03-27T06:44:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.598+0000 7f3565839640 1 Processor -- start 2024-03-27T06:44:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f3565839640 1 -- start start 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f3565839640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3560100a70 0x7f3560199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f3565839640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3560101430 0x7f3560199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f3565839640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3560106a80 0x7f35601a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f3565839640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3560110f30 con 0x7f3560100a70 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f3565839640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f3560110db0 con 0x7f3560106a80 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f3565839640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f35601110b0 con 0x7f3560101430 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f355effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3560100a70 0x7f3560199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f355effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3560100a70 0x7f3560199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33542/0 (socket says 172.21.15.16:33542) 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f355effd640 1 -- 172.21.15.16:0/311035193 learned_addr learned my addr 172.21.15.16:0/311035193 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f355e7fc640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3560101430 0x7f3560199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f355f7fe640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3560106a80 0x7f35601a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f355effd640 1 -- 172.21.15.16:0/311035193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3560101430 msgr2=0x7f3560199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f355effd640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3560101430 0x7f3560199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f355effd640 1 -- 172.21.15.16:0/311035193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3560106a80 msgr2=0x7f35601a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f355effd640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3560106a80 0x7f35601a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f355effd640 1 -- 172.21.15.16:0/311035193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35601a48f0 con 0x7f3560100a70 2024-03-27T06:44:12.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.599+0000 7f355effd640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3560100a70 0x7f3560199330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3558009230 tx=0x7f355802dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.600+0000 7f3564837640 1 -- 172.21.15.16:0/311035193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558043020 con 0x7f3560100a70 2024-03-27T06:44:12.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.600+0000 7f3565839640 1 -- 172.21.15.16:0/311035193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35601a4ac0 con 0x7f3560100a70 2024-03-27T06:44:12.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.600+0000 7f3564837640 1 -- 172.21.15.16:0/311035193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3558004250 con 0x7f3560100a70 2024-03-27T06:44:12.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.600+0000 7f3564837640 1 -- 172.21.15.16:0/311035193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558003970 con 0x7f3560100a70 2024-03-27T06:44:12.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.600+0000 7f3565839640 1 -- 172.21.15.16:0/311035193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35601a4fb0 con 0x7f3560100a70 2024-03-27T06:44:12.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f3564837640 1 -- 172.21.15.16:0/311035193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7f3558046020 con 0x7f3560100a70 2024-03-27T06:44:12.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f3568423740 1 Processor -- start 2024-03-27T06:44:12.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f3568423740 1 -- start start 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f3568423740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e3db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f3568423740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e3e0d0 0x5614d8e46770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f3568423740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e46cb0 0x5614d8e490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f3568423740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5614d8da1710 con 0x5614d8e46cb0 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f3568423740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5614d8d49af0 con 0x5614d8e3e0d0 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f3568423740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5614d8d9d6c0 con 0x5614d8e3d7b0 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f3564837640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f354c0a3930 0x7f354c0a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f355f7fe640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f354c0a3930 0x7f354c0a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f355e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e46cb0 0x5614d8e490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f355effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e3e0d0 0x5614d8e46770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f355e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e46cb0 0x5614d8e490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33554/0 (socket says 172.21.15.16:33554) 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.602+0000 7f355e7fc640 1 -- 172.21.15.16:0/1438049888 learned_addr learned my addr 172.21.15.16:0/1438049888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f355f7fe640 1 --2- 172.21.15.16:0/1438049888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e3db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f3564837640 1 -- 172.21.15.16:0/311035193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f35580f2270 con 0x7f3560100a70 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f355e7fc640 1 -- 172.21.15.16:0/1438049888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 msgr2=0x5614d8e3db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f355e7fc640 1 --2- 172.21.15.16:0/1438049888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e3db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f355e7fc640 1 -- 172.21.15.16:0/1438049888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e3e0d0 msgr2=0x5614d8e46770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f355e7fc640 1 --2- 172.21.15.16:0/1438049888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e3e0d0 0x5614d8e46770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f355e7fc640 1 -- 172.21.15.16:0/1438049888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614d8ab53f0 con 0x5614d8e46cb0 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f355e7fc640 1 --2- 172.21.15.16:0/1438049888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e46cb0 0x5614d8e490a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3548009c90 tx=0x7f3548011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75002564e52f3e74 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f355f7fe640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f354c0a3930 0x7f354c0a5df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3550005fd0 tx=0x7f3550009e80 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f3555ffb640 1 -- 172.21.15.16:0/1438049888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548019620 con 0x5614d8e46cb0 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f3555ffb640 1 -- 172.21.15.16:0/1438049888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3548019cc0 con 0x5614d8e46cb0 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.603+0000 7f3555ffb640 1 -- 172.21.15.16:0/1438049888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548018d10 con 0x5614d8e46cb0 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.604+0000 7f3568423740 1 -- 172.21.15.16:0/1438049888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e46cb0 msgr2=0x5614d8e490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.604+0000 7f3568423740 1 --2- 172.21.15.16:0/1438049888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e46cb0 0x5614d8e490a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3548009c90 tx=0x7f3548011810 comp rx=0 tx=0).stop 2024-03-27T06:44:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.604+0000 7f3568423740 1 -- 172.21.15.16:0/1438049888 shutdown_connections 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.604+0000 7f3568423740 1 --2- 172.21.15.16:0/1438049888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e46cb0 0x5614d8e490a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.604+0000 7f3568423740 1 --2- 172.21.15.16:0/1438049888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e3e0d0 0x5614d8e46770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.604+0000 7f3568423740 1 --2- 172.21.15.16:0/1438049888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e3db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.604+0000 7f3568423740 1 -- 172.21.15.16:0/1438049888 >> 172.21.15.16:0/1438049888 conn(0x5614d8e37ee0 msgr2=0x5614d8e382c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.604+0000 7f3568423740 1 -- 172.21.15.16:0/1438049888 shutdown_connections 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.604+0000 7f3568423740 1 -- 172.21.15.16:0/1438049888 wait complete. 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.604+0000 7f3568423740 1 -- start start 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.604+0000 7f3568423740 1 Processor -- start 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f3568423740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e43c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f3568423740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e44190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f3568423740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e46cb0 0x5614d8e446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f3568423740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5614d8e49aa0 con 0x5614d8e3e0d0 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f3568423740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5614d8d5ceb0 con 0x5614d8e46cb0 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f3568423740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5614d8e49c20 con 0x5614d8e3d7b0 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e44190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f3568423740 10 client.0 _ll_register_callbacks cb 0x5614d8d996a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e44190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33564/0 (socket says 172.21.15.16:33564) 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355e7fc640 1 -- 172.21.15.16:0/775818629 learned_addr learned my addr 172.21.15.16:0/775818629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355effd640 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e43c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355f7fe640 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e46cb0 0x5614d8e446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355e7fc640 1 -- 172.21.15.16:0/775818629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 msgr2=0x5614d8e43c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355e7fc640 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e43c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355e7fc640 1 -- 172.21.15.16:0/775818629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e46cb0 msgr2=0x5614d8e446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355e7fc640 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e46cb0 0x5614d8e446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355e7fc640 1 -- 172.21.15.16:0/775818629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614d8b7f7d0 con 0x5614d8e3e0d0 2024-03-27T06:44:12.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355effd640 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e43c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f355e7fc640 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e44190 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f354801b040 tx=0x7f3548005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb84feff42f9b85 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f353effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f353effd640 1 -- 172.21.15.16:0/775818629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548029020 con 0x5614d8e3e0d0 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f353effd640 1 -- 172.21.15.16:0/775818629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3548005040 con 0x5614d8e3e0d0 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.605+0000 7f3568423740 1 -- 172.21.15.16:0/775818629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3548019cc0 con 0x5614d8e3e0d0 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f353effd640 1 -- 172.21.15.16:0/775818629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3548034050 con 0x5614d8e3e0d0 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f3568423740 10 client.15750 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f3568423740 1 -- 172.21.15.16:0/775818629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614d8e55ca0 con 0x5614d8e3e0d0 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f353effd640 1 -- 172.21.15.16:0/775818629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3548019cc0 con 0x5614d8e3e0d0 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f353effd640 1 -- 172.21.15.16:0/775818629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f35480046c0 con 0x5614d8e3e0d0 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f3568423740 10 client.15750 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f3568423740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f3568423740 20 client.15750 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:12.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f3568423740 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5614d8e57160 0x5614d8e59550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f3568423740 4 client.15750 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f3568423740 1 -- 172.21.15.16:0/775818629 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614d8daec50 con 0x5614d8e57160 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f355effd640 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5614d8e57160 0x5614d8e59550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f355effd640 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5614d8e57160 0x5614d8e59550 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6de097813074873d server_cookie=aba824a5a95e059b in_seq=0 out_seq=0 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.606+0000 7f353effd640 10 client.15750 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f353effd640 1 -- 172.21.15.16:0/775818629 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5614d8daec50 con 0x5614d8e57160 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f353effd640 10 client.15750 handle_command_reply: tid=0 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f3568423740 1 client.15750 shutdown 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f3568423740 1 -- 172.21.15.16:0/775818629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 msgr2=0x5614d8e44190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f3568423740 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e44190 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f354801b040 tx=0x7f3548005d30 comp rx=0 tx=0).stop 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f3568423740 1 -- 172.21.15.16:0/775818629 shutdown_connections 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f3568423740 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5614d8e57160 0x5614d8e59550 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f3568423740 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e46cb0 0x5614d8e446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f3568423740 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e44190 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f353effd640 0 client.15750 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:12.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f3568423740 1 --2- 172.21.15.16:0/775818629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e43c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f3568423740 1 -- 172.21.15.16:0/775818629 >> 172.21.15.16:0/775818629 conn(0x5614d8e37ee0 msgr2=0x5614d8e39c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.607+0000 7f3568423740 1 -- 172.21.15.16:0/775818629 shutdown_connections 2024-03-27T06:44:12.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.608+0000 7f3568423740 1 -- 172.21.15.16:0/775818629 wait complete. 2024-03-27T06:44:12.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.608+0000 7f3568423740 20 client.15750 trim_cache size 0 max 16384 2024-03-27T06:44:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f3568423740 1 Processor -- start 2024-03-27T06:44:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f3568423740 1 -- start start 2024-03-27T06:44:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f3568423740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e3d7b0 0x5614d8e46380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f3568423740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e52840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f3568423740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e46cb0 0x5614d8e5eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f3568423740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35601ad800 con 0x5614d8e3e0d0 2024-03-27T06:44:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f3568423740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f35601af340 con 0x5614d8e3d7b0 2024-03-27T06:44:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f3568423740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f35601aded0 con 0x5614d8e46cb0 2024-03-27T06:44:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e52840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e3d7b0 0x5614d8e46380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e52840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33566/0 (socket says 172.21.15.16:33566) 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355e7fc640 1 -- 172.21.15.16:0/660533248 learned_addr learned my addr 172.21.15.16:0/660533248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355f7fe640 1 --2- 172.21.15.16:0/660533248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e46cb0 0x5614d8e5eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355e7fc640 1 -- 172.21.15.16:0/660533248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e46cb0 msgr2=0x5614d8e5eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355e7fc640 1 --2- 172.21.15.16:0/660533248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e46cb0 0x5614d8e5eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355e7fc640 1 -- 172.21.15.16:0/660533248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e3d7b0 msgr2=0x5614d8e46380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355e7fc640 1 --2- 172.21.15.16:0/660533248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e3d7b0 0x5614d8e46380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355e7fc640 1 -- 172.21.15.16:0/660533248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614d8e37d10 con 0x5614d8e3e0d0 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355effd640 1 --2- 172.21.15.16:0/660533248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e3d7b0 0x5614d8e46380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.621+0000 7f355f7fe640 1 --2- 172.21.15.16:0/660533248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e46cb0 0x5614d8e5eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f355e7fc640 1 --2- 172.21.15.16:0/660533248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e52840 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3548009820 tx=0x7f354801afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14a571daa13f9a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f35557fa640 1 -- 172.21.15.16:0/660533248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548029040 con 0x5614d8e3e0d0 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f35557fa640 1 -- 172.21.15.16:0/660533248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3548018d20 con 0x5614d8e3e0d0 2024-03-27T06:44:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f35557fa640 1 -- 172.21.15.16:0/660533248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548004430 con 0x5614d8e3e0d0 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f3568423740 1 -- 172.21.15.16:0/660533248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 msgr2=0x5614d8e52840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f3568423740 1 --2- 172.21.15.16:0/660533248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e52840 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3548009820 tx=0x7f354801afd0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f3568423740 1 -- 172.21.15.16:0/660533248 shutdown_connections 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f3568423740 1 --2- 172.21.15.16:0/660533248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e46cb0 0x5614d8e5eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f3568423740 1 --2- 172.21.15.16:0/660533248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e52840 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f3568423740 1 --2- 172.21.15.16:0/660533248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e3d7b0 0x5614d8e46380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f3568423740 1 -- 172.21.15.16:0/660533248 >> 172.21.15.16:0/660533248 conn(0x5614d8e37ee0 msgr2=0x5614d8e444a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f3568423740 1 -- 172.21.15.16:0/660533248 shutdown_connections 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.622+0000 7f3568423740 1 -- 172.21.15.16:0/660533248 wait complete. 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f3568423740 1 -- start start 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f3568423740 1 Processor -- start 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f3568423740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e40710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f3568423740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e40c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f3568423740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e46cb0 0x5614d8e41190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f3568423740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35601aec70 con 0x5614d8e3e0d0 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f3568423740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f35601ae5a0 con 0x5614d8e46cb0 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f3568423740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f35601ab7e0 con 0x5614d8e3d7b0 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f3568423740 10 client.0 _ll_register_callbacks cb 0x7f3558005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e40c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e40c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33582/0 (socket says 172.21.15.16:33582) 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355e7fc640 1 -- 172.21.15.16:0/1927463599 learned_addr learned my addr 172.21.15.16:0/1927463599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355f7fe640 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e46cb0 0x5614d8e41190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355effd640 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e40710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355e7fc640 1 -- 172.21.15.16:0/1927463599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 msgr2=0x5614d8e40710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355e7fc640 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e40710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355e7fc640 1 -- 172.21.15.16:0/1927463599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e46cb0 msgr2=0x5614d8e41190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355e7fc640 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e46cb0 0x5614d8e41190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355e7fc640 1 -- 172.21.15.16:0/1927463599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614d8ebc3d0 con 0x5614d8e3e0d0 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355f7fe640 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e46cb0 0x5614d8e41190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.623+0000 7f355effd640 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e40710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f355e7fc640 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e40c50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3548005d70 tx=0x7f3548004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4448aedd6e7eef85 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f353effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f353effd640 1 -- 172.21.15.16:0/1927463599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548029040 con 0x5614d8e3e0d0 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f353effd640 1 -- 172.21.15.16:0/1927463599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3548004860 con 0x5614d8e3e0d0 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f3568423740 1 -- 172.21.15.16:0/1927463599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3548018d20 con 0x5614d8e3e0d0 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f353effd640 1 -- 172.21.15.16:0/1927463599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3548018760 con 0x5614d8e3e0d0 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f3568423740 10 client.15756 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f3568423740 1 -- 172.21.15.16:0/1927463599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614d8ebd550 con 0x5614d8e3e0d0 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f353effd640 1 -- 172.21.15.16:0/1927463599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3548018d20 con 0x5614d8e3e0d0 2024-03-27T06:44:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f353effd640 1 -- 172.21.15.16:0/1927463599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3548019680 con 0x5614d8e3e0d0 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f3568423740 10 client.15756 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f3568423740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f3568423740 20 client.15756 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f3568423740 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5614d8e57160 0x5614d8ebfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f3568423740 4 client.15756 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f3568423740 1 -- 172.21.15.16:0/1927463599 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614d8da8fa0 con 0x5614d8e57160 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.624+0000 7f355effd640 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5614d8e57160 0x5614d8ebfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f355effd640 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5614d8e57160 0x5614d8ebfc30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa05d35340db40f server_cookie=107e35ed5a15786 in_seq=0 out_seq=0 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f353effd640 10 client.15756 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f353effd640 1 -- 172.21.15.16:0/1927463599 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614d8da8fa0 con 0x5614d8e57160 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f353effd640 10 client.15756 handle_command_reply: tid=0 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 1 client.15756 shutdown 2024-03-27T06:44:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 1 -- 172.21.15.16:0/1927463599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 msgr2=0x5614d8e40c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e40c50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3548005d70 tx=0x7f3548004460 comp rx=0 tx=0).stop 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 1 -- 172.21.15.16:0/1927463599 shutdown_connections 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5614d8e57160 0x5614d8ebfc30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614d8e46cb0 0x5614d8e41190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8e3e0d0 0x5614d8e40c50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f353effd640 0 client.15756 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 1 --2- 172.21.15.16:0/1927463599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5614d8e3d7b0 0x5614d8e40710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 1 -- 172.21.15.16:0/1927463599 >> 172.21.15.16:0/1927463599 conn(0x5614d8e37ee0 msgr2=0x5614d8e36770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 1 -- 172.21.15.16:0/1927463599 shutdown_connections 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 1 -- 172.21.15.16:0/1927463599 wait complete. 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.625+0000 7f3568423740 20 client.15756 trim_cache size 0 max 16384 2024-03-27T06:44:12.628 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:44:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 -- 172.21.15.16:0/311035193 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f354c0a3930 msgr2=0x7f354c0a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f354c0a3930 0x7f354c0a5df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3550005fd0 tx=0x7f3550009e80 comp rx=0 tx=0).stop 2024-03-27T06:44:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 -- 172.21.15.16:0/311035193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3560100a70 msgr2=0x7f3560199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3560100a70 0x7f3560199330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3558009230 tx=0x7f355802dc90 comp rx=0 tx=0).stop 2024-03-27T06:44:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 -- 172.21.15.16:0/311035193 shutdown_connections 2024-03-27T06:44:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3560106a80 0x7f35601a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f354c0a3930 0x7f354c0a5df0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3560101430 0x7f3560199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 --2- 172.21.15.16:0/311035193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3560100a70 0x7f3560199330 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 -- 172.21.15.16:0/311035193 >> 172.21.15.16:0/311035193 conn(0x7f3560077bb0 msgr2=0x7f3560103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 -- 172.21.15.16:0/311035193 shutdown_connections 2024-03-27T06:44:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:12.626+0000 7f3565839640 1 -- 172.21.15.16:0/311035193 wait complete. 2024-03-27T06:44:13.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:12 smithi156 ceph-mon[36261]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 34 op/s 2024-03-27T06:44:13.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:12 smithi170 ceph-mon[36438]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 34 op/s 2024-03-27T06:44:13.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:12 smithi016 ceph-mon[31908]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 34 op/s 2024-03-27T06:44:13.632 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-27T06:44:13.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:44:13.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-27T06:44:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.757+0000 7fab16d55640 1 -- 172.21.15.16:0/1030322479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab10107b80 msgr2=0x7fab10107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.757+0000 7fab16d55640 1 --2- 172.21.15.16:0/1030322479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab10107b80 0x7fab10107f60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fab04009930 tx=0x7fab0402bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.757+0000 7fab16d55640 1 -- 172.21.15.16:0/1030322479 shutdown_connections 2024-03-27T06:44:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.757+0000 7fab16d55640 1 --2- 172.21.15.16:0/1030322479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1006aba0 0x7fab1010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.757+0000 7fab16d55640 1 --2- 172.21.15.16:0/1030322479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1006a1e0 0x7fab1006a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.757+0000 7fab16d55640 1 --2- 172.21.15.16:0/1030322479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab10107b80 0x7fab10107f60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.757+0000 7fab16d55640 1 -- 172.21.15.16:0/1030322479 >> 172.21.15.16:0/1030322479 conn(0x7fab10077b20 msgr2=0x7fab10077f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.757+0000 7fab16d55640 1 -- 172.21.15.16:0/1030322479 shutdown_connections 2024-03-27T06:44:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.757+0000 7fab16d55640 1 -- 172.21.15.16:0/1030322479 wait complete. 2024-03-27T06:44:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab16d55640 1 Processor -- start 2024-03-27T06:44:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab16d55640 1 -- start start 2024-03-27T06:44:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab16d55640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1006a1e0 0x7fab10198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab16d55640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1006aba0 0x7fab10198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab16d55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab10107b80 0x7fab1019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab16d55640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab10111fd0 con 0x7fab10107b80 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab16d55640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fab10111e50 con 0x7fab1006a1e0 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab16d55640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab10112150 con 0x7fab1006aba0 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab152f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab10107b80 0x7fab1019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab0ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1006aba0 0x7fab10198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab14af1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1006a1e0 0x7fab10198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab152f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab10107b80 0x7fab1019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33602/0 (socket says 172.21.15.16:33602) 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab0ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1006aba0 0x7fab10198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:40876/0 (socket says 172.21.15.16:40876) 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab152f2640 1 -- 172.21.15.16:0/902679946 learned_addr learned my addr 172.21.15.16:0/902679946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab152f2640 1 -- 172.21.15.16:0/902679946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1006aba0 msgr2=0x7fab10198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab152f2640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1006aba0 0x7fab10198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab152f2640 1 -- 172.21.15.16:0/902679946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1006a1e0 msgr2=0x7fab10198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab152f2640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1006a1e0 0x7fab10198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab152f2640 1 -- 172.21.15.16:0/902679946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1019d4f0 con 0x7fab10107b80 2024-03-27T06:44:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.758+0000 7fab152f2640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab10107b80 0x7fab1019cdc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fab0800eb10 tx=0x7fab0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.759+0000 7fab0dffb640 1 -- 172.21.15.16:0/902679946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08004b20 con 0x7fab10107b80 2024-03-27T06:44:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.759+0000 7fab0dffb640 1 -- 172.21.15.16:0/902679946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fab08004cc0 con 0x7fab10107b80 2024-03-27T06:44:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.759+0000 7fab0dffb640 1 -- 172.21.15.16:0/902679946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab080107f0 con 0x7fab10107b80 2024-03-27T06:44:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.759+0000 7fab16d55640 1 -- 172.21.15.16:0/902679946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1019d7e0 con 0x7fab10107b80 2024-03-27T06:44:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.759+0000 7fab16d55640 1 -- 172.21.15.16:0/902679946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab101a5070 con 0x7fab10107b80 2024-03-27T06:44:13.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.761+0000 7fab0dffb640 1 -- 172.21.15.16:0/902679946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7fab08005020 con 0x7fab10107b80 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.761+0000 7fab19a0f740 1 Processor -- start 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.761+0000 7fab19a0f740 1 -- start start 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.761+0000 7fab0dffb640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fab000a3930 0x7fab000a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b51230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab0ffff640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fab000a3930 0x7fab000a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b51770 0x564643b59e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b5a350 0x564643b5c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab0dffb640 1 -- 172.21.15.16:0/902679946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fab080d0870 con 0x7fab10107b80 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab19a0f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564643a5bc20 con 0x564643b50e50 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab19a0f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x564643ab4430 con 0x564643b51770 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab19a0f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564643a5c900 con 0x564643b5a350 2024-03-27T06:44:13.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab14af1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b51230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab14af1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b51230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33610/0 (socket says 172.21.15.16:33610) 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab14af1640 1 -- 172.21.15.16:0/480660347 learned_addr learned my addr 172.21.15.16:0/480660347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab0ffff640 1 --2- 172.21.15.16:0/480660347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b51770 0x564643b59e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab152f2640 1 --2- 172.21.15.16:0/480660347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b5a350 0x564643b5c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab14af1640 1 -- 172.21.15.16:0/480660347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b5a350 msgr2=0x564643b5c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab14af1640 1 --2- 172.21.15.16:0/480660347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b5a350 0x564643b5c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab14af1640 1 -- 172.21.15.16:0/480660347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b51770 msgr2=0x564643b59e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab14af1640 1 --2- 172.21.15.16:0/480660347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b51770 0x564643b59e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab14af1640 1 -- 172.21.15.16:0/480660347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564643aaf180 con 0x564643b50e50 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab0ffff640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fab000a3930 0x7fab000a5df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab101004c0 tx=0x7faafc011920 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7fab14af1640 1 --2- 172.21.15.16:0/480660347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b51230 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fab04036040 tx=0x7fab04009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9932231df078e95b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7faafaffd640 1 -- 172.21.15.16:0/480660347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0402c690 con 0x564643b50e50 2024-03-27T06:44:13.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.762+0000 7faafaffd640 1 -- 172.21.15.16:0/480660347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fab04004040 con 0x564643b50e50 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.763+0000 7faafaffd640 1 -- 172.21.15.16:0/480660347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab040046c0 con 0x564643b50e50 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.763+0000 7fab19a0f740 1 -- 172.21.15.16:0/480660347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 msgr2=0x564643b51230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.763+0000 7fab19a0f740 1 --2- 172.21.15.16:0/480660347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b51230 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fab04036040 tx=0x7fab04009560 comp rx=0 tx=0).stop 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.763+0000 7fab19a0f740 1 -- 172.21.15.16:0/480660347 shutdown_connections 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.763+0000 7fab19a0f740 1 --2- 172.21.15.16:0/480660347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b5a350 0x564643b5c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.763+0000 7fab19a0f740 1 --2- 172.21.15.16:0/480660347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b51770 0x564643b59e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.763+0000 7fab19a0f740 1 --2- 172.21.15.16:0/480660347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b51230 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.763+0000 7fab19a0f740 1 -- 172.21.15.16:0/480660347 >> 172.21.15.16:0/480660347 conn(0x564643b4b580 msgr2=0x564643b4b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.763+0000 7fab19a0f740 1 -- 172.21.15.16:0/480660347 shutdown_connections 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.763+0000 7fab19a0f740 1 -- 172.21.15.16:0/480660347 wait complete. 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab19a0f740 1 -- start start 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab19a0f740 1 Processor -- start 2024-03-27T06:44:13.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b59cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b51770 0x564643b57040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab19a0f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564643a6fcc0 con 0x564643b50e50 2024-03-27T06:44:13.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab19a0f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x564643ab03e0 con 0x564643b5a350 2024-03-27T06:44:13.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab19a0f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564643b5d0e0 con 0x564643b51770 2024-03-27T06:44:13.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab19a0f740 10 client.0 _ll_register_callbacks cb 0x564643aaaf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:13.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab14af1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b59cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab14af1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b59cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33620/0 (socket says 172.21.15.16:33620) 2024-03-27T06:44:13.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab14af1640 1 -- 172.21.15.16:0/1156296007 learned_addr learned my addr 172.21.15.16:0/1156296007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:13.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab0ffff640 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b51770 0x564643b57040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab152f2640 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab14af1640 1 -- 172.21.15.16:0/1156296007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b51770 msgr2=0x564643b57040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab14af1640 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b51770 0x564643b57040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab14af1640 1 -- 172.21.15.16:0/1156296007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 msgr2=0x564643b575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab14af1640 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab14af1640 1 -- 172.21.15.16:0/1156296007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564643a76ef0 con 0x564643b50e50 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.764+0000 7fab152f2640 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7fab14af1640 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b59cb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab04005ab0 tx=0x7fab0402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6a250a25a4df7a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7faaeffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7faaeffff640 1 -- 172.21.15.16:0/1156296007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0404f030 con 0x564643b50e50 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7faaeffff640 1 -- 172.21.15.16:0/1156296007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fab04004400 con 0x564643b50e50 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7fab19a0f740 1 -- 172.21.15.16:0/1156296007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab04004040 con 0x564643b50e50 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7faaeffff640 1 -- 172.21.15.16:0/1156296007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab0402c9d0 con 0x564643b50e50 2024-03-27T06:44:13.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7fab19a0f740 10 client.15768 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7fab19a0f740 1 -- 172.21.15.16:0/1156296007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5646438907e0 con 0x564643b50e50 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7faaeffff640 1 -- 172.21.15.16:0/1156296007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fab04004040 con 0x564643b50e50 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7faaeffff640 1 -- 172.21.15.16:0/1156296007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fab040046c0 con 0x564643b50e50 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7fab19a0f740 10 client.15768 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7fab19a0f740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7fab19a0f740 20 client.15768 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7fab19a0f740 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564643b6a050 0x564643b6c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7fab19a0f740 4 client.15768 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.765+0000 7fab19a0f740 1 -- 172.21.15.16:0/1156296007 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564643ac1970 con 0x564643b6a050 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.766+0000 7fab0ffff640 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564643b6a050 0x564643b6c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.766+0000 7fab0ffff640 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564643b6a050 0x564643b6c440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caa989b7567e15ea server_cookie=1623f4b8cc9fc0d7 in_seq=0 out_seq=0 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.766+0000 7faaeffff640 10 client.15768 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7faaeffff640 1 -- 172.21.15.16:0/1156296007 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564643ac1970 con 0x564643b6a050 2024-03-27T06:44:13.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7faaeffff640 10 client.15768 handle_command_reply: tid=0 2024-03-27T06:44:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 1 client.15768 shutdown 2024-03-27T06:44:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 1 -- 172.21.15.16:0/1156296007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 msgr2=0x564643b59cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b59cb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab04005ab0 tx=0x7fab0402bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 1 -- 172.21.15.16:0/1156296007 shutdown_connections 2024-03-27T06:44:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564643b6a050 0x564643b6c440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b51770 0x564643b57040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7faaeffff640 0 client.15768 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 1 --2- 172.21.15.16:0/1156296007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b59cb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 1 -- 172.21.15.16:0/1156296007 >> 172.21.15.16:0/1156296007 conn(0x564643b4b580 msgr2=0x564643b4d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 1 -- 172.21.15.16:0/1156296007 shutdown_connections 2024-03-27T06:44:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 1 -- 172.21.15.16:0/1156296007 wait complete. 2024-03-27T06:44:13.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.767+0000 7fab19a0f740 20 client.15768 trim_cache size 0 max 16384 2024-03-27T06:44:13.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab19a0f740 1 Processor -- start 2024-03-27T06:44:13.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab19a0f740 1 -- start start 2024-03-27T06:44:13.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b4a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b51770 0x564643b71a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b62320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab19a0f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab101ad930 con 0x564643b50e50 2024-03-27T06:44:13.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab19a0f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fab101af470 con 0x564643b5a350 2024-03-27T06:44:13.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab19a0f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab101ae000 con 0x564643b51770 2024-03-27T06:44:13.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab14af1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b4a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab14af1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b4a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33636/0 (socket says 172.21.15.16:33636) 2024-03-27T06:44:13.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab0ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b51770 0x564643b71a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab14af1640 1 -- 172.21.15.16:0/3506471741 learned_addr learned my addr 172.21.15.16:0/3506471741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:13.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab152f2640 1 --2- 172.21.15.16:0/3506471741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b62320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab14af1640 1 -- 172.21.15.16:0/3506471741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b51770 msgr2=0x564643b71a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab14af1640 1 --2- 172.21.15.16:0/3506471741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b51770 0x564643b71a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab14af1640 1 -- 172.21.15.16:0/3506471741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 msgr2=0x564643b62320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab14af1640 1 --2- 172.21.15.16:0/3506471741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b62320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab14af1640 1 -- 172.21.15.16:0/3506471741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564643b515a0 con 0x564643b50e50 2024-03-27T06:44:13.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.785+0000 7fab14af1640 1 --2- 172.21.15.16:0/3506471741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b4a590 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fab04004a60 tx=0x7fab040097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ae0f000bffba0d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7faafa7fc640 1 -- 172.21.15.16:0/3506471741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04009520 con 0x564643b50e50 2024-03-27T06:44:13.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7faafa7fc640 1 -- 172.21.15.16:0/3506471741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fab04050020 con 0x564643b50e50 2024-03-27T06:44:13.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7faafa7fc640 1 -- 172.21.15.16:0/3506471741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0402cdd0 con 0x564643b50e50 2024-03-27T06:44:13.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7fab19a0f740 1 -- 172.21.15.16:0/3506471741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 msgr2=0x564643b4a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7fab19a0f740 1 --2- 172.21.15.16:0/3506471741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b4a590 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fab04004a60 tx=0x7fab040097c0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7fab19a0f740 1 -- 172.21.15.16:0/3506471741 shutdown_connections 2024-03-27T06:44:13.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7fab19a0f740 1 --2- 172.21.15.16:0/3506471741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b62320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7fab19a0f740 1 --2- 172.21.15.16:0/3506471741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b51770 0x564643b71a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7fab19a0f740 1 --2- 172.21.15.16:0/3506471741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b50e50 0x564643b4a590 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7fab19a0f740 1 -- 172.21.15.16:0/3506471741 >> 172.21.15.16:0/3506471741 conn(0x564643b4b580 msgr2=0x564643b51c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7fab19a0f740 1 -- 172.21.15.16:0/3506471741 shutdown_connections 2024-03-27T06:44:13.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7fab19a0f740 1 -- 172.21.15.16:0/3506471741 wait complete. 2024-03-27T06:44:13.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.786+0000 7fab19a0f740 1 -- start start 2024-03-27T06:44:13.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab19a0f740 1 Processor -- start 2024-03-27T06:44:13.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b50e50 0x564643b62520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b51770 0x564643b5f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab19a0f740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b5f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab19a0f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab101aeda0 con 0x564643b51770 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab19a0f740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fab101ae6d0 con 0x564643b5a350 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab19a0f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab101ab910 con 0x564643b50e50 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab19a0f740 10 client.0 _ll_register_callbacks cb 0x7fab08004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b51770 0x564643b5f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b51770 0x564643b5f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33646/0 (socket says 172.21.15.16:33646) 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab0ffff640 1 -- 172.21.15.16:0/2822707406 learned_addr learned my addr 172.21.15.16:0/2822707406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab14af1640 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b50e50 0x564643b62520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab152f2640 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b5f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab0ffff640 1 -- 172.21.15.16:0/2822707406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b50e50 msgr2=0x564643b62520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab0ffff640 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b50e50 0x564643b62520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab0ffff640 1 -- 172.21.15.16:0/2822707406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 msgr2=0x564643b5f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab0ffff640 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b5f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.787+0000 7fab0ffff640 1 -- 172.21.15.16:0/2822707406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646438f0f70 con 0x564643b51770 2024-03-27T06:44:13.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7fab0ffff640 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b51770 0x564643b5f300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faafc013010 tx=0x7faafc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f48d6b110eba806 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7faaeffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:13.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7faaeffff640 1 -- 172.21.15.16:0/2822707406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc01a040 con 0x564643b51770 2024-03-27T06:44:13.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7faaeffff640 1 -- 172.21.15.16:0/2822707406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7faafc003ac0 con 0x564643b51770 2024-03-27T06:44:13.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7fab19a0f740 1 -- 172.21.15.16:0/2822707406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab04050020 con 0x564643b51770 2024-03-27T06:44:13.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7faaeffff640 1 -- 172.21.15.16:0/2822707406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faafc03a050 con 0x564643b51770 2024-03-27T06:44:13.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7fab19a0f740 10 client.15774 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:13.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7fab19a0f740 1 -- 172.21.15.16:0/2822707406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564643788170 con 0x564643b51770 2024-03-27T06:44:13.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7faaeffff640 1 -- 172.21.15.16:0/2822707406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fab04050020 con 0x564643b51770 2024-03-27T06:44:13.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7faaeffff640 1 -- 172.21.15.16:0/2822707406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7faafc005310 con 0x564643b51770 2024-03-27T06:44:13.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7fab19a0f740 10 client.15774 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:13.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7fab19a0f740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:13.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7fab19a0f740 20 client.15774 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:13.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7fab19a0f740 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564643bd1490 0x564643bd38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7fab19a0f740 4 client.15774 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-27T06:44:13.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.788+0000 7fab19a0f740 1 -- 172.21.15.16:0/2822707406 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564643abbcc0 con 0x564643bd1490 2024-03-27T06:44:13.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.789+0000 7fab14af1640 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564643bd1490 0x564643bd38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.789+0000 7faaeffff640 10 client.15774 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:13.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.789+0000 7fab14af1640 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564643bd1490 0x564643bd38b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=633bdc97d130f853 server_cookie=fe3c500eda6aa841 in_seq=0 out_seq=0 2024-03-27T06:44:13.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7faaeffff640 1 -- 172.21.15.16:0/2822707406 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564643abbcc0 con 0x564643bd1490 2024-03-27T06:44:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7faaeffff640 10 client.15774 handle_command_reply: tid=0 2024-03-27T06:44:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 1 client.15774 shutdown 2024-03-27T06:44:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 1 -- 172.21.15.16:0/2822707406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b51770 msgr2=0x564643b5f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b51770 0x564643b5f300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faafc013010 tx=0x7faafc004730 comp rx=0 tx=0).stop 2024-03-27T06:44:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 1 -- 172.21.15.16:0/2822707406 shutdown_connections 2024-03-27T06:44:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x564643bd1490 0x564643bd38b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564643b5a350 0x564643b5f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564643b51770 0x564643b5f300 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 1 --2- 172.21.15.16:0/2822707406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564643b50e50 0x564643b62520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 1 -- 172.21.15.16:0/2822707406 >> 172.21.15.16:0/2822707406 conn(0x564643b4b580 msgr2=0x564643b4a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7faaeffff640 0 client.15774 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:13.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 1 -- 172.21.15.16:0/2822707406 shutdown_connections 2024-03-27T06:44:13.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 1 -- 172.21.15.16:0/2822707406 wait complete. 2024-03-27T06:44:13.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.790+0000 7fab19a0f740 20 client.15774 trim_cache size 0 max 16384 2024-03-27T06:44:13.798 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:44:13.799 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-03-27T06:44:13.799 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "e310668e-5ad9-4e0b-9bf1-59fb212f2c9f", 2024-03-27T06:44:13.799 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-03-27T06:44:13.799 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:44:13.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 -- 172.21.15.16:0/902679946 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fab000a3930 msgr2=0x7fab000a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fab000a3930 0x7fab000a5df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab101004c0 tx=0x7faafc011920 comp rx=0 tx=0).stop 2024-03-27T06:44:13.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 -- 172.21.15.16:0/902679946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab10107b80 msgr2=0x7fab1019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab10107b80 0x7fab1019cdc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fab0800eb10 tx=0x7fab0800c2d0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 -- 172.21.15.16:0/902679946 shutdown_connections 2024-03-27T06:44:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab10107b80 0x7fab1019cdc0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fab000a3930 0x7fab000a5df0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1006aba0 0x7fab10198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 --2- 172.21.15.16:0/902679946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab1006a1e0 0x7fab10198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 -- 172.21.15.16:0/902679946 >> 172.21.15.16:0/902679946 conn(0x7fab10077b20 msgr2=0x7fab100fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 -- 172.21.15.16:0/902679946 shutdown_connections 2024-03-27T06:44:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.791+0000 7fab16d55640 1 -- 172.21.15.16:0/902679946 wait complete. 2024-03-27T06:44:13.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T06:44:13.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T06:44:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.912+0000 7f394fb63640 1 -- 172.21.15.16:0/543508226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39481066d0 msgr2=0x7f3948106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.912+0000 7f394fb63640 1 --2- 172.21.15.16:0/543508226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39481066d0 0x7f3948106b50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3938009900 tx=0x7f393802bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.912+0000 7f394fb63640 1 -- 172.21.15.16:0/543508226 shutdown_connections 2024-03-27T06:44:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.912+0000 7f394fb63640 1 --2- 172.21.15.16:0/543508226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3948107090 0x7f394810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.912+0000 7f394fb63640 1 --2- 172.21.15.16:0/543508226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39481066d0 0x7f3948106b50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.912+0000 7f394fb63640 1 --2- 172.21.15.16:0/543508226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3948069130 0x7f3948069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.912+0000 7f394fb63640 1 -- 172.21.15.16:0/543508226 >> 172.21.15.16:0/543508226 conn(0x7f3948077b50 msgr2=0x7f3948077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394fb63640 1 -- 172.21.15.16:0/543508226 shutdown_connections 2024-03-27T06:44:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394fb63640 1 -- 172.21.15.16:0/543508226 wait complete. 2024-03-27T06:44:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394fb63640 1 Processor -- start 2024-03-27T06:44:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394fb63640 1 -- start start 2024-03-27T06:44:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394fb63640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3948069130 0x7f39481984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394fb63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39481066d0 0x7f3948198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394fb63640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3948107090 0x7f394819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394fb63640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3948111f60 con 0x7f39481066d0 2024-03-27T06:44:13.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394fb63640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f3948111de0 con 0x7f3948069130 2024-03-27T06:44:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394fb63640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39481120e0 con 0x7f3948107090 2024-03-27T06:44:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394d0fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39481066d0 0x7f3948198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394d8ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3948069130 0x7f39481984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394d0fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39481066d0 0x7f3948198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33662/0 (socket says 172.21.15.16:33662) 2024-03-27T06:44:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.913+0000 7f394d0fe640 1 -- 172.21.15.16:0/746117991 learned_addr learned my addr 172.21.15.16:0/746117991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f394e100640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3948107090 0x7f394819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f394d0fe640 1 -- 172.21.15.16:0/746117991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3948107090 msgr2=0x7f394819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f394d0fe640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3948107090 0x7f394819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f394d0fe640 1 -- 172.21.15.16:0/746117991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3948069130 msgr2=0x7f39481984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f394d0fe640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3948069130 0x7f39481984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f394d0fe640 1 -- 172.21.15.16:0/746117991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394819d4b0 con 0x7f39481066d0 2024-03-27T06:44:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f394d0fe640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39481066d0 0x7f3948198a00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3938005ab0 tx=0x7f393802dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f393effd640 1 -- 172.21.15.16:0/746117991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938043020 con 0x7f39481066d0 2024-03-27T06:44:13.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f393effd640 1 -- 172.21.15.16:0/746117991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3938004250 con 0x7f39481066d0 2024-03-27T06:44:13.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f394fb63640 1 -- 172.21.15.16:0/746117991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394819d740 con 0x7f39481066d0 2024-03-27T06:44:13.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f393effd640 1 -- 172.21.15.16:0/746117991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938003ac0 con 0x7f39481066d0 2024-03-27T06:44:13.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.914+0000 7f394fb63640 1 -- 172.21.15.16:0/746117991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39481a5030 con 0x7f39481066d0 2024-03-27T06:44:13.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f393effd640 1 -- 172.21.15.16:0/746117991 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7f3938046020 con 0x7f39481066d0 2024-03-27T06:44:13.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f3952820740 1 Processor -- start 2024-03-27T06:44:13.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f3952820740 1 -- start start 2024-03-27T06:44:13.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f3952820740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae17f80 0x55745ae18360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f3952820740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 0x55745ae25620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f3952820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae25b60 0x55745ae27f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f3952820740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55745ad7c860 con 0x55745ae188a0 2024-03-27T06:44:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f3952820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55745ad248d0 con 0x55745ae25b60 2024-03-27T06:44:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f3952820740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55745ad78810 con 0x55745ae17f80 2024-03-27T06:44:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f394d8ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 0x55745ae25620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f394e100640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae17f80 0x55745ae18360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.917+0000 7f394d8ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 0x55745ae25620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33678/0 (socket says 172.21.15.16:33678) 2024-03-27T06:44:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f394d8ff640 1 -- 172.21.15.16:0/3263494088 learned_addr learned my addr 172.21.15.16:0/3263494088 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f394d0fe640 1 --2- 172.21.15.16:0/3263494088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae25b60 0x55745ae27f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f393effd640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f39340a39b0 0x7f39340a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f394e100640 1 -- 172.21.15.16:0/3263494088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae25b60 msgr2=0x55745ae27f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f394e100640 1 --2- 172.21.15.16:0/3263494088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae25b60 0x55745ae27f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f394e100640 1 -- 172.21.15.16:0/3263494088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 msgr2=0x55745ae25620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f394e100640 1 --2- 172.21.15.16:0/3263494088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 0x55745ae25620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f394e100640 1 -- 172.21.15.16:0/3263494088 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745ad2f120 con 0x55745ae17f80 2024-03-27T06:44:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f394e100640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f39340a39b0 0x7f39340a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f393effd640 1 -- 172.21.15.16:0/746117991 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f39380f2260 con 0x7f39481066d0 2024-03-27T06:44:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f394e100640 1 --2- 172.21.15.16:0/3263494088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae17f80 0x55745ae18360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39440103f0 tx=0x7f39440108b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc7768619507b1f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f3933fff640 1 -- 172.21.15.16:0/3263494088 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944024070 con 0x55745ae17f80 2024-03-27T06:44:13.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f3933fff640 1 -- 172.21.15.16:0/3263494088 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3944003b30 con 0x55745ae17f80 2024-03-27T06:44:13.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.918+0000 7f3933fff640 1 -- 172.21.15.16:0/3263494088 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394401e9c0 con 0x55745ae17f80 2024-03-27T06:44:13.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.919+0000 7f3952820740 1 -- 172.21.15.16:0/3263494088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae17f80 msgr2=0x55745ae18360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.919+0000 7f394e100640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f39340a39b0 0x7f39340a5e70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3944021000 tx=0x7f394401e000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.919+0000 7f3952820740 1 --2- 172.21.15.16:0/3263494088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae17f80 0x55745ae18360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39440103f0 tx=0x7f39440108b0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.919+0000 7f3952820740 1 -- 172.21.15.16:0/3263494088 shutdown_connections 2024-03-27T06:44:13.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.919+0000 7f3952820740 1 --2- 172.21.15.16:0/3263494088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae25b60 0x55745ae27f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.919+0000 7f3952820740 1 --2- 172.21.15.16:0/3263494088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 0x55745ae25620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.919+0000 7f3952820740 1 --2- 172.21.15.16:0/3263494088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae17f80 0x55745ae18360 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.919+0000 7f3952820740 1 -- 172.21.15.16:0/3263494088 >> 172.21.15.16:0/3263494088 conn(0x55745ae12980 msgr2=0x55745ae12d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.919+0000 7f3952820740 1 -- 172.21.15.16:0/3263494088 shutdown_connections 2024-03-27T06:44:13.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.919+0000 7f3952820740 1 -- 172.21.15.16:0/3263494088 wait complete. 2024-03-27T06:44:13.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.919+0000 7f3952820740 1 -- start start 2024-03-27T06:44:13.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f3952820740 1 Processor -- start 2024-03-27T06:44:13.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f3952820740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae17f80 0x55745ae22bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f3952820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae188a0 0x55745ae23100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f3952820740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae25b60 0x55745ae23640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f3952820740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55745ae28920 con 0x55745ae17f80 2024-03-27T06:44:13.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f3952820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55745ad37cd0 con 0x55745ae188a0 2024-03-27T06:44:13.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f3952820740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55745ae28aa0 con 0x55745ae25b60 2024-03-27T06:44:13.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f3952820740 10 client.0 _ll_register_callbacks cb 0x55745ad29ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:13.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394d8ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae17f80 0x55745ae22bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394d8ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae17f80 0x55745ae22bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33680/0 (socket says 172.21.15.16:33680) 2024-03-27T06:44:13.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394d8ff640 1 -- 172.21.15.16:0/1096819310 learned_addr learned my addr 172.21.15.16:0/1096819310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394e100640 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae25b60 0x55745ae23640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394d0fe640 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae188a0 0x55745ae23100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394d8ff640 1 -- 172.21.15.16:0/1096819310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae25b60 msgr2=0x55745ae23640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394d8ff640 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae25b60 0x55745ae23640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394d8ff640 1 -- 172.21.15.16:0/1096819310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae188a0 msgr2=0x55745ae23100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394d8ff640 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae188a0 0x55745ae23100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394d8ff640 1 -- 172.21.15.16:0/1096819310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745ab5a7d0 con 0x55745ae17f80 2024-03-27T06:44:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394d0fe640 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae188a0 0x55745ae23100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.920+0000 7f394e100640 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae25b60 0x55745ae23640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f394d8ff640 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae17f80 0x55745ae22bc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3940012720 tx=0x7f3940012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce5a71fdeea1bbb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f39317fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f39317fa640 1 -- 172.21.15.16:0/1096819310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940004050 con 0x55745ae17f80 2024-03-27T06:44:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f39317fa640 1 -- 172.21.15.16:0/1096819310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f39400041f0 con 0x55745ae17f80 2024-03-27T06:44:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f3952820740 1 -- 172.21.15.16:0/1096819310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55745ad89da0 con 0x55745ae17f80 2024-03-27T06:44:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f39317fa640 1 -- 172.21.15.16:0/1096819310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f39400043f0 con 0x55745ae17f80 2024-03-27T06:44:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f3952820740 10 client.15786 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f3952820740 1 -- 172.21.15.16:0/1096819310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55745aa903f0 con 0x55745ae17f80 2024-03-27T06:44:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f39317fa640 1 -- 172.21.15.16:0/1096819310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55745ad89da0 con 0x55745ae17f80 2024-03-27T06:44:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f39317fa640 1 -- 172.21.15.16:0/1096819310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3940003ca0 con 0x55745ae17f80 2024-03-27T06:44:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f3952820740 10 client.15786 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f3952820740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f3952820740 20 client.15786 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f3952820740 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55745ae35f00 0x55745ae38350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f3952820740 4 client.15786 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f3952820740 1 -- 172.21.15.16:0/1096819310 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55745ad840f0 con 0x55745ae35f00 2024-03-27T06:44:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.921+0000 7f394d0fe640 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55745ae35f00 0x55745ae38350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.922+0000 7f394d0fe640 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55745ae35f00 0x55745ae38350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d00bc88c2ec36d server_cookie=b5af34f62cc23a4f in_seq=0 out_seq=0 2024-03-27T06:44:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.922+0000 7f39317fa640 10 client.15786 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.922+0000 7f39317fa640 1 -- 172.21.15.16:0/1096819310 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55745ad840f0 con 0x55745ae35f00 2024-03-27T06:44:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.922+0000 7f39317fa640 10 client.15786 handle_command_reply: tid=0 2024-03-27T06:44:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.922+0000 7f3952820740 1 client.15786 shutdown 2024-03-27T06:44:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f3952820740 1 -- 172.21.15.16:0/1096819310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae17f80 msgr2=0x55745ae22bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f3952820740 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae17f80 0x55745ae22bc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3940012720 tx=0x7f3940012be0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f3952820740 1 -- 172.21.15.16:0/1096819310 shutdown_connections 2024-03-27T06:44:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f3952820740 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55745ae35f00 0x55745ae38350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f3952820740 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae25b60 0x55745ae23640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f3952820740 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae188a0 0x55745ae23100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f39317fa640 0 client.15786 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f3952820740 1 --2- 172.21.15.16:0/1096819310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae17f80 0x55745ae22bc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f3952820740 1 -- 172.21.15.16:0/1096819310 >> 172.21.15.16:0/1096819310 conn(0x55745ae12980 msgr2=0x55745ae146d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f3952820740 1 -- 172.21.15.16:0/1096819310 shutdown_connections 2024-03-27T06:44:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f3952820740 1 -- 172.21.15.16:0/1096819310 wait complete. 2024-03-27T06:44:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.923+0000 7f3952820740 20 client.15786 trim_cache size 0 max 16384 2024-03-27T06:44:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f3952820740 1 Processor -- start 2024-03-27T06:44:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f3952820740 1 -- start start 2024-03-27T06:44:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f3952820740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae17f80 0x55745ae252f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f3952820740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 0x55745ae21d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f3952820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae25b60 0x55745ae31730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f3952820740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39481ab600 con 0x55745ae188a0 2024-03-27T06:44:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f3952820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f39481ad140 con 0x55745ae25b60 2024-03-27T06:44:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f3952820740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39481abcd0 con 0x55745ae17f80 2024-03-27T06:44:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f394d0fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 0x55745ae21d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f394d0fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 0x55745ae21d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33690/0 (socket says 172.21.15.16:33690) 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f394d0fe640 1 -- 172.21.15.16:0/4012749339 learned_addr learned my addr 172.21.15.16:0/4012749339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.940+0000 7f394d8ff640 1 --2- 172.21.15.16:0/4012749339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae17f80 0x55745ae252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.941+0000 7f394e100640 1 --2- 172.21.15.16:0/4012749339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae25b60 0x55745ae31730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.941+0000 7f394d0fe640 1 -- 172.21.15.16:0/4012749339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae17f80 msgr2=0x55745ae252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.941+0000 7f394d0fe640 1 --2- 172.21.15.16:0/4012749339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae17f80 0x55745ae252f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.941+0000 7f394d0fe640 1 -- 172.21.15.16:0/4012749339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae25b60 msgr2=0x55745ae31730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.941+0000 7f394d0fe640 1 --2- 172.21.15.16:0/4012749339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae25b60 0x55745ae31730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.941+0000 7f394d0fe640 1 -- 172.21.15.16:0/4012749339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745aa500f0 con 0x55745ae188a0 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.941+0000 7f394d0fe640 1 --2- 172.21.15.16:0/4012749339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 0x55745ae21d20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3938006010 tx=0x7f393804a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69cace2b5c58f3f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.941+0000 7f39337fe640 1 -- 172.21.15.16:0/4012749339 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39380bde70 con 0x55745ae188a0 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.941+0000 7f39337fe640 1 -- 172.21.15.16:0/4012749339 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f39380bea30 con 0x55745ae188a0 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.941+0000 7f39337fe640 1 -- 172.21.15.16:0/4012749339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39380bdc90 con 0x55745ae188a0 2024-03-27T06:44:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 -- 172.21.15.16:0/4012749339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 msgr2=0x55745ae21d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 --2- 172.21.15.16:0/4012749339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 0x55745ae21d20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3938006010 tx=0x7f393804a040 comp rx=0 tx=0).stop 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 -- 172.21.15.16:0/4012749339 shutdown_connections 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 --2- 172.21.15.16:0/4012749339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae25b60 0x55745ae31730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 --2- 172.21.15.16:0/4012749339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae188a0 0x55745ae21d20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 --2- 172.21.15.16:0/4012749339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae17f80 0x55745ae252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 -- 172.21.15.16:0/4012749339 >> 172.21.15.16:0/4012749339 conn(0x55745ae12980 msgr2=0x55745ae13cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 -- 172.21.15.16:0/4012749339 shutdown_connections 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 -- 172.21.15.16:0/4012749339 wait complete. 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 -- start start 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 Processor -- start 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae17f80 0x55745ae28f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae188a0 0x55745ae29450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae25b60 0x55745ae9a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39481aca70 con 0x55745ae25b60 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f39481ac3a0 con 0x55745ae17f80 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39481a9600 con 0x55745ae188a0 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.942+0000 7f3952820740 10 client.0 _ll_register_callbacks cb 0x7f3940009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394e100640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae25b60 0x55745ae9a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394d8ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae17f80 0x55745ae28f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394e100640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae25b60 0x55745ae9a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33698/0 (socket says 172.21.15.16:33698) 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394d8ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae17f80 0x55745ae28f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60614/0 (socket says 172.21.15.16:60614) 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394e100640 1 -- 172.21.15.16:0/1871194989 learned_addr learned my addr 172.21.15.16:0/1871194989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394d0fe640 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae188a0 0x55745ae29450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394e100640 1 -- 172.21.15.16:0/1871194989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae188a0 msgr2=0x55745ae29450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394e100640 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae188a0 0x55745ae29450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394e100640 1 -- 172.21.15.16:0/1871194989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae17f80 msgr2=0x55745ae28f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394e100640 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae17f80 0x55745ae28f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394e100640 1 -- 172.21.15.16:0/1871194989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745ae9a830 con 0x55745ae25b60 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f394e100640 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae25b60 0x55745ae9a240 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3944009fd0 tx=0x7f3944005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3fb3a8015a3e73 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f39317fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f39317fa640 1 -- 172.21.15.16:0/1871194989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944024040 con 0x55745ae25b60 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f39317fa640 1 -- 172.21.15.16:0/1871194989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f39440136d0 con 0x55745ae25b60 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f3952820740 1 -- 172.21.15.16:0/1871194989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39380bea30 con 0x55745ae25b60 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f39317fa640 1 -- 172.21.15.16:0/1871194989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3944041050 con 0x55745ae25b60 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f3952820740 10 client.15792 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.943+0000 7f3952820740 1 -- 172.21.15.16:0/1871194989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55745ae9bb50 con 0x55745ae25b60 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f39317fa640 1 -- 172.21.15.16:0/1871194989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f39380bea30 con 0x55745ae25b60 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f39317fa640 1 -- 172.21.15.16:0/1871194989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f394401ece0 con 0x55745ae25b60 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f3952820740 10 client.15792 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f3952820740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f3952820740 20 client.15792 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f3952820740 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55745ae9c070 0x55745ae9e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f3952820740 4 client.15792 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f3952820740 1 -- 172.21.15.16:0/1871194989 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55745ad01540 con 0x55745ae9c070 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f394d8ff640 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55745ae9c070 0x55745ae9e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f39317fa640 10 client.15792 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f394d8ff640 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55745ae9c070 0x55745ae9e490 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781124fd7ae42d2e server_cookie=1175d3e7d77f4ca0 in_seq=0 out_seq=0 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f39317fa640 1 -- 172.21.15.16:0/1871194989 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55745ad01540 con 0x55745ae9c070 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f39317fa640 10 client.15792 handle_command_reply: tid=0 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.944+0000 7f3952820740 1 client.15792 shutdown 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f3952820740 1 -- 172.21.15.16:0/1871194989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae25b60 msgr2=0x55745ae9a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f3952820740 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae25b60 0x55745ae9a240 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3944009fd0 tx=0x7f3944005b70 comp rx=0 tx=0).stop 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f3952820740 1 -- 172.21.15.16:0/1871194989 shutdown_connections 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f3952820740 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55745ae9c070 0x55745ae9e490 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f3952820740 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ae25b60 0x55745ae9a240 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f3952820740 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55745ae188a0 0x55745ae29450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f39317fa640 0 client.15792 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f3952820740 1 --2- 172.21.15.16:0/1871194989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55745ae17f80 0x55745ae28f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f3952820740 1 -- 172.21.15.16:0/1871194989 >> 172.21.15.16:0/1871194989 conn(0x55745ae12980 msgr2=0x55745ae22b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f3952820740 1 -- 172.21.15.16:0/1871194989 shutdown_connections 2024-03-27T06:44:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f3952820740 1 -- 172.21.15.16:0/1871194989 wait complete. 2024-03-27T06:44:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f3952820740 20 client.15792 trim_cache size 0 max 16384 2024-03-27T06:44:13.963 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:44:13.963 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-27T06:44:13.963 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-03-27T06:44:13.963 INFO:teuthology.orchestra.run.smithi016.stdout: "e310668e-5ad9-4e0b-9bf1-59fb212f2c9f": { 2024-03-27T06:44:13.964 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-03-27T06:44:13.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "e310668e-5ad9-4e0b-9bf1-59fb212f2c9f", 2024-03-27T06:44:13.964 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-03-27T06:44:13.964 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:44:13.964 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:44:13.964 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:44:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 -- 172.21.15.16:0/746117991 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f39340a39b0 msgr2=0x7f39340a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f39340a39b0 0x7f39340a5e70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3944021000 tx=0x7f394401e000 comp rx=0 tx=0).stop 2024-03-27T06:44:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 -- 172.21.15.16:0/746117991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39481066d0 msgr2=0x7f3948198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39481066d0 0x7f3948198a00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3938005ab0 tx=0x7f393802dd00 comp rx=0 tx=0).stop 2024-03-27T06:44:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 -- 172.21.15.16:0/746117991 shutdown_connections 2024-03-27T06:44:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f39340a39b0 0x7f39340a5e70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3948107090 0x7f394819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39481066d0 0x7f3948198a00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 --2- 172.21.15.16:0/746117991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3948069130 0x7f39481984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 -- 172.21.15.16:0/746117991 >> 172.21.15.16:0/746117991 conn(0x7f3948077b50 msgr2=0x7f39480fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 -- 172.21.15.16:0/746117991 shutdown_connections 2024-03-27T06:44:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:13.945+0000 7f394fb63640 1 -- 172.21.15.16:0/746117991 wait complete. 2024-03-27T06:44:13.966 INFO:tasks.cephfs.filesystem:scrub status for tag:e310668e-5ad9-4e0b-9bf1-59fb212f2c9f - {'path': '/', 'tag': 'e310668e-5ad9-4e0b-9bf1-59fb212f2c9f', 'options': 'recursive,force'} 2024-03-27T06:44:15.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:14 smithi156 ceph-mon[36261]: scrub queued for path: / 2024-03-27T06:44:15.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:14 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:15.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:14 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:44:15.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:14 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:15.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:14 smithi156 ceph-mon[36261]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-27T06:44:15.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:14 smithi156 ceph-mon[36261]: scrub summary: idle 2024-03-27T06:44:15.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:14 smithi170 ceph-mon[36438]: scrub queued for path: / 2024-03-27T06:44:15.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:14 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:15.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:14 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:44:15.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:14 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:15.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:14 smithi170 ceph-mon[36438]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-27T06:44:15.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:14 smithi170 ceph-mon[36438]: scrub summary: idle 2024-03-27T06:44:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:14 smithi016 ceph-mon[31908]: scrub queued for path: / 2024-03-27T06:44:15.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:14 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:15.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:14 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:44:15.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:14 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:15.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:14 smithi016 ceph-mon[31908]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-27T06:44:15.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:14 smithi016 ceph-mon[31908]: scrub summary: idle 2024-03-27T06:44:17.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:16 smithi156 ceph-mon[36261]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 31 op/s 2024-03-27T06:44:17.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:16 smithi170 ceph-mon[36438]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 31 op/s 2024-03-27T06:44:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:16 smithi016 ceph-mon[31908]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 31 op/s 2024-03-27T06:44:19.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:18 smithi156 ceph-mon[36261]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-27T06:44:19.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:18 smithi170 ceph-mon[36438]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-27T06:44:19.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:18 smithi016 ceph-mon[31908]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-27T06:44:21.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:20 smithi156 ceph-mon[36261]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-03-27T06:44:21.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:20 smithi170 ceph-mon[36438]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-03-27T06:44:21.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:20 smithi016 ceph-mon[31908]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-03-27T06:44:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.058+0000 7fd16664e640 1 -- 172.21.15.16:0/1251880234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd160106f10 msgr2=0x7fd16010ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.058+0000 7fd16664e640 1 --2- 172.21.15.16:0/1251880234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd160106f10 0x7fd16010ea50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd158009ca0 tx=0x7fd15802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.058+0000 7fd16664e640 1 -- 172.21.15.16:0/1251880234 shutdown_connections 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.058+0000 7fd16664e640 1 --2- 172.21.15.16:0/1251880234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd160106f10 0x7fd16010ea50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.058+0000 7fd16664e640 1 --2- 172.21.15.16:0/1251880234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd16006af70 0x7fd1601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.058+0000 7fd16664e640 1 --2- 172.21.15.16:0/1251880234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd16006a5c0 0x7fd16006a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.058+0000 7fd16664e640 1 -- 172.21.15.16:0/1251880234 >> 172.21.15.16:0/1251880234 conn(0x7fd160077820 msgr2=0x7fd160077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.058+0000 7fd16664e640 1 -- 172.21.15.16:0/1251880234 shutdown_connections 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.058+0000 7fd16664e640 1 -- 172.21.15.16:0/1251880234 wait complete. 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd16664e640 1 Processor -- start 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd16664e640 1 -- start start 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd16664e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd16006a5c0 0x7fd1601983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd16664e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd160106f10 0x7fd1601988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd16664e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16019cc70 0x7fd16019d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd16664e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd160112090 con 0x7fd16019cc70 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd16664e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd160111f10 con 0x7fd16006a5c0 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd16664e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd160112210 con 0x7fd160106f10 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd164beb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16019cc70 0x7fd16019d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd164beb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16019cc70 0x7fd16019d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35924/0 (socket says 172.21.15.16:35924) 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd164beb640 1 -- 172.21.15.16:0/10974445 learned_addr learned my addr 172.21.15.16:0/10974445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd15ffff640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd16006a5c0 0x7fd1601983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.059+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd160106f10 0x7fd1601988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd164beb640 1 -- 172.21.15.16:0/10974445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd160106f10 msgr2=0x7fd1601988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd164beb640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd160106f10 0x7fd1601988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd164beb640 1 -- 172.21.15.16:0/10974445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd16006a5c0 msgr2=0x7fd1601983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd164beb640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd16006a5c0 0x7fd1601983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd164beb640 1 -- 172.21.15.16:0/10974445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16019d760 con 0x7fd16019cc70 2024-03-27T06:44:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd15ffff640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd16006a5c0 0x7fd1601983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd164beb640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16019cc70 0x7fd16019d120 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd158037040 tx=0x7fd158032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd15d7fa640 1 -- 172.21.15.16:0/10974445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158039040 con 0x7fd16019cc70 2024-03-27T06:44:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd15d7fa640 1 -- 172.21.15.16:0/10974445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd158044070 con 0x7fd16019cc70 2024-03-27T06:44:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd16664e640 1 -- 172.21.15.16:0/10974445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16019d9f0 con 0x7fd16019cc70 2024-03-27T06:44:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd15d7fa640 1 -- 172.21.15.16:0/10974445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158004330 con 0x7fd16019cc70 2024-03-27T06:44:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.060+0000 7fd16664e640 1 -- 172.21.15.16:0/10974445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1601a5210 con 0x7fd16019cc70 2024-03-27T06:44:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd169237740 1 Processor -- start 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd15d7fa640 1 -- 172.21.15.16:0/10974445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7fd158003970 con 0x7fd16019cc70 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd169237740 1 -- start start 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd169237740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 0x55a4d06814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd169237740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 0x55a4d068a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd169237740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d068cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd169237740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4d0647cd0 con 0x55a4d0681a10 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd169237740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55a4d058c950 con 0x55a4d06810f0 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd169237740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a4d05e4e50 con 0x55a4d068a7e0 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd15ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 0x55a4d068a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd15f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 0x55a4d06814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd15d7fa640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd1480a3a50 0x7fd1480a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd15f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 0x55a4d06814d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:55914/0 (socket says 172.21.15.16:55914) 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.063+0000 7fd15f7fe640 1 -- 172.21.15.16:0/3218812685 learned_addr learned my addr 172.21.15.16:0/3218812685 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/3218812685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d068cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd164beb640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd1480a3a50 0x7fd1480a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd15d7fa640 1 -- 172.21.15.16:0/10974445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd1580f2600 con 0x7fd16019cc70 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd15ffff640 1 -- 172.21.15.16:0/3218812685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 msgr2=0x55a4d068cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd15ffff640 1 --2- 172.21.15.16:0/3218812685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d068cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd15ffff640 1 -- 172.21.15.16:0/3218812685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 msgr2=0x55a4d06814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd15ffff640 1 --2- 172.21.15.16:0/3218812685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 0x55a4d06814d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd15ffff640 1 -- 172.21.15.16:0/3218812685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d02f9210 con 0x55a4d0681a10 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/3218812685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 0x55a4d06814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd15ffff640 1 --2- 172.21.15.16:0/3218812685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 0x55a4d068a2a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd150009fd0 tx=0x7fd1500118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c9b4bb0dcbb125 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd164beb640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd1480a3a50 0x7fd1480a5f10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd1580efcb0 tx=0x7fd158088fc0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd1467fc640 1 -- 172.21.15.16:0/3218812685 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150004030 con 0x55a4d0681a10 2024-03-27T06:44:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd1467fc640 1 -- 172.21.15.16:0/3218812685 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd150005520 con 0x55a4d0681a10 2024-03-27T06:44:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.064+0000 7fd1467fc640 1 -- 172.21.15.16:0/3218812685 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150004d40 con 0x55a4d0681a10 2024-03-27T06:44:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.065+0000 7fd169237740 1 -- 172.21.15.16:0/3218812685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 msgr2=0x55a4d068a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.065+0000 7fd169237740 1 --2- 172.21.15.16:0/3218812685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 0x55a4d068a2a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd150009fd0 tx=0x7fd1500118d0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.065+0000 7fd169237740 1 -- 172.21.15.16:0/3218812685 shutdown_connections 2024-03-27T06:44:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.065+0000 7fd169237740 1 --2- 172.21.15.16:0/3218812685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d068cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.065+0000 7fd169237740 1 --2- 172.21.15.16:0/3218812685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 0x55a4d068a2a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.065+0000 7fd169237740 1 --2- 172.21.15.16:0/3218812685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 0x55a4d06814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.065+0000 7fd169237740 1 -- 172.21.15.16:0/3218812685 >> 172.21.15.16:0/3218812685 conn(0x55a4d067ba10 msgr2=0x55a4d067bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.065+0000 7fd169237740 1 -- 172.21.15.16:0/3218812685 shutdown_connections 2024-03-27T06:44:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.065+0000 7fd169237740 1 -- 172.21.15.16:0/3218812685 wait complete. 2024-03-27T06:44:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd169237740 1 -- start start 2024-03-27T06:44:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd169237740 1 Processor -- start 2024-03-27T06:44:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd169237740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 0x55a4d0687780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd169237740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 0x55a4d0687cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd169237740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d0688200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd169237740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4d05e0e00 con 0x55a4d0681a10 2024-03-27T06:44:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd169237740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55a4d058d520 con 0x55a4d06810f0 2024-03-27T06:44:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd169237740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a4d05a08e0 con 0x55a4d068a7e0 2024-03-27T06:44:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd169237740 10 client.0 _ll_register_callbacks cb 0x55a4d05dba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd15ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 0x55a4d0687cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd15ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 0x55a4d0687cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35942/0 (socket says 172.21.15.16:35942) 2024-03-27T06:44:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd15ffff640 1 -- 172.21.15.16:0/3534001626 learned_addr learned my addr 172.21.15.16:0/3534001626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 0x55a4d0687780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d0688200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.066+0000 7fd15ffff640 1 -- 172.21.15.16:0/3534001626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 msgr2=0x55a4d0688200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd15ffff640 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d0688200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd15ffff640 1 -- 172.21.15.16:0/3534001626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 msgr2=0x55a4d0687780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd15ffff640 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 0x55a4d0687780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd15ffff640 1 -- 172.21.15.16:0/3534001626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d03c5620 con 0x55a4d0681a10 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 0x55a4d0687780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd15ffff640 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 0x55a4d0687cc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd150004880 tx=0x7fd150005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f20f0c2daf375e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd13f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd13f7fe640 1 -- 172.21.15.16:0/3534001626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150028070 con 0x55a4d0681a10 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd13f7fe640 1 -- 172.21.15.16:0/3534001626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd15000f7b0 con 0x55a4d0681a10 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd169237740 1 -- 172.21.15.16:0/3534001626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd150005520 con 0x55a4d0681a10 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd13f7fe640 1 -- 172.21.15.16:0/3534001626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd150034050 con 0x55a4d0681a10 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd169237740 10 client.15804 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd169237740 1 -- 172.21.15.16:0/3534001626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a4d06992d0 con 0x55a4d0681a10 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.067+0000 7fd13f7fe640 1 -- 172.21.15.16:0/3534001626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd150005520 con 0x55a4d0681a10 2024-03-27T06:44:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.068+0000 7fd13f7fe640 1 -- 172.21.15.16:0/3534001626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd150019570 con 0x55a4d0681a10 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.068+0000 7fd169237740 10 client.15804 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.068+0000 7fd169237740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.068+0000 7fd169237740 20 client.15804 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.068+0000 7fd169237740 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55a4d069a6e0 0x55a4d069cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.068+0000 7fd169237740 4 client.15804 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.068+0000 7fd169237740 1 -- 172.21.15.16:0/3534001626 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4d05f2390 con 0x55a4d069a6e0 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.068+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55a4d069a6e0 0x55a4d069cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.068+0000 7fd13f7fe640 10 client.15804 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.068+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55a4d069a6e0 0x55a4d069cad0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b2e6ed17e2290c6 server_cookie=6fa5c30898b974b9 in_seq=0 out_seq=0 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd13f7fe640 1 -- 172.21.15.16:0/3534001626 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a4d05f2390 con 0x55a4d069a6e0 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd13f7fe640 10 client.15804 handle_command_reply: tid=0 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd169237740 1 client.15804 shutdown 2024-03-27T06:44:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd169237740 1 -- 172.21.15.16:0/3534001626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 msgr2=0x55a4d0687cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd169237740 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 0x55a4d0687cc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd150004880 tx=0x7fd150005180 comp rx=0 tx=0).stop 2024-03-27T06:44:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd169237740 1 -- 172.21.15.16:0/3534001626 shutdown_connections 2024-03-27T06:44:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd169237740 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55a4d069a6e0 0x55a4d069cad0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd169237740 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d0688200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd169237740 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d0681a10 0x55a4d0687cc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd13f7fe640 0 client.15804 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd169237740 1 --2- 172.21.15.16:0/3534001626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d06810f0 0x55a4d0687780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.069+0000 7fd169237740 1 -- 172.21.15.16:0/3534001626 >> 172.21.15.16:0/3534001626 conn(0x55a4d067ba10 msgr2=0x55a4d067d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.070+0000 7fd169237740 1 -- 172.21.15.16:0/3534001626 shutdown_connections 2024-03-27T06:44:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.070+0000 7fd169237740 1 -- 172.21.15.16:0/3534001626 wait complete. 2024-03-27T06:44:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.070+0000 7fd169237740 20 client.15804 trim_cache size 0 max 16384 2024-03-27T06:44:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.095+0000 7fd169237740 1 Processor -- start 2024-03-27T06:44:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.095+0000 7fd169237740 1 -- start start 2024-03-27T06:44:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.095+0000 7fd169237740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d06810f0 0x55a4d0695eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.095+0000 7fd169237740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 0x55a4d067d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.095+0000 7fd169237740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d06a2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.095+0000 7fd169237740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601ad760 con 0x55a4d06810f0 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.095+0000 7fd169237740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601af2a0 con 0x55a4d0681a10 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.095+0000 7fd169237740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601ade30 con 0x55a4d068a7e0 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d06810f0 0x55a4d0695eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 0x55a4d067d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d06810f0 0x55a4d0695eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35952/0 (socket says 172.21.15.16:35952) 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15f7fe640 1 -- 172.21.15.16:0/1970147643 learned_addr learned my addr 172.21.15.16:0/1970147643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15ffff640 1 --2- 172.21.15.16:0/1970147643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 0x55a4d067d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:55942/0 (socket says 172.21.15.16:55942) 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/1970147643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d06a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15f7fe640 1 -- 172.21.15.16:0/1970147643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 msgr2=0x55a4d06a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/1970147643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d06a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15f7fe640 1 -- 172.21.15.16:0/1970147643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 msgr2=0x55a4d067d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/1970147643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 0x55a4d067d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15f7fe640 1 -- 172.21.15.16:0/1970147643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d067b840 con 0x55a4d06810f0 2024-03-27T06:44:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15ffff640 1 --2- 172.21.15.16:0/1970147643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 0x55a4d067d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/1970147643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d06810f0 0x55a4d0695eb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55a4d069d200 tx=0x7fd154009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d211df215e5c0c27 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd145ffb640 1 -- 172.21.15.16:0/1970147643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15401a040 con 0x55a4d06810f0 2024-03-27T06:44:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd145ffb640 1 -- 172.21.15.16:0/1970147643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd1540255b0 con 0x55a4d06810f0 2024-03-27T06:44:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.096+0000 7fd145ffb640 1 -- 172.21.15.16:0/1970147643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154003a80 con 0x55a4d06810f0 2024-03-27T06:44:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.097+0000 7fd169237740 1 -- 172.21.15.16:0/1970147643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d06810f0 msgr2=0x55a4d0695eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.097+0000 7fd169237740 1 --2- 172.21.15.16:0/1970147643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d06810f0 0x55a4d0695eb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55a4d069d200 tx=0x7fd154009ac0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.097+0000 7fd169237740 1 -- 172.21.15.16:0/1970147643 shutdown_connections 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.097+0000 7fd169237740 1 --2- 172.21.15.16:0/1970147643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d068a7e0 0x55a4d06a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.097+0000 7fd169237740 1 --2- 172.21.15.16:0/1970147643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 0x55a4d067d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.097+0000 7fd169237740 1 --2- 172.21.15.16:0/1970147643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d06810f0 0x55a4d0695eb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.097+0000 7fd169237740 1 -- 172.21.15.16:0/1970147643 >> 172.21.15.16:0/1970147643 conn(0x55a4d067ba10 msgr2=0x55a4d067ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.097+0000 7fd169237740 1 -- 172.21.15.16:0/1970147643 shutdown_connections 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.097+0000 7fd169237740 1 -- 172.21.15.16:0/1970147643 wait complete. 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.097+0000 7fd169237740 1 -- start start 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd169237740 1 Processor -- start 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd169237740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d06810f0 0x55a4d06fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd169237740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 0x55a4d06fd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd169237740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d068a7e0 0x55a4d0691e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd169237740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601aebd0 con 0x55a4d068a7e0 2024-03-27T06:44:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd169237740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601ae500 con 0x55a4d0681a10 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd169237740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601ab740 con 0x55a4d06810f0 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd169237740 10 client.0 _ll_register_callbacks cb 0x7fd154009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd15f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d068a7e0 0x55a4d0691e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd15f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d06810f0 0x55a4d06fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd15ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 0x55a4d06fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd15f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d06810f0 0x55a4d06fcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:32804/0 (socket says 172.21.15.16:32804) 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd15f7fe640 1 -- 172.21.15.16:0/2634362150 learned_addr learned my addr 172.21.15.16:0/2634362150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd15f7fe640 1 -- 172.21.15.16:0/2634362150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 msgr2=0x55a4d06fd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/2634362150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 0x55a4d06fd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd15f7fe640 1 -- 172.21.15.16:0/2634362150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d068a7e0 msgr2=0x55a4d0691e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/2634362150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d068a7e0 0x55a4d0691e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.098+0000 7fd15f7fe640 1 -- 172.21.15.16:0/2634362150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d06924a0 con 0x55a4d06810f0 2024-03-27T06:44:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd15f7fe640 1 --2- 172.21.15.16:0/2634362150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d06810f0 0x55a4d06fcee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1540128a0 tx=0x7fd154005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1e415f9e330b00d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd13f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd13f7fe640 1 -- 172.21.15.16:0/2634362150 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15400fdc0 con 0x55a4d06810f0 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd169237740 1 -- 172.21.15.16:0/2634362150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1540255b0 con 0x55a4d06810f0 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd13f7fe640 1 -- 172.21.15.16:0/2634362150 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd154012030 con 0x55a4d06810f0 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd13f7fe640 1 -- 172.21.15.16:0/2634362150 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd154012230 con 0x55a4d06810f0 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd169237740 10 client.25648 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd169237740 1 -- 172.21.15.16:0/2634362150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a4d07047d0 con 0x55a4d06810f0 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd13f7fe640 1 -- 172.21.15.16:0/2634362150 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd1540255b0 con 0x55a4d06810f0 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd13f7fe640 1 -- 172.21.15.16:0/2634362150 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd154003c50 con 0x55a4d06810f0 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd169237740 10 client.25648 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd169237740 10 client.25648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd169237740 20 client.25648 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd169237740 1 --2- 172.21.15.16:0/2634362150 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55a4d0704ea0 0x55a4d07072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd169237740 4 client.25648 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:44:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.099+0000 7fd169237740 1 -- 172.21.15.16:0/2634362150 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4d05ec6e0 con 0x55a4d0704ea0 2024-03-27T06:44:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.100+0000 7fd15ffff640 1 --2- 172.21.15.16:0/2634362150 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55a4d0704ea0 0x55a4d07072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.100+0000 7fd13f7fe640 10 client.25648 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.100+0000 7fd15ffff640 1 --2- 172.21.15.16:0/2634362150 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55a4d0704ea0 0x55a4d07072c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b368f2649087f62 server_cookie=e3c74055cd8814fe in_seq=0 out_seq=0 2024-03-27T06:44:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.106+0000 7fd13f7fe640 1 -- 172.21.15.16:0/2634362150 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4d05ec6e0 con 0x55a4d0704ea0 2024-03-27T06:44:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.106+0000 7fd13f7fe640 10 client.25648 handle_command_reply: tid=0 2024-03-27T06:44:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.106+0000 7fd169237740 1 client.25648 shutdown 2024-03-27T06:44:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.106+0000 7fd169237740 1 -- 172.21.15.16:0/2634362150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d06810f0 msgr2=0x55a4d06fcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.106+0000 7fd169237740 1 --2- 172.21.15.16:0/2634362150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d06810f0 0x55a4d06fcee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1540128a0 tx=0x7fd154005f90 comp rx=0 tx=0).stop 2024-03-27T06:44:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd169237740 1 -- 172.21.15.16:0/2634362150 shutdown_connections 2024-03-27T06:44:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd169237740 1 --2- 172.21.15.16:0/2634362150 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55a4d0704ea0 0x55a4d07072c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd169237740 1 --2- 172.21.15.16:0/2634362150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4d068a7e0 0x55a4d0691e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd169237740 1 --2- 172.21.15.16:0/2634362150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4d0681a10 0x55a4d06fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd169237740 1 --2- 172.21.15.16:0/2634362150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4d06810f0 0x55a4d06fcee0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd13f7fe640 0 client.25648 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd169237740 1 -- 172.21.15.16:0/2634362150 >> 172.21.15.16:0/2634362150 conn(0x55a4d067ba10 msgr2=0x55a4d067d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd169237740 1 -- 172.21.15.16:0/2634362150 shutdown_connections 2024-03-27T06:44:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd169237740 1 -- 172.21.15.16:0/2634362150 wait complete. 2024-03-27T06:44:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd169237740 20 client.25648 trim_cache size 0 max 16384 2024-03-27T06:44:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:44:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:44:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:44:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:44:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 -- 172.21.15.16:0/10974445 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd1480a3a50 msgr2=0x7fd1480a5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd1480a3a50 0x7fd1480a5f10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd1580efcb0 tx=0x7fd158088fc0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 -- 172.21.15.16:0/10974445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16019cc70 msgr2=0x7fd16019d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16019cc70 0x7fd16019d120 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd158037040 tx=0x7fd158032570 comp rx=0 tx=0).stop 2024-03-27T06:44:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 -- 172.21.15.16:0/10974445 shutdown_connections 2024-03-27T06:44:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16019cc70 0x7fd16019d120 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd1480a3a50 0x7fd1480a5f10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd160106f10 0x7fd1601988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 --2- 172.21.15.16:0/10974445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd16006a5c0 0x7fd1601983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 -- 172.21.15.16:0/10974445 >> 172.21.15.16:0/10974445 conn(0x7fd160077820 msgr2=0x7fd1600febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 -- 172.21.15.16:0/10974445 shutdown_connections 2024-03-27T06:44:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:22.107+0000 7fd16664e640 1 -- 172.21.15.16:0/10974445 wait complete. 2024-03-27T06:44:23.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:22 smithi156 ceph-mon[36261]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-27T06:44:23.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:22 smithi170 ceph-mon[36438]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-27T06:44:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:22 smithi016 ceph-mon[31908]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-27T06:44:25.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:24 smithi156 ceph-mon[36261]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-03-27T06:44:25.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:24 smithi170 ceph-mon[36438]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-03-27T06:44:25.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:24 smithi016 ceph-mon[31908]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-03-27T06:44:27.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:26 smithi170 ceph-mon[36438]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-03-27T06:44:27.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:26 smithi016 ceph-mon[31908]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-03-27T06:44:27.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:26 smithi156 ceph-mon[36261]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-03-27T06:44:29.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:28 smithi170 ceph-mon[36438]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-27T06:44:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:28 smithi016 ceph-mon[31908]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-27T06:44:29.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:28 smithi156 ceph-mon[36261]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-27T06:44:31.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:30 smithi170 ceph-mon[36438]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-27T06:44:31.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:30 smithi016 ceph-mon[31908]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-27T06:44:31.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:30 smithi156 ceph-mon[36261]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-27T06:44:33.274 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:32 smithi156 ceph-mon[36261]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:44:33.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:32 smithi170 ceph-mon[36438]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:44:33.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:32 smithi016 ceph-mon[31908]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:44:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.197+0000 7f75e87d7640 1 -- 172.21.15.16:0/2511194284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e0069b70 msgr2=0x7f75e010e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.197+0000 7f75e87d7640 1 --2- 172.21.15.16:0/2511194284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e0069b70 0x7f75e010e9e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009ca0 tx=0x7f75d802c180 comp rx=0 tx=0).stop 2024-03-27T06:44:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.197+0000 7f75e87d7640 1 -- 172.21.15.16:0/2511194284 shutdown_connections 2024-03-27T06:44:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.197+0000 7f75e87d7640 1 --2- 172.21.15.16:0/2511194284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e0069b70 0x7f75e010e9e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.197+0000 7f75e87d7640 1 --2- 172.21.15.16:0/2511194284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75e00691d0 0x7f75e0069630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.197+0000 7f75e87d7640 1 --2- 172.21.15.16:0/2511194284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0107c40 0x7f75e0108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.197+0000 7f75e87d7640 1 -- 172.21.15.16:0/2511194284 >> 172.21.15.16:0/2511194284 conn(0x7f75e00778a0 msgr2=0x7f75e0077cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.197+0000 7f75e87d7640 1 -- 172.21.15.16:0/2511194284 shutdown_connections 2024-03-27T06:44:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.197+0000 7f75e87d7640 1 -- 172.21.15.16:0/2511194284 wait complete. 2024-03-27T06:44:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.198+0000 7f75e87d7640 1 Processor -- start 2024-03-27T06:44:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.198+0000 7f75e87d7640 1 -- start start 2024-03-27T06:44:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.198+0000 7f75e87d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75e00691d0 0x7f75e0100d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.198+0000 7f75e87d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e0069b70 0x7f75e01012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.198+0000 7f75e87d7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0107c40 0x7f75e01017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e87d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75e0112020 con 0x7f75e0069b70 2024-03-27T06:44:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e87d7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f75e0111ea0 con 0x7f75e0107c40 2024-03-27T06:44:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e87d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01121a0 con 0x7f75e00691d0 2024-03-27T06:44:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e5d72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e0069b70 0x7f75e01012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e6573640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75e00691d0 0x7f75e0100d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e5d72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e0069b70 0x7f75e01012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37544/0 (socket says 172.21.15.16:37544) 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e5d72640 1 -- 172.21.15.16:0/3317722941 learned_addr learned my addr 172.21.15.16:0/3317722941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e6d74640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0107c40 0x7f75e01017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e6573640 1 -- 172.21.15.16:0/3317722941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0107c40 msgr2=0x7f75e01017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e6573640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0107c40 0x7f75e01017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e6573640 1 -- 172.21.15.16:0/3317722941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e0069b70 msgr2=0x7f75e01012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e6573640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e0069b70 0x7f75e01012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e6573640 1 -- 172.21.15.16:0/3317722941 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e01a68e0 con 0x7f75e00691d0 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e5d72640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e0069b70 0x7f75e01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e6d74640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0107c40 0x7f75e01017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75e6573640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75e00691d0 0x7f75e0100d70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00e9f0 tx=0x7f75dc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.199+0000 7f75d77fe640 1 -- 172.21.15.16:0/3317722941 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00cd80 con 0x7f75e00691d0 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.200+0000 7f75d77fe640 1 -- 172.21.15.16:0/3317722941 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f75dc0035f0 con 0x7f75e00691d0 2024-03-27T06:44:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.200+0000 7f75d77fe640 1 -- 172.21.15.16:0/3317722941 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc010690 con 0x7f75e00691d0 2024-03-27T06:44:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.200+0000 7f75e87d7640 1 -- 172.21.15.16:0/3317722941 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e01a6bd0 con 0x7f75e00691d0 2024-03-27T06:44:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.200+0000 7f75e87d7640 1 -- 172.21.15.16:0/3317722941 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e01a7170 con 0x7f75e00691d0 2024-03-27T06:44:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.202+0000 7f75eb3c0740 1 Processor -- start 2024-03-27T06:44:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75eb3c0740 1 -- start start 2024-03-27T06:44:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75d77fe640 1 -- 172.21.15.16:0/3317722941 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7f75dc0037a0 con 0x7f75e00691d0 2024-03-27T06:44:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206490f0 0x561c206494d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c20649a10 0x561c206522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206527e0 0x561c20654bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c2060fcd0 con 0x561c206490f0 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x561c20554950 con 0x561c206527e0 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561c205ace50 con 0x561c20649a10 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75e6d74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206490f0 0x561c206494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75e6d74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206490f0 0x561c206494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37548/0 (socket says 172.21.15.16:37548) 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75e6d74640 1 -- 172.21.15.16:0/257461532 learned_addr learned my addr 172.21.15.16:0/257461532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75e5d72640 1 --2- 172.21.15.16:0/257461532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206527e0 0x561c20654bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.203+0000 7f75e6573640 1 --2- 172.21.15.16:0/257461532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c20649a10 0x561c206522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75e6d74640 1 -- 172.21.15.16:0/257461532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c20649a10 msgr2=0x561c206522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75d77fe640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75cc0a3a90 0x7f75cc0a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75e6d74640 1 --2- 172.21.15.16:0/257461532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c20649a10 0x561c206522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75e6d74640 1 -- 172.21.15.16:0/257461532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206527e0 msgr2=0x561c20654bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75e6d74640 1 --2- 172.21.15.16:0/257461532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206527e0 0x561c20654bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75e6d74640 1 -- 172.21.15.16:0/257461532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c202c1210 con 0x561c206490f0 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75e5d72640 1 --2- 172.21.15.16:0/257461532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206527e0 0x561c20654bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75d77fe640 1 -- 172.21.15.16:0/3317722941 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f75dc0d0590 con 0x7f75e00691d0 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75e6d74640 1 --2- 172.21.15.16:0/257461532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206490f0 0x561c206494d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009ca0 tx=0x7f75d8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2946e0e53392ded5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75e6d74640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75cc0a3a90 0x7f75cc0a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75d4ff9640 1 -- 172.21.15.16:0/257461532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8043070 con 0x561c206490f0 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.204+0000 7f75d4ff9640 1 -- 172.21.15.16:0/257461532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f75d8032c20 con 0x561c206490f0 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.205+0000 7f75eb3c0740 1 -- 172.21.15.16:0/257461532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206490f0 msgr2=0x561c206494d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.205+0000 7f75e6d74640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75cc0a3a90 0x7f75cc0a5f50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f75d8045000 tx=0x7f75d8048750 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.205+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/257461532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206490f0 0x561c206494d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009ca0 tx=0x7f75d8009900 comp rx=0 tx=0).stop 2024-03-27T06:44:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.205+0000 7f75eb3c0740 1 -- 172.21.15.16:0/257461532 shutdown_connections 2024-03-27T06:44:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.205+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/257461532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206527e0 0x561c20654bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.205+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/257461532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c20649a10 0x561c206522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.205+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/257461532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206490f0 0x561c206494d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.205+0000 7f75eb3c0740 1 -- 172.21.15.16:0/257461532 >> 172.21.15.16:0/257461532 conn(0x561c20643a10 msgr2=0x561c20643df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.205+0000 7f75eb3c0740 1 -- 172.21.15.16:0/257461532 shutdown_connections 2024-03-27T06:44:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.205+0000 7f75eb3c0740 1 -- 172.21.15.16:0/257461532 wait complete. 2024-03-27T06:44:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75eb3c0740 1 -- start start 2024-03-27T06:44:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75eb3c0740 1 Processor -- start 2024-03-27T06:44:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 0x561c2064f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206527e0 0x561c20650190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c205a8e00 con 0x561c20649a10 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x561c20555520 con 0x561c206490f0 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561c205688e0 con 0x561c206527e0 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75eb3c0740 10 client.0 _ll_register_callbacks cb 0x561c205a3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75e5d72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75e5d72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37560/0 (socket says 172.21.15.16:37560) 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75e5d72640 1 -- 172.21.15.16:0/2183599318 learned_addr learned my addr 172.21.15.16:0/2183599318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75e6573640 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 0x561c2064f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.206+0000 7f75e6d74640 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206527e0 0x561c20650190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75e6573640 1 -- 172.21.15.16:0/2183599318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206527e0 msgr2=0x561c20650190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75e6573640 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206527e0 0x561c20650190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75e6573640 1 -- 172.21.15.16:0/2183599318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 msgr2=0x561c2064fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75e6573640 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75e6573640 1 -- 172.21.15.16:0/2183599318 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c2038d620 con 0x561c206490f0 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75e5d72640 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75e6573640 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 0x561c2064f710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0cdad0 tx=0x7f75dc0d3bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed6ebefc4de6170 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75c9ffb640 1 -- 172.21.15.16:0/2183599318 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc09c050 con 0x561c206490f0 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75eb3c0740 1 -- 172.21.15.16:0/2183599318 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75d8032c20 con 0x561c206490f0 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75c9ffb640 1 -- 172.21.15.16:0/2183599318 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f75dc09bb60 con 0x561c206490f0 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75c9ffb640 1 -- 172.21.15.16:0/2183599318 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75dc09be70 con 0x561c206490f0 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75eb3c0740 10 client.25664 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.207+0000 7f75eb3c0740 1 -- 172.21.15.16:0/2183599318 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c20661350 con 0x561c206490f0 2024-03-27T06:44:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.208+0000 7f75c9ffb640 1 -- 172.21.15.16:0/2183599318 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f75d8032c20 con 0x561c206490f0 2024-03-27T06:44:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.208+0000 7f75c9ffb640 1 -- 172.21.15.16:0/2183599318 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f75dc0d2b20 con 0x561c206490f0 2024-03-27T06:44:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.208+0000 7f75eb3c0740 10 client.25664 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.208+0000 7f75eb3c0740 10 client.25664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.208+0000 7f75eb3c0740 20 client.25664 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.208+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x561c20662810 0x561c20664cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.208+0000 7f75eb3c0740 4 client.25664 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.208+0000 7f75eb3c0740 1 -- 172.21.15.16:0/2183599318 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c205ba390 con 0x561c20662810 2024-03-27T06:44:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.208+0000 7f75e5d72640 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x561c20662810 0x561c20664cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.208+0000 7f75e5d72640 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x561c20662810 0x561c20664cb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c272e461033c38 server_cookie=c04bfbd677d8a398 in_seq=0 out_seq=0 2024-03-27T06:44:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.208+0000 7f75c9ffb640 10 client.25664 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75c9ffb640 1 -- 172.21.15.16:0/2183599318 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561c205ba390 con 0x561c20662810 2024-03-27T06:44:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75c9ffb640 10 client.25664 handle_command_reply: tid=0 2024-03-27T06:44:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75eb3c0740 1 client.25664 shutdown 2024-03-27T06:44:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75eb3c0740 1 -- 172.21.15.16:0/2183599318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 msgr2=0x561c2064f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 0x561c2064f710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0cdad0 tx=0x7f75dc0d3bb0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75eb3c0740 1 -- 172.21.15.16:0/2183599318 shutdown_connections 2024-03-27T06:44:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x561c20662810 0x561c20664cb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206527e0 0x561c20650190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75c9ffb640 0 client.25664 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/2183599318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 0x561c2064f710 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75eb3c0740 1 -- 172.21.15.16:0/2183599318 >> 172.21.15.16:0/2183599318 conn(0x561c20643a10 msgr2=0x561c20645760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75eb3c0740 1 -- 172.21.15.16:0/2183599318 shutdown_connections 2024-03-27T06:44:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.209+0000 7f75eb3c0740 1 -- 172.21.15.16:0/2183599318 wait complete. 2024-03-27T06:44:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.210+0000 7f75eb3c0740 20 client.25664 trim_cache size 0 max 16384 2024-03-27T06:44:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.232+0000 7f75eb3c0740 1 Processor -- start 2024-03-27T06:44:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.232+0000 7f75eb3c0740 1 -- start start 2024-03-27T06:44:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206490f0 0x561c20651e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c20649a10 0x561c2064e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206527e0 0x561c2065def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01afa20 con 0x561c206527e0 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01b1560 con 0x561c20649a10 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01b00f0 con 0x561c206490f0 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e5d72640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c20649a10 0x561c2064e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6d74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206527e0 0x561c2065def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6573640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206490f0 0x561c20651e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6d74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206527e0 0x561c2065def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37562/0 (socket says 172.21.15.16:37562) 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e5d72640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c20649a10 0x561c2064e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:36220/0 (socket says 172.21.15.16:36220) 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6573640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206490f0 0x561c20651e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:42468/0 (socket says 172.21.15.16:42468) 2024-03-27T06:44:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6d74640 1 -- 172.21.15.16:0/1526934262 learned_addr learned my addr 172.21.15.16:0/1526934262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6d74640 1 -- 172.21.15.16:0/1526934262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206490f0 msgr2=0x561c20651e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6d74640 1 --2- 172.21.15.16:0/1526934262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206490f0 0x561c20651e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6d74640 1 -- 172.21.15.16:0/1526934262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c20649a10 msgr2=0x561c2064e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6d74640 1 --2- 172.21.15.16:0/1526934262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c20649a10 0x561c2064e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6d74640 1 -- 172.21.15.16:0/1526934262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c20665330 con 0x561c206527e0 2024-03-27T06:44:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e5d72640 1 --2- 172.21.15.16:0/1526934262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c20649a10 0x561c2064e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6573640 1 --2- 172.21.15.16:0/1526934262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206490f0 0x561c20651e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75e6d74640 1 --2- 172.21.15.16:0/1526934262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206527e0 0x561c2065def0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f75d8048c50 tx=0x7f75d804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1368405bf5450c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.233+0000 7f75cbfff640 1 -- 172.21.15.16:0/1526934262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8032720 con 0x561c206527e0 2024-03-27T06:44:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.234+0000 7f75cbfff640 1 -- 172.21.15.16:0/1526934262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f75d8032dc0 con 0x561c206527e0 2024-03-27T06:44:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.234+0000 7f75cbfff640 1 -- 172.21.15.16:0/1526934262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8004300 con 0x561c206527e0 2024-03-27T06:44:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.234+0000 7f75eb3c0740 1 -- 172.21.15.16:0/1526934262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206527e0 msgr2=0x561c2065def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.234+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/1526934262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206527e0 0x561c2065def0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f75d8048c50 tx=0x7f75d804f000 comp rx=0 tx=0).stop 2024-03-27T06:44:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.234+0000 7f75eb3c0740 1 -- 172.21.15.16:0/1526934262 shutdown_connections 2024-03-27T06:44:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.234+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/1526934262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c206527e0 0x561c2065def0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.234+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/1526934262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c20649a10 0x561c2064e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.234+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/1526934262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206490f0 0x561c20651e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.234+0000 7f75eb3c0740 1 -- 172.21.15.16:0/1526934262 >> 172.21.15.16:0/1526934262 conn(0x561c20643a10 msgr2=0x561c206421e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.234+0000 7f75eb3c0740 1 -- 172.21.15.16:0/1526934262 shutdown_connections 2024-03-27T06:44:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.234+0000 7f75eb3c0740 1 -- 172.21.15.16:0/1526934262 wait complete. 2024-03-27T06:44:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75eb3c0740 1 -- start start 2024-03-27T06:44:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75eb3c0740 1 Processor -- start 2024-03-27T06:44:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 0x561c2064c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75eb3c0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206527e0 0x561c2064ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01b0e90 con 0x561c20649a10 2024-03-27T06:44:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01b07c0 con 0x561c206490f0 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75eb3c0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01ada00 con 0x561c206527e0 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75eb3c0740 10 client.0 _ll_register_callbacks cb 0x7f75dc003db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75e5d72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75e5d72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37576/0 (socket says 172.21.15.16:37576) 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75e5d72640 1 -- 172.21.15.16:0/3216790146 learned_addr learned my addr 172.21.15.16:0/3216790146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75e6573640 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 0x561c2064c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.235+0000 7f75e6d74640 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206527e0 0x561c2064ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75e6573640 1 -- 172.21.15.16:0/3216790146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206527e0 msgr2=0x561c2064ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75e6573640 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206527e0 0x561c2064ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75e6573640 1 -- 172.21.15.16:0/3216790146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 msgr2=0x561c2064c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75e6573640 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75e6573640 1 -- 172.21.15.16:0/3216790146 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c206c7d40 con 0x561c206490f0 2024-03-27T06:44:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75e6d74640 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206527e0 0x561c2064ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75e5d72640 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75e6573640 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 0x561c2064c450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f75dc098ee0 tx=0x7f75dc004230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dbbbc82e9bc497d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75c9ffb640 1 -- 172.21.15.16:0/3216790146 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc09c050 con 0x561c206490f0 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75c9ffb640 1 -- 172.21.15.16:0/3216790146 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f75dc0d00b0 con 0x561c206490f0 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75eb3c0740 1 -- 172.21.15.16:0/3216790146 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75d8032dc0 con 0x561c206490f0 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75c9ffb640 1 -- 172.21.15.16:0/3216790146 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75dc003bc0 con 0x561c206490f0 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75eb3c0740 10 client.25670 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75eb3c0740 1 -- 172.21.15.16:0/3216790146 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c206c9030 con 0x561c206490f0 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.236+0000 7f75c9ffb640 1 -- 172.21.15.16:0/3216790146 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f75d8032dc0 con 0x561c206490f0 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.237+0000 7f75c9ffb640 1 -- 172.21.15.16:0/3216790146 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f75dc0b5aa0 con 0x561c206490f0 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.237+0000 7f75eb3c0740 10 client.25670 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.237+0000 7f75eb3c0740 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.237+0000 7f75eb3c0740 20 client.25670 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.237+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x561c206c97b0 0x561c206cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.237+0000 7f75eb3c0740 4 client.25670 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:44:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.237+0000 7f75eb3c0740 1 -- 172.21.15.16:0/3216790146 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c205b46e0 con 0x561c206c97b0 2024-03-27T06:44:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.237+0000 7f75e5d72640 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x561c206c97b0 0x561c206cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.237+0000 7f75c9ffb640 10 client.25670 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.237+0000 7f75e5d72640 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x561c206c97b0 0x561c206cbbd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c29be4d60750638 server_cookie=48798dd426b7cc38 in_seq=0 out_seq=0 2024-03-27T06:44:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.240+0000 7f75c9ffb640 1 -- 172.21.15.16:0/3216790146 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c205b46e0 con 0x561c206c97b0 2024-03-27T06:44:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.240+0000 7f75c9ffb640 10 client.25670 handle_command_reply: tid=0 2024-03-27T06:44:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.240+0000 7f75eb3c0740 1 client.25670 shutdown 2024-03-27T06:44:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.240+0000 7f75eb3c0740 1 -- 172.21.15.16:0/3216790146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 msgr2=0x561c2064c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.240+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 0x561c2064c450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f75dc098ee0 tx=0x7f75dc004230 comp rx=0 tx=0).stop 2024-03-27T06:44:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.240+0000 7f75eb3c0740 1 -- 172.21.15.16:0/3216790146 shutdown_connections 2024-03-27T06:44:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.240+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x561c206c97b0 0x561c206cbbd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.240+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c206527e0 0x561c2064ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.240+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c20649a10 0x561c2064c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.240+0000 7f75eb3c0740 1 --2- 172.21.15.16:0/3216790146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c206490f0 0x561c2064c450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.240+0000 7f75eb3c0740 1 -- 172.21.15.16:0/3216790146 >> 172.21.15.16:0/3216790146 conn(0x561c20643a10 msgr2=0x561c20642a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75c9ffb640 0 client.25670 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75eb3c0740 1 -- 172.21.15.16:0/3216790146 shutdown_connections 2024-03-27T06:44:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75eb3c0740 1 -- 172.21.15.16:0/3216790146 wait complete. 2024-03-27T06:44:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75eb3c0740 20 client.25670 trim_cache size 0 max 16384 2024-03-27T06:44:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:44:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:44:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:44:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:44:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75e87d7640 1 -- 172.21.15.16:0/3317722941 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75cc0a3a90 msgr2=0x7f75cc0a5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75e87d7640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75cc0a3a90 0x7f75cc0a5f50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f75d8045000 tx=0x7f75d8048750 comp rx=0 tx=0).stop 2024-03-27T06:44:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75e87d7640 1 -- 172.21.15.16:0/3317722941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75e00691d0 msgr2=0x7f75e0100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75e87d7640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75e00691d0 0x7f75e0100d70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00e9f0 tx=0x7f75dc00eeb0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75e87d7640 1 -- 172.21.15.16:0/3317722941 shutdown_connections 2024-03-27T06:44:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75e87d7640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75e0107c40 0x7f75e01017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75e87d7640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e0069b70 0x7f75e01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75e87d7640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f75cc0a3a90 0x7f75cc0a5f50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75e87d7640 1 --2- 172.21.15.16:0/3317722941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75e00691d0 0x7f75e0100d70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75e87d7640 1 -- 172.21.15.16:0/3317722941 >> 172.21.15.16:0/3317722941 conn(0x7f75e00778a0 msgr2=0x7f75e00ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.241+0000 7f75e87d7640 1 -- 172.21.15.16:0/3317722941 shutdown_connections 2024-03-27T06:44:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:35.242+0000 7f75e87d7640 1 -- 172.21.15.16:0/3317722941 wait complete. 2024-03-27T06:44:35.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:34 smithi170 ceph-mon[36438]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:44:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:34 smithi016 ceph-mon[31908]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:44:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:34 smithi156 ceph-mon[36261]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-27T06:44:37.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:36 smithi170 ceph-mon[36438]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-03-27T06:44:37.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:36 smithi016 ceph-mon[31908]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-03-27T06:44:37.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:36 smithi156 ceph-mon[36261]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-03-27T06:44:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:38 smithi170 ceph-mon[36438]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:44:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:38 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:44:39.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:38 smithi016 ceph-mon[31908]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:44:39.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:38 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:44:39.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:38 smithi156 ceph-mon[36261]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:44:39.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:38 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:44:40.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:40 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:44:40.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:40 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:44:40.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:40 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:44:40.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:40 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:44:40.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:40 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:44:40.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:40 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:44:40.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:40 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:44:40.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:40 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:44:40.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:40 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:44:40.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:40 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:44:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:40 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:44:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:40 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:44:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:40 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:44:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:40 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:44:40.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:40 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:44:41.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:41 smithi156 ceph-mon[36261]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T06:44:41.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:41 smithi170 ceph-mon[36438]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T06:44:41.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:41 smithi016 ceph-mon[31908]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T06:44:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:43 smithi156 ceph-mon[36261]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T06:44:43.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:43 smithi170 ceph-mon[36438]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T06:44:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:43 smithi016 ceph-mon[31908]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T06:44:43.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T06:44:43.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T06:44:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.082+0000 7f512ca33640 1 -- 172.21.15.16:0/2723340498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5128069130 msgr2=0x7f5128069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.082+0000 7f512ca33640 1 --2- 172.21.15.16:0/2723340498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5128069130 0x7f5128069510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5118009900 tx=0x7f511802bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.082+0000 7f512ca33640 1 -- 172.21.15.16:0/2723340498 shutdown_connections 2024-03-27T06:44:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.082+0000 7f512ca33640 1 --2- 172.21.15.16:0/2723340498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128107090 0x7f512810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.082+0000 7f512ca33640 1 --2- 172.21.15.16:0/2723340498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51281066d0 0x7f5128106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.082+0000 7f512ca33640 1 --2- 172.21.15.16:0/2723340498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5128069130 0x7f5128069510 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.082+0000 7f512ca33640 1 -- 172.21.15.16:0/2723340498 >> 172.21.15.16:0/2723340498 conn(0x7f5128077b50 msgr2=0x7f5128077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.082+0000 7f512ca33640 1 -- 172.21.15.16:0/2723340498 shutdown_connections 2024-03-27T06:44:44.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.082+0000 7f512ca33640 1 -- 172.21.15.16:0/2723340498 wait complete. 2024-03-27T06:44:44.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f512ca33640 1 Processor -- start 2024-03-27T06:44:44.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f512ca33640 1 -- start start 2024-03-27T06:44:44.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f512ca33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128069130 0x7f5128100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f512ca33640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51281066d0 0x7f5128100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f512ca33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5128107090 0x7f5128101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f512ca33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5128111fa0 con 0x7f5128069130 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f512ca33640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5128111e20 con 0x7f51281066d0 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f512ca33640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5128112120 con 0x7f5128107090 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f5126d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5128107090 0x7f5128101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f512659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128069130 0x7f5128100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f5126d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5128107090 0x7f5128101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:41252/0 (socket says 172.21.15.16:41252) 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f5126d9d640 1 -- 172.21.15.16:0/2895639222 learned_addr learned my addr 172.21.15.16:0/2895639222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f5125d9b640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51281066d0 0x7f5128100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f5126d9d640 1 -- 172.21.15.16:0/2895639222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51281066d0 msgr2=0x7f5128100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f5126d9d640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51281066d0 0x7f5128100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f5126d9d640 1 -- 172.21.15.16:0/2895639222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128069130 msgr2=0x7f5128100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f5126d9d640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128069130 0x7f5128100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f5126d9d640 1 -- 172.21.15.16:0/2895639222 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51281a6a00 con 0x7f5128107090 2024-03-27T06:44:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.083+0000 7f512659c640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128069130 0x7f5128100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.084+0000 7f5126d9d640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5128107090 0x7f5128101490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f512000efd0 tx=0x7f512000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.084+0000 7f51177fe640 1 -- 172.21.15.16:0/2895639222 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120003b70 con 0x7f5128107090 2024-03-27T06:44:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.084+0000 7f51177fe640 1 -- 172.21.15.16:0/2895639222 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5120003d10 con 0x7f5128107090 2024-03-27T06:44:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.084+0000 7f51177fe640 1 -- 172.21.15.16:0/2895639222 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120010a90 con 0x7f5128107090 2024-03-27T06:44:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.084+0000 7f512ca33640 1 -- 172.21.15.16:0/2895639222 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51281a6cf0 con 0x7f5128107090 2024-03-27T06:44:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.084+0000 7f512ca33640 1 -- 172.21.15.16:0/2895639222 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51281a71f0 con 0x7f5128107090 2024-03-27T06:44:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.087+0000 7f51177fe640 1 -- 172.21.15.16:0/2895639222 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7f5120003ec0 con 0x7f5128107090 2024-03-27T06:44:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.087+0000 7f512f617740 1 Processor -- start 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f512f617740 1 -- start start 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f512f617740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86060 0x55c515e86440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f512f617740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86a10 0x55c515e97a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f512f617740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e97f60 0x55c515e9a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f512f617740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c515dea860 con 0x55c515e86a10 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f512f617740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55c515d928d0 con 0x55c515e97f60 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f512f617740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c515de6810 con 0x55c515e86060 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f5125d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86060 0x55c515e86440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f5125d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86060 0x55c515e86440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:41254/0 (socket says 172.21.15.16:41254) 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f5125d9b640 1 -- 172.21.15.16:0/3995775826 learned_addr learned my addr 172.21.15.16:0/3995775826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f5125d9b640 1 --2- 172.21.15.16:0/3995775826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e97f60 0x55c515e9a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f512659c640 1 --2- 172.21.15.16:0/3995775826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86a10 0x55c515e97a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.088+0000 7f51177fe640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51100a3a30 0x7f51100a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f51177fe640 1 -- 172.21.15.16:0/2895639222 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f51200d06b0 con 0x7f5128107090 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f5125d9b640 1 -- 172.21.15.16:0/3995775826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e97f60 msgr2=0x55c515e9a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f5125d9b640 1 --2- 172.21.15.16:0/3995775826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e97f60 0x55c515e9a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f5125d9b640 1 -- 172.21.15.16:0/3995775826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86a10 msgr2=0x55c515e97a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f5125d9b640 1 --2- 172.21.15.16:0/3995775826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86a10 0x55c515e97a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f5125d9b640 1 -- 172.21.15.16:0/3995775826 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c515d9d120 con 0x55c515e86060 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f512659c640 1 --2- 172.21.15.16:0/3995775826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86a10 0x55c515e97a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f5126d9d640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51100a3a30 0x7f51100a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f5125d9b640 1 --2- 172.21.15.16:0/3995775826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86060 0x55c515e86440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f511c011810 tx=0x7f511c0097d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3062fb02230f4982 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f5114ff9640 1 -- 172.21.15.16:0/3995775826 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c004030 con 0x55c515e86060 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f5114ff9640 1 -- 172.21.15.16:0/3995775826 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f511c011e50 con 0x55c515e86060 2024-03-27T06:44:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.089+0000 7f5114ff9640 1 -- 172.21.15.16:0/3995775826 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c003830 con 0x55c515e86060 2024-03-27T06:44:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.090+0000 7f512f617740 1 -- 172.21.15.16:0/3995775826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86060 msgr2=0x55c515e86440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.090+0000 7f512f617740 1 --2- 172.21.15.16:0/3995775826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86060 0x55c515e86440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f511c011810 tx=0x7f511c0097d0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.090+0000 7f5126d9d640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51100a3a30 0x7f51100a5ef0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f51200cdc50 tx=0x7f5120009f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.090+0000 7f512f617740 1 -- 172.21.15.16:0/3995775826 shutdown_connections 2024-03-27T06:44:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.090+0000 7f512f617740 1 --2- 172.21.15.16:0/3995775826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e97f60 0x55c515e9a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.090+0000 7f512f617740 1 --2- 172.21.15.16:0/3995775826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86a10 0x55c515e97a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.090+0000 7f512f617740 1 --2- 172.21.15.16:0/3995775826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86060 0x55c515e86440 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.090+0000 7f512f617740 1 -- 172.21.15.16:0/3995775826 >> 172.21.15.16:0/3995775826 conn(0x55c515e80980 msgr2=0x55c515e80d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.090+0000 7f512f617740 1 -- 172.21.15.16:0/3995775826 shutdown_connections 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.090+0000 7f512f617740 1 -- 172.21.15.16:0/3995775826 wait complete. 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.090+0000 7f512f617740 1 -- start start 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f512f617740 1 Processor -- start 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f512f617740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86060 0x55c515e94fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f512f617740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86a10 0x55c515e954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f512f617740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e97f60 0x55c515e95a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f512f617740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c515e9ad20 con 0x55c515e86060 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f512f617740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55c515da5cd0 con 0x55c515e97f60 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f512f617740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c515e9aea0 con 0x55c515e86a10 2024-03-27T06:44:44.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f512f617740 10 client.0 _ll_register_callbacks cb 0x55c515d97ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f5125d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86060 0x55c515e94fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f5125d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86060 0x55c515e94fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34546/0 (socket says 172.21.15.16:34546) 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f5125d9b640 1 -- 172.21.15.16:0/2865107983 learned_addr learned my addr 172.21.15.16:0/2865107983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f512659c640 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86a10 0x55c515e954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f5125d9b640 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e97f60 0x55c515e95a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f5125d9b640 1 -- 172.21.15.16:0/2865107983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86a10 msgr2=0x55c515e954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f5125d9b640 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86a10 0x55c515e954e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f5125d9b640 1 -- 172.21.15.16:0/2865107983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e97f60 msgr2=0x55c515e95a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f5125d9b640 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e97f60 0x55c515e95a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.091+0000 7f5125d9b640 1 -- 172.21.15.16:0/2865107983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c515bc87d0 con 0x55c515e86060 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.092+0000 7f5125d9b640 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86060 0x55c515e94fa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f511c004490 tx=0x7f511c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb4ad2e00dc6ea5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.092+0000 7f510dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.092+0000 7f510dffb640 1 -- 172.21.15.16:0/2865107983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c023020 con 0x55c515e86060 2024-03-27T06:44:44.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.092+0000 7f510dffb640 1 -- 172.21.15.16:0/2865107983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f511c019040 con 0x55c515e86060 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.092+0000 7f512f617740 1 -- 172.21.15.16:0/2865107983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f511c011e50 con 0x55c515e86060 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.092+0000 7f510dffb640 1 -- 172.21.15.16:0/2865107983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f511c034050 con 0x55c515e86060 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.092+0000 7f512f617740 10 client.15840 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.092+0000 7f512f617740 1 -- 172.21.15.16:0/2865107983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c515afe3f0 con 0x55c515e86060 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.092+0000 7f510dffb640 1 -- 172.21.15.16:0/2865107983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f511c011e50 con 0x55c515e86060 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.092+0000 7f510dffb640 1 -- 172.21.15.16:0/2865107983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f511c02d5e0 con 0x55c515e86060 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.092+0000 7f512f617740 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.093+0000 7f512f617740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.093+0000 7f512f617740 20 client.15840 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.093+0000 7f512f617740 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55c515ea81d0 0x55c515eaa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.093+0000 7f512f617740 4 client.15840 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.093+0000 7f512f617740 1 -- 172.21.15.16:0/2865107983 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c515df7da0 con 0x55c515ea81d0 2024-03-27T06:44:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.093+0000 7f512659c640 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55c515ea81d0 0x55c515eaa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.093+0000 7f510dffb640 10 client.15840 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.093+0000 7f512659c640 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55c515ea81d0 0x55c515eaa5c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3037c5b07ad2e1 server_cookie=dfd338cfc9245234 in_seq=0 out_seq=0 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f510dffb640 1 -- 172.21.15.16:0/2865107983 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c515df7da0 con 0x55c515ea81d0 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f510dffb640 10 client.15840 handle_command_reply: tid=0 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 1 client.15840 shutdown 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 1 -- 172.21.15.16:0/2865107983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86060 msgr2=0x55c515e94fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86060 0x55c515e94fa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f511c004490 tx=0x7f511c009ca0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 1 -- 172.21.15.16:0/2865107983 shutdown_connections 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55c515ea81d0 0x55c515eaa5c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e97f60 0x55c515e95a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e86a10 0x55c515e954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f510dffb640 0 client.15840 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 1 --2- 172.21.15.16:0/2865107983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515e86060 0x55c515e94fa0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 1 -- 172.21.15.16:0/2865107983 >> 172.21.15.16:0/2865107983 conn(0x55c515e80980 msgr2=0x55c515e826d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 1 -- 172.21.15.16:0/2865107983 shutdown_connections 2024-03-27T06:44:44.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 1 -- 172.21.15.16:0/2865107983 wait complete. 2024-03-27T06:44:44.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.094+0000 7f512f617740 20 client.15840 trim_cache size 0 max 16384 2024-03-27T06:44:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512f617740 1 Processor -- start 2024-03-27T06:44:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512f617740 1 -- start start 2024-03-27T06:44:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512f617740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515e976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512f617740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 0x55c515ea3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512f617740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515eafd30 0x55c515e7f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512f617740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51281ab780 con 0x55c515eafd30 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512f617740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f51281ad2c0 con 0x55c515e86a10 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512f617740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f51281abe50 con 0x55c515e97f60 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 0x55c515ea3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f5125d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515e976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f5125d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515e976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60242/0 (socket says 172.21.15.16:60242) 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f5125d9b640 1 -- 172.21.15.16:0/2377406433 learned_addr learned my addr 172.21.15.16:0/2377406433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f5125d9b640 1 --2- 172.21.15.16:0/2377406433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515eafd30 0x55c515e7f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512659c640 1 -- 172.21.15.16:0/2377406433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 msgr2=0x55c515e976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512659c640 1 --2- 172.21.15.16:0/2377406433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515e976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512659c640 1 -- 172.21.15.16:0/2377406433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515eafd30 msgr2=0x55c515e7f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.129+0000 7f512659c640 1 --2- 172.21.15.16:0/2377406433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515eafd30 0x55c515e7f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.130+0000 7f512659c640 1 -- 172.21.15.16:0/2377406433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c515abe0f0 con 0x55c515e97f60 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.130+0000 7f5125d9b640 1 --2- 172.21.15.16:0/2377406433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515e976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.130+0000 7f512659c640 1 --2- 172.21.15.16:0/2377406433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 0x55c515ea3b30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5118009230 tx=0x7f5118009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38a5d3d73b574033 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.130+0000 7f510ffff640 1 -- 172.21.15.16:0/2377406433 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118046a30 con 0x55c515e97f60 2024-03-27T06:44:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.130+0000 7f510ffff640 1 -- 172.21.15.16:0/2377406433 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f511804b070 con 0x55c515e97f60 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.130+0000 7f510ffff640 1 -- 172.21.15.16:0/2377406433 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118004380 con 0x55c515e97f60 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.131+0000 7f512f617740 1 -- 172.21.15.16:0/2377406433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 msgr2=0x55c515ea3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.131+0000 7f512f617740 1 --2- 172.21.15.16:0/2377406433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 0x55c515ea3b30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5118009230 tx=0x7f5118009260 comp rx=0 tx=0).stop 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.131+0000 7f512f617740 1 -- 172.21.15.16:0/2377406433 shutdown_connections 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.131+0000 7f512f617740 1 --2- 172.21.15.16:0/2377406433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515eafd30 0x55c515e7f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.131+0000 7f512f617740 1 --2- 172.21.15.16:0/2377406433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 0x55c515ea3b30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.131+0000 7f512f617740 1 --2- 172.21.15.16:0/2377406433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515e976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.131+0000 7f512f617740 1 -- 172.21.15.16:0/2377406433 >> 172.21.15.16:0/2377406433 conn(0x55c515e80980 msgr2=0x55c515e93fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.131+0000 7f512f617740 1 -- 172.21.15.16:0/2377406433 shutdown_connections 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.131+0000 7f512f617740 1 -- 172.21.15.16:0/2377406433 wait complete. 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.131+0000 7f512f617740 1 -- start start 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.131+0000 7f512f617740 1 Processor -- start 2024-03-27T06:44:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f512f617740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515ea0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f512f617740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 0x55c515e9b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f512f617740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515eafd30 0x55c515e9bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f512f617740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51281acbf0 con 0x55c515eafd30 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f512f617740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f51281ac520 con 0x55c515e86a10 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f512f617740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f51281a9760 con 0x55c515e97f60 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f5125d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515ea0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f512f617740 10 client.0 _ll_register_callbacks cb 0x7f511c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f5125d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515ea0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60258/0 (socket says 172.21.15.16:60258) 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f5125d9b640 1 -- 172.21.15.16:0/562378171 learned_addr learned my addr 172.21.15.16:0/562378171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f512659c640 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 0x55c515e9b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f5125d9b640 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515eafd30 0x55c515e9bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f5125d9b640 1 -- 172.21.15.16:0/562378171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 msgr2=0x55c515e9b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f5125d9b640 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 0x55c515e9b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f5125d9b640 1 -- 172.21.15.16:0/562378171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515eafd30 msgr2=0x55c515e9bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f5125d9b640 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515eafd30 0x55c515e9bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f5125d9b640 1 -- 172.21.15.16:0/562378171 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c515e9c460 con 0x55c515e86a10 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.132+0000 7f512659c640 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 0x55c515e9b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f5125d9b640 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515ea0350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f511c00f790 tx=0x7f511c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26bc05556946f7c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f510dffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f510dffb640 1 -- 172.21.15.16:0/562378171 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c040060 con 0x55c515e86a10 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f510dffb640 1 -- 172.21.15.16:0/562378171 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f511c011030 con 0x55c515e86a10 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f512f617740 1 -- 172.21.15.16:0/562378171 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f511804b070 con 0x55c515e86a10 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f510dffb640 1 -- 172.21.15.16:0/562378171 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f511c023030 con 0x55c515e86a10 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f512f617740 10 client.25688 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f512f617740 1 -- 172.21.15.16:0/562378171 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c515f0e2c0 con 0x55c515e86a10 2024-03-27T06:44:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f510dffb640 1 -- 172.21.15.16:0/562378171 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f511804b070 con 0x55c515e86a10 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f510dffb640 1 -- 172.21.15.16:0/562378171 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f511c01f900 con 0x55c515e86a10 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f512f617740 10 client.25688 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f512f617740 10 client.25688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f512f617740 20 client.25688 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f512f617740 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55c515f0e920 0x55c515f10d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.133+0000 7f512f617740 4 client.25688 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512f617740 1 -- 172.21.15.16:0/562378171 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c515df20f0 con 0x55c515f0e920 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512659c640 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55c515f0e920 0x55c515f10d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512659c640 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55c515f0e920 0x55c515f10d40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b1a14bcce27237f server_cookie=d9ce16836b1afad in_seq=0 out_seq=0 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f510dffb640 10 client.25688 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f510dffb640 1 -- 172.21.15.16:0/562378171 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c515df20f0 con 0x55c515f0e920 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f510dffb640 10 client.25688 handle_command_reply: tid=0 2024-03-27T06:44:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512f617740 1 client.25688 shutdown 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512f617740 1 -- 172.21.15.16:0/562378171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 msgr2=0x55c515ea0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512f617740 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515ea0350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f511c00f790 tx=0x7f511c009ca0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512f617740 1 -- 172.21.15.16:0/562378171 shutdown_connections 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512f617740 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55c515f0e920 0x55c515f10d40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512f617740 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c515eafd30 0x55c515e9bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f510dffb640 0 client.25688 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512f617740 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c515e97f60 0x55c515e9b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512f617740 1 --2- 172.21.15.16:0/562378171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c515e86a10 0x55c515ea0350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.134+0000 7f512f617740 1 -- 172.21.15.16:0/562378171 >> 172.21.15.16:0/562378171 conn(0x55c515e80980 msgr2=0x55c515e93740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512f617740 1 -- 172.21.15.16:0/562378171 shutdown_connections 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512f617740 1 -- 172.21.15.16:0/562378171 wait complete. 2024-03-27T06:44:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512f617740 20 client.25688 trim_cache size 0 max 16384 2024-03-27T06:44:44.136 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:44:44.136 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-03-27T06:44:44.136 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-03-27T06:44:44.136 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:44:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 -- 172.21.15.16:0/2895639222 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51100a3a30 msgr2=0x7f51100a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51100a3a30 0x7f51100a5ef0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f51200cdc50 tx=0x7f5120009f90 comp rx=0 tx=0).stop 2024-03-27T06:44:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 -- 172.21.15.16:0/2895639222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5128107090 msgr2=0x7f5128101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5128107090 0x7f5128101490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f512000efd0 tx=0x7f512000c6a0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 -- 172.21.15.16:0/2895639222 shutdown_connections 2024-03-27T06:44:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51100a3a30 0x7f51100a5ef0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5128107090 0x7f5128101490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51281066d0 0x7f5128100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 --2- 172.21.15.16:0/2895639222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128069130 0x7f5128100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 -- 172.21.15.16:0/2895639222 >> 172.21.15.16:0/2895639222 conn(0x7f5128077b50 msgr2=0x7f51280fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 -- 172.21.15.16:0/2895639222 shutdown_connections 2024-03-27T06:44:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.135+0000 7f512ca33640 1 -- 172.21.15.16:0/2895639222 wait complete. 2024-03-27T06:44:44.143 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-27T06:44:44.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:44:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.255+0000 7ff359807640 1 -- 172.21.15.16:0/4152695518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3541036e0 msgr2=0x7ff35410bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.255+0000 7ff359807640 1 --2- 172.21.15.16:0/4152695518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3541036e0 0x7ff35410bbc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff348009860 tx=0x7ff34802bb70 comp rx=0 tx=0).stop 2024-03-27T06:44:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.255+0000 7ff359807640 1 -- 172.21.15.16:0/4152695518 shutdown_connections 2024-03-27T06:44:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.255+0000 7ff359807640 1 --2- 172.21.15.16:0/4152695518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff35410c190 0x7ff35410e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.255+0000 7ff359807640 1 --2- 172.21.15.16:0/4152695518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3541036e0 0x7ff35410bbc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.255+0000 7ff359807640 1 --2- 172.21.15.16:0/4152695518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff354102dc0 0x7ff3541031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.255+0000 7ff359807640 1 -- 172.21.15.16:0/4152695518 >> 172.21.15.16:0/4152695518 conn(0x7ff354077b50 msgr2=0x7ff354077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.255+0000 7ff359807640 1 -- 172.21.15.16:0/4152695518 shutdown_connections 2024-03-27T06:44:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.255+0000 7ff359807640 1 -- 172.21.15.16:0/4152695518 wait complete. 2024-03-27T06:44:44.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.255+0000 7ff359807640 1 Processor -- start 2024-03-27T06:44:44.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff359807640 1 -- start start 2024-03-27T06:44:44.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff359807640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff354102dc0 0x7ff354198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff359807640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3541036e0 0x7ff354198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff359807640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff35410c190 0x7ff35419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff359807640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3541120b0 con 0x7ff354102dc0 2024-03-27T06:44:44.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff352ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff354102dc0 0x7ff354198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff359807640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ff354111f30 con 0x7ff3541036e0 2024-03-27T06:44:44.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff352ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff354102dc0 0x7ff354198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34576/0 (socket says 172.21.15.16:34576) 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff352ffd640 1 -- 172.21.15.16:0/1121374244 learned_addr learned my addr 172.21.15.16:0/1121374244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff3527fc640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3541036e0 0x7ff354198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff359807640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff354112230 con 0x7ff35410c190 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.256+0000 7ff3537fe640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff35410c190 0x7ff35419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.257+0000 7ff3527fc640 1 -- 172.21.15.16:0/1121374244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff35410c190 msgr2=0x7ff35419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.257+0000 7ff3527fc640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff35410c190 0x7ff35419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.257+0000 7ff3527fc640 1 -- 172.21.15.16:0/1121374244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff354102dc0 msgr2=0x7ff354198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.257+0000 7ff3527fc640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff354102dc0 0x7ff354198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.257+0000 7ff3527fc640 1 -- 172.21.15.16:0/1121374244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35419d530 con 0x7ff3541036e0 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.257+0000 7ff352ffd640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff354102dc0 0x7ff354198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.257+0000 7ff3537fe640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff35410c190 0x7ff35419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.257+0000 7ff3527fc640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3541036e0 0x7ff354198ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff34802c0b0 tx=0x7ff3480045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.257+0000 7ff358805640 1 -- 172.21.15.16:0/1121374244 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348041020 con 0x7ff3541036e0 2024-03-27T06:44:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.257+0000 7ff358805640 1 -- 172.21.15.16:0/1121374244 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff34802ca10 con 0x7ff3541036e0 2024-03-27T06:44:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.257+0000 7ff358805640 1 -- 172.21.15.16:0/1121374244 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348039040 con 0x7ff3541036e0 2024-03-27T06:44:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.258+0000 7ff359807640 1 -- 172.21.15.16:0/1121374244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35419d7c0 con 0x7ff3541036e0 2024-03-27T06:44:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.258+0000 7ff359807640 1 -- 172.21.15.16:0/1121374244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3541a50b0 con 0x7ff3541036e0 2024-03-27T06:44:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.259+0000 7ff359807640 1 -- 172.21.15.16:0/1121374244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3540694f0 con 0x7ff3541036e0 2024-03-27T06:44:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.261+0000 7ff358805640 1 -- 172.21.15.16:0/1121374244 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7ff348044020 con 0x7ff3541036e0 2024-03-27T06:44:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.262+0000 7ff358805640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff33c0a39b0 0x7ff33c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.262+0000 7ff352ffd640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff33c0a39b0 0x7ff33c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.262+0000 7ff358805640 1 -- 172.21.15.16:0/1121374244 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff348072080 con 0x7ff3541036e0 2024-03-27T06:44:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.262+0000 7ff352ffd640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff33c0a39b0 0x7ff33c0a5e70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff344011550 tx=0x7ff344005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.266+0000 7ff358805640 1 -- 172.21.15.16:0/1121374244 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff34804b070 con 0x7ff3541036e0 2024-03-27T06:44:44.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.503+0000 7ff359807640 1 -- 172.21.15.16:0/1121374244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff354107280 con 0x7ff3541036e0 2024-03-27T06:44:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.504+0000 7ff358805640 1 -- 172.21.15.16:0/1121374244 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11932 (secure 0 0 0) 0x7ff3480bdf80 con 0x7ff3541036e0 2024-03-27T06:44:44.499 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:44:44.499 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:43:32.824696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/4279306199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4279306199},{"type":"v1","addr":"172.21.15.170:6835","nonce":4279306199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24467,24527,24500,24472,24493]},"id":1}]} 2024-03-27T06:44:44.500 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-03-27T06:44:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 -- 172.21.15.16:0/1121374244 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff33c0a39b0 msgr2=0x7ff33c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff33c0a39b0 0x7ff33c0a5e70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff344011550 tx=0x7ff344005f90 comp rx=0 tx=0).stop 2024-03-27T06:44:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 -- 172.21.15.16:0/1121374244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3541036e0 msgr2=0x7ff354198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3541036e0 0x7ff354198ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff34802c0b0 tx=0x7ff3480045b0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 -- 172.21.15.16:0/1121374244 shutdown_connections 2024-03-27T06:44:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff33c0a39b0 0x7ff33c0a5e70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff35410c190 0x7ff35419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3541036e0 0x7ff354198ad0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 --2- 172.21.15.16:0/1121374244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff354102dc0 0x7ff354198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 -- 172.21.15.16:0/1121374244 >> 172.21.15.16:0/1121374244 conn(0x7ff354077b50 msgr2=0x7ff3540fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 -- 172.21.15.16:0/1121374244 shutdown_connections 2024-03-27T06:44:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.507+0000 7ff359807640 1 -- 172.21.15.16:0/1121374244 wait complete. 2024-03-27T06:44:44.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-27T06:44:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.630+0000 7fbe6e89d640 1 -- 172.21.15.16:0/242738340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe681066d0 msgr2=0x7fbe68106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.630+0000 7fbe6e89d640 1 --2- 172.21.15.16:0/242738340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe681066d0 0x7fbe68106b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009900 tx=0x7fbe5c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.630+0000 7fbe6e89d640 1 -- 172.21.15.16:0/242738340 shutdown_connections 2024-03-27T06:44:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.630+0000 7fbe6e89d640 1 --2- 172.21.15.16:0/242738340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe68107090 0x7fbe6810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.630+0000 7fbe6e89d640 1 --2- 172.21.15.16:0/242738340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe681066d0 0x7fbe68106b50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.630+0000 7fbe6e89d640 1 --2- 172.21.15.16:0/242738340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe68069130 0x7fbe68069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.630+0000 7fbe6e89d640 1 -- 172.21.15.16:0/242738340 >> 172.21.15.16:0/242738340 conn(0x7fbe68077b50 msgr2=0x7fbe68077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.630+0000 7fbe6e89d640 1 -- 172.21.15.16:0/242738340 shutdown_connections 2024-03-27T06:44:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.630+0000 7fbe6e89d640 1 -- 172.21.15.16:0/242738340 wait complete. 2024-03-27T06:44:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.631+0000 7fbe6e89d640 1 Processor -- start 2024-03-27T06:44:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.631+0000 7fbe6e89d640 1 -- start start 2024-03-27T06:44:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.631+0000 7fbe6e89d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe68069130 0x7fbe68198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.631+0000 7fbe6e89d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe681066d0 0x7fbe68198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe6e89d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe68107090 0x7fbe6819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe681066d0 0x7fbe68198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe681066d0 0x7fbe68198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34596/0 (socket says 172.21.15.16:34596) 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe677fe640 1 -- 172.21.15.16:0/2300289113 learned_addr learned my addr 172.21.15.16:0/2300289113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe6e89d640 1 -- 172.21.15.16:0/2300289113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe68111f60 con 0x7fbe681066d0 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe67fff640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe68069130 0x7fbe68198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe6e89d640 1 -- 172.21.15.16:0/2300289113 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbe68111de0 con 0x7fbe68069130 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe6e89d640 1 -- 172.21.15.16:0/2300289113 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbe681120e0 con 0x7fbe68107090 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe68107090 0x7fbe6819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe677fe640 1 -- 172.21.15.16:0/2300289113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe68107090 msgr2=0x7fbe6819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe677fe640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe68107090 0x7fbe6819ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe677fe640 1 -- 172.21.15.16:0/2300289113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe68069130 msgr2=0x7fbe68198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe677fe640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe68069130 0x7fbe68198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe677fe640 1 -- 172.21.15.16:0/2300289113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6819d580 con 0x7fbe681066d0 2024-03-27T06:44:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe68107090 0x7fbe6819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe67fff640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe68069130 0x7fbe68198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.632+0000 7fbe677fe640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe681066d0 0x7fbe68198ad0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c005940 tx=0x7fbe5c02dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.633+0000 7fbe657fa640 1 -- 172.21.15.16:0/2300289113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c043020 con 0x7fbe681066d0 2024-03-27T06:44:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.633+0000 7fbe6e89d640 1 -- 172.21.15.16:0/2300289113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6819d810 con 0x7fbe681066d0 2024-03-27T06:44:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.633+0000 7fbe657fa640 1 -- 172.21.15.16:0/2300289113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe5c004250 con 0x7fbe681066d0 2024-03-27T06:44:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.633+0000 7fbe657fa640 1 -- 172.21.15.16:0/2300289113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c003a10 con 0x7fbe681066d0 2024-03-27T06:44:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.633+0000 7fbe6e89d640 1 -- 172.21.15.16:0/2300289113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe681a5100 con 0x7fbe681066d0 2024-03-27T06:44:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.636+0000 7fbe657fa640 1 -- 172.21.15.16:0/2300289113 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7fbe5c046020 con 0x7fbe681066d0 2024-03-27T06:44:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.636+0000 7fbe71565740 1 Processor -- start 2024-03-27T06:44:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.636+0000 7fbe71565740 1 -- start start 2024-03-27T06:44:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.636+0000 7fbe71565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a21b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.636+0000 7fbe71565740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a2a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.636+0000 7fbe71565740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a2d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.636+0000 7fbe657fa640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe500a3930 0x7fbe500a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.636+0000 7fbe71565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d901985710 con 0x55d901a217b0 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.636+0000 7fbe71565740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55d90192daf0 con 0x55d901a2acb0 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.636+0000 7fbe71565740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d9019816c0 con 0x55d901a220d0 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe6ce3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a21b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe6ce3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a21b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34612/0 (socket says 172.21.15.16:34612) 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe6ce3a640 1 -- 172.21.15.16:0/2372204031 learned_addr learned my addr 172.21.15.16:0/2372204031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe67fff640 1 --2- 172.21.15.16:0/2372204031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a2a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe500a3930 0x7fbe500a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe6ce3a640 1 -- 172.21.15.16:0/2372204031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 msgr2=0x55d901a2a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/2372204031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a2a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe6ce3a640 1 -- 172.21.15.16:0/2372204031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 msgr2=0x55d901a2d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe677fe640 1 --2- 172.21.15.16:0/2372204031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a2d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe657fa640 1 -- 172.21.15.16:0/2300289113 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbe5c0f2370 con 0x7fbe681066d0 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/2372204031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a2d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe6ce3a640 1 -- 172.21.15.16:0/2372204031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9016993f0 con 0x55d901a217b0 2024-03-27T06:44:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe677fe640 1 --2- 172.21.15.16:0/2372204031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a2d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:44:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.637+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/2372204031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a21b90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbe60005f40 tx=0x7fbe60012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fccb84993b6d2e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.638+0000 7fbe4e7fc640 1 -- 172.21.15.16:0/2372204031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60023030 con 0x55d901a217b0 2024-03-27T06:44:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.638+0000 7fbe4e7fc640 1 -- 172.21.15.16:0/2372204031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe60012470 con 0x55d901a217b0 2024-03-27T06:44:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.638+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe500a3930 0x7fbe500a5df0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe60023000 tx=0x7fbe60022000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.638+0000 7fbe4e7fc640 1 -- 172.21.15.16:0/2372204031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6001f970 con 0x55d901a217b0 2024-03-27T06:44:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.639+0000 7fbe71565740 1 -- 172.21.15.16:0/2372204031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 msgr2=0x55d901a21b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.639+0000 7fbe71565740 1 --2- 172.21.15.16:0/2372204031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a21b90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbe60005f40 tx=0x7fbe60012f80 comp rx=0 tx=0).stop 2024-03-27T06:44:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.639+0000 7fbe71565740 1 -- 172.21.15.16:0/2372204031 shutdown_connections 2024-03-27T06:44:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.639+0000 7fbe71565740 1 --2- 172.21.15.16:0/2372204031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a2d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.639+0000 7fbe71565740 1 --2- 172.21.15.16:0/2372204031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a2a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.639+0000 7fbe71565740 1 --2- 172.21.15.16:0/2372204031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a21b90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.639+0000 7fbe71565740 1 -- 172.21.15.16:0/2372204031 >> 172.21.15.16:0/2372204031 conn(0x55d901a1bee0 msgr2=0x55d901a1c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.639+0000 7fbe71565740 1 -- 172.21.15.16:0/2372204031 shutdown_connections 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.639+0000 7fbe71565740 1 -- 172.21.15.16:0/2372204031 wait complete. 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.639+0000 7fbe71565740 1 -- start start 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.639+0000 7fbe71565740 1 Processor -- start 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe71565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe71565740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a220d0 0x55d901a28260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe71565740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a2acb0 0x55d901a287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe71565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d901a31eb0 con 0x55d901a217b0 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe71565740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55d901940eb0 con 0x55d901a220d0 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe71565740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d901a32030 con 0x55d901a2acb0 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe677fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a220d0 0x55d901a28260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34614/0 (socket says 172.21.15.16:34614) 2024-03-27T06:44:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe677fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a220d0 0x55d901a28260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60316/0 (socket says 172.21.15.16:60316) 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe67fff640 1 -- 172.21.15.16:0/4057231458 learned_addr learned my addr 172.21.15.16:0/4057231458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a2acb0 0x55d901a287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe71565740 10 client.0 _ll_register_callbacks cb 0x55d90197d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe6ce3a640 1 -- 172.21.15.16:0/4057231458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a220d0 msgr2=0x55d901a28260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a220d0 0x55d901a28260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe6ce3a640 1 -- 172.21.15.16:0/4057231458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 msgr2=0x55d901a27d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe6ce3a640 1 -- 172.21.15.16:0/4057231458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9017637d0 con 0x55d901a2acb0 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe677fe640 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a220d0 0x55d901a28260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.640+0000 7fbe67fff640 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a2acb0 0x55d901a287a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000e7e0 tx=0x7fbe6001fad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96b21d2e346da110 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe477fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe477fe640 1 -- 172.21.15.16:0/4057231458 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600045f0 con 0x55d901a2acb0 2024-03-27T06:44:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe477fe640 1 -- 172.21.15.16:0/4057231458 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbe60004790 con 0x55d901a2acb0 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe71565740 1 -- 172.21.15.16:0/4057231458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe60012470 con 0x55d901a2acb0 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe477fe640 1 -- 172.21.15.16:0/4057231458 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbe60004a00 con 0x55d901a2acb0 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe71565740 10 client.25702 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe71565740 1 -- 172.21.15.16:0/4057231458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d901a3e0e0 con 0x55d901a2acb0 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe477fe640 1 -- 172.21.15.16:0/4057231458 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe60004ba0 con 0x55d901a2acb0 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe477fe640 1 -- 172.21.15.16:0/4057231458 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbe6001e070 con 0x55d901a2acb0 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe71565740 10 client.25702 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe71565740 10 client.25702 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe71565740 20 client.25702 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.641+0000 7fbe71565740 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55d901a3f430 0x55d901a41820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.642+0000 7fbe71565740 4 client.25702 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.642+0000 7fbe71565740 1 -- 172.21.15.16:0/4057231458 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbe60012470 con 0x55d901a3f430 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.642+0000 7fbe67fff640 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55d901a3f430 0x55d901a41820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.642+0000 7fbe477fe640 10 client.25702 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:44:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.642+0000 7fbe67fff640 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55d901a3f430 0x55d901a41820 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a1fb31d15d1d0cc1 server_cookie=cac648e894a36d31 in_seq=0 out_seq=0 2024-03-27T06:44:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe477fe640 1 -- 172.21.15.16:0/4057231458 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fbe60012470 con 0x55d901a3f430 2024-03-27T06:44:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe477fe640 10 client.25702 handle_command_reply: tid=0 2024-03-27T06:44:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe71565740 1 client.25702 shutdown 2024-03-27T06:44:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe71565740 1 -- 172.21.15.16:0/4057231458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a2acb0 msgr2=0x55d901a287a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe71565740 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a2acb0 0x55d901a287a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000e7e0 tx=0x7fbe6001fad0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe71565740 1 -- 172.21.15.16:0/4057231458 shutdown_connections 2024-03-27T06:44:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe71565740 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55d901a3f430 0x55d901a41820 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe477fe640 0 client.25702 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:44:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe71565740 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a2acb0 0x55d901a287a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe71565740 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a220d0 0x55d901a28260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe71565740 1 --2- 172.21.15.16:0/4057231458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.643+0000 7fbe71565740 1 -- 172.21.15.16:0/4057231458 >> 172.21.15.16:0/4057231458 conn(0x55d901a1bee0 msgr2=0x55d901a1dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.644+0000 7fbe71565740 1 -- 172.21.15.16:0/4057231458 shutdown_connections 2024-03-27T06:44:44.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.644+0000 7fbe71565740 1 -- 172.21.15.16:0/4057231458 wait complete. 2024-03-27T06:44:44.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.644+0000 7fbe71565740 20 client.25702 trim_cache size 0 max 16384 2024-03-27T06:44:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe71565740 1 Processor -- start 2024-03-27T06:44:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe71565740 1 -- start start 2024-03-27T06:44:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe71565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe71565740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a3ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe71565740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34616/0 (socket says 172.21.15.16:34616) 2024-03-27T06:44:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe71565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe681ab700 con 0x55d901a217b0 2024-03-27T06:44:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe67fff640 1 -- 172.21.15.16:0/1664511266 learned_addr learned my addr 172.21.15.16:0/1664511266 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe71565740 1 -- 172.21.15.16:0/1664511266 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbe681ad240 con 0x55d901a2acb0 2024-03-27T06:44:44.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe71565740 1 -- 172.21.15.16:0/1664511266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbe681abdd0 con 0x55d901a220d0 2024-03-27T06:44:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe677fe640 1 --2- 172.21.15.16:0/1664511266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a3ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.670+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/1664511266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.671+0000 7fbe677fe640 1 -- 172.21.15.16:0/1664511266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 msgr2=0x55d901a470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.671+0000 7fbe677fe640 1 --2- 172.21.15.16:0/1664511266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.671+0000 7fbe677fe640 1 -- 172.21.15.16:0/1664511266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 msgr2=0x55d901a27600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.671+0000 7fbe677fe640 1 --2- 172.21.15.16:0/1664511266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.671+0000 7fbe677fe640 1 -- 172.21.15.16:0/1664511266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d901a1e1d0 con 0x55d901a220d0 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.671+0000 7fbe67fff640 1 --2- 172.21.15.16:0/1664511266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.671+0000 7fbe677fe640 1 --2- 172.21.15.16:0/1664511266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a3ac80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0ef9c0 tx=0x7fbe5c0ba040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f878399ed840d5f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.671+0000 7fbe4dffb640 1 -- 172.21.15.16:0/1664511266 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0c3f40 con 0x55d901a220d0 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.671+0000 7fbe4dffb640 1 -- 172.21.15.16:0/1664511266 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe5c0beb00 con 0x55d901a220d0 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.671+0000 7fbe4dffb640 1 -- 172.21.15.16:0/1664511266 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0c3940 con 0x55d901a220d0 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.672+0000 7fbe71565740 1 -- 172.21.15.16:0/1664511266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 msgr2=0x55d901a3ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.672+0000 7fbe71565740 1 --2- 172.21.15.16:0/1664511266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a3ac80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0ef9c0 tx=0x7fbe5c0ba040 comp rx=0 tx=0).stop 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.672+0000 7fbe71565740 1 -- 172.21.15.16:0/1664511266 shutdown_connections 2024-03-27T06:44:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.672+0000 7fbe71565740 1 --2- 172.21.15.16:0/1664511266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.672+0000 7fbe71565740 1 --2- 172.21.15.16:0/1664511266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a3ac80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.672+0000 7fbe71565740 1 --2- 172.21.15.16:0/1664511266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a27600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.672+0000 7fbe71565740 1 -- 172.21.15.16:0/1664511266 >> 172.21.15.16:0/1664511266 conn(0x55d901a1bee0 msgr2=0x55d901a29cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.672+0000 7fbe71565740 1 -- 172.21.15.16:0/1664511266 shutdown_connections 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.672+0000 7fbe71565740 1 -- 172.21.15.16:0/1664511266 wait complete. 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.672+0000 7fbe71565740 1 -- start start 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe71565740 1 Processor -- start 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe71565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a2d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe71565740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a2dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe71565740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a30970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe71565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe681acb70 con 0x55d901a217b0 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe71565740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbe681ac4a0 con 0x55d901a2acb0 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe71565740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbe681a96e0 con 0x55d901a220d0 2024-03-27T06:44:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe71565740 10 client.0 _ll_register_callbacks cb 0x7fbe600128f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a2d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a2dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a2d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34624/0 (socket says 172.21.15.16:34624) 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe67fff640 1 -- 172.21.15.16:0/2429951007 learned_addr learned my addr 172.21.15.16:0/2429951007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a30970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe67fff640 1 -- 172.21.15.16:0/2429951007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 msgr2=0x55d901a2dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe67fff640 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a2dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe67fff640 1 -- 172.21.15.16:0/2429951007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 msgr2=0x55d901a30970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe67fff640 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a30970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe67fff640 1 -- 172.21.15.16:0/2429951007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d901a30f60 con 0x55d901a217b0 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe6ce3a640 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a30970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.673+0000 7fbe677fe640 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a2dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe67fff640 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a2d860 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbe58012de0 tx=0x7fbe5801e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de28cfcc1a252afc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe477fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe477fe640 1 -- 172.21.15.16:0/2429951007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58004030 con 0x55d901a217b0 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe477fe640 1 -- 172.21.15.16:0/2429951007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fbe580041d0 con 0x55d901a217b0 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe71565740 1 -- 172.21.15.16:0/2429951007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5c0beb00 con 0x55d901a217b0 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe477fe640 1 -- 172.21.15.16:0/2429951007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbe58030050 con 0x55d901a217b0 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe71565740 10 client.15870 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe71565740 1 -- 172.21.15.16:0/2429951007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d901a36d90 con 0x55d901a217b0 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe477fe640 1 -- 172.21.15.16:0/2429951007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe5c0beb00 con 0x55d901a217b0 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe477fe640 1 -- 172.21.15.16:0/2429951007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbe58003790 con 0x55d901a217b0 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe71565740 10 client.15870 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.674+0000 7fbe71565740 10 client.15870 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.675+0000 7fbe71565740 20 client.15870 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.675+0000 7fbe71565740 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55d901a3f430 0x55d901a37540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.675+0000 7fbe71565740 4 client.15870 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.675+0000 7fbe71565740 1 -- 172.21.15.16:0/2429951007 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d901992c50 con 0x55d901a3f430 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.675+0000 7fbe677fe640 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55d901a3f430 0x55d901a37540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.675+0000 7fbe477fe640 10 client.15870 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.675+0000 7fbe677fe640 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55d901a3f430 0x55d901a37540 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd23daea0565d4d7 server_cookie=2e50687608b346ea in_seq=0 out_seq=0 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe477fe640 1 -- 172.21.15.16:0/2429951007 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d901992c50 con 0x55d901a3f430 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe477fe640 10 client.15870 handle_command_reply: tid=0 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 1 client.15870 shutdown 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 1 -- 172.21.15.16:0/2429951007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 msgr2=0x55d901a2d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a2d860 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbe58012de0 tx=0x7fbe5801e7c0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 1 -- 172.21.15.16:0/2429951007 shutdown_connections 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55d901a3f430 0x55d901a37540 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d901a2acb0 0x55d901a30970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe477fe640 0 client.15870 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:44:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d901a220d0 0x55d901a2dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 1 --2- 172.21.15.16:0/2429951007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d901a217b0 0x55d901a2d860 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 1 -- 172.21.15.16:0/2429951007 >> 172.21.15.16:0/2429951007 conn(0x55d901a1bee0 msgr2=0x55d901a27ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 1 -- 172.21.15.16:0/2429951007 shutdown_connections 2024-03-27T06:44:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 1 -- 172.21.15.16:0/2429951007 wait complete. 2024-03-27T06:44:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.676+0000 7fbe71565740 20 client.15870 trim_cache size 0 max 16384 2024-03-27T06:44:44.677 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:44:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 -- 172.21.15.16:0/2300289113 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe500a3930 msgr2=0x7fbe500a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe500a3930 0x7fbe500a5df0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe60023000 tx=0x7fbe60022000 comp rx=0 tx=0).stop 2024-03-27T06:44:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 -- 172.21.15.16:0/2300289113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe681066d0 msgr2=0x7fbe68198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe681066d0 0x7fbe68198ad0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c005940 tx=0x7fbe5c02dc90 comp rx=0 tx=0).stop 2024-03-27T06:44:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 -- 172.21.15.16:0/2300289113 shutdown_connections 2024-03-27T06:44:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe500a3930 0x7fbe500a5df0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe68107090 0x7fbe6819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe681066d0 0x7fbe68198ad0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 --2- 172.21.15.16:0/2300289113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe68069130 0x7fbe68198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 -- 172.21.15.16:0/2300289113 >> 172.21.15.16:0/2300289113 conn(0x7fbe68077b50 msgr2=0x7fbe680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 -- 172.21.15.16:0/2300289113 shutdown_connections 2024-03-27T06:44:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.677+0000 7fbe6e89d640 1 -- 172.21.15.16:0/2300289113 wait complete. 2024-03-27T06:44:44.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-27T06:44:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.800+0000 7fda1dd13640 1 -- 172.21.15.16:0/2545786719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda181066d0 msgr2=0x7fda18106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.800+0000 7fda1dd13640 1 --2- 172.21.15.16:0/2545786719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda181066d0 0x7fda18106b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009900 tx=0x7fda0c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.801+0000 7fda1dd13640 1 -- 172.21.15.16:0/2545786719 shutdown_connections 2024-03-27T06:44:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.801+0000 7fda1dd13640 1 --2- 172.21.15.16:0/2545786719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda18107090 0x7fda1810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.801+0000 7fda1dd13640 1 --2- 172.21.15.16:0/2545786719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda181066d0 0x7fda18106b50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.801+0000 7fda1dd13640 1 --2- 172.21.15.16:0/2545786719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda18069130 0x7fda18069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.801+0000 7fda1dd13640 1 -- 172.21.15.16:0/2545786719 >> 172.21.15.16:0/2545786719 conn(0x7fda18077b50 msgr2=0x7fda18077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.801+0000 7fda1dd13640 1 -- 172.21.15.16:0/2545786719 shutdown_connections 2024-03-27T06:44:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.801+0000 7fda1dd13640 1 -- 172.21.15.16:0/2545786719 wait complete. 2024-03-27T06:44:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.801+0000 7fda1dd13640 1 Processor -- start 2024-03-27T06:44:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda1dd13640 1 -- start start 2024-03-27T06:44:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda1dd13640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda18069130 0x7fda181984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda1dd13640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda181066d0 0x7fda181989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda1dd13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda18107090 0x7fda1819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda1dd13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda18111f60 con 0x7fda18107090 2024-03-27T06:44:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda1dd13640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fda18111de0 con 0x7fda18069130 2024-03-27T06:44:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda1dd13640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fda181120e0 con 0x7fda181066d0 2024-03-27T06:44:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda16ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda181066d0 0x7fda181989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda177fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda18069130 0x7fda181984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda16ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda181066d0 0x7fda181989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:41374/0 (socket says 172.21.15.16:41374) 2024-03-27T06:44:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda16ffd640 1 -- 172.21.15.16:0/3525738781 learned_addr learned my addr 172.21.15.16:0/3525738781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda177fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda18069130 0x7fda181984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60354/0 (socket says 172.21.15.16:60354) 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.802+0000 7fda17fff640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda18107090 0x7fda1819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.803+0000 7fda16ffd640 1 -- 172.21.15.16:0/3525738781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda18069130 msgr2=0x7fda181984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.803+0000 7fda16ffd640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda18069130 0x7fda181984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.803+0000 7fda16ffd640 1 -- 172.21.15.16:0/3525738781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda18107090 msgr2=0x7fda1819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.803+0000 7fda16ffd640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda18107090 0x7fda1819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.803+0000 7fda16ffd640 1 -- 172.21.15.16:0/3525738781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1819d490 con 0x7fda181066d0 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.803+0000 7fda16ffd640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda181066d0 0x7fda181989e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fda0c005940 tx=0x7fda0c02dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.803+0000 7fda14ff9640 1 -- 172.21.15.16:0/3525738781 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c043020 con 0x7fda181066d0 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.803+0000 7fda14ff9640 1 -- 172.21.15.16:0/3525738781 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda0c004250 con 0x7fda181066d0 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.803+0000 7fda1dd13640 1 -- 172.21.15.16:0/3525738781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1819d720 con 0x7fda181066d0 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.803+0000 7fda14ff9640 1 -- 172.21.15.16:0/3525738781 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c003a10 con 0x7fda181066d0 2024-03-27T06:44:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.803+0000 7fda1dd13640 1 -- 172.21.15.16:0/3525738781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda181a5010 con 0x7fda181066d0 2024-03-27T06:44:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda209dc740 1 Processor -- start 2024-03-27T06:44:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda209dc740 1 -- start start 2024-03-27T06:44:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda209dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947b9a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda209dc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9ab40 0x55e947ba7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda209dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947baa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda209dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e947afe710 con 0x55e947b9a190 2024-03-27T06:44:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda209dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e947aa6af0 con 0x55e947ba80c0 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda209dc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e947afa6c0 con 0x55e947b9ab40 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda14ff9640 1 -- 172.21.15.16:0/3525738781 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7fda0c046020 con 0x7fda181066d0 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947b9a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947b9a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34664/0 (socket says 172.21.15.16:34664) 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.806+0000 7fda17fff640 1 -- 172.21.15.16:0/1165032572 learned_addr learned my addr 172.21.15.16:0/1165032572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.807+0000 7fda16ffd640 1 --2- 172.21.15.16:0/1165032572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947baa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.807+0000 7fda177fe640 1 --2- 172.21.15.16:0/1165032572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9ab40 0x55e947ba7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.807+0000 7fda17fff640 1 -- 172.21.15.16:0/1165032572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9ab40 msgr2=0x55e947ba7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.807+0000 7fda17fff640 1 --2- 172.21.15.16:0/1165032572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9ab40 0x55e947ba7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.807+0000 7fda17fff640 1 -- 172.21.15.16:0/1165032572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 msgr2=0x55e947baa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.807+0000 7fda17fff640 1 --2- 172.21.15.16:0/1165032572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947baa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.807+0000 7fda17fff640 1 -- 172.21.15.16:0/1165032572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9478123f0 con 0x55e947b9a190 2024-03-27T06:44:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.807+0000 7fda16ffd640 1 --2- 172.21.15.16:0/1165032572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947baa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.807+0000 7fda177fe640 1 --2- 172.21.15.16:0/1165032572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9ab40 0x55e947ba7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.807+0000 7fda17fff640 1 --2- 172.21.15.16:0/1165032572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947b9a570 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fda10009f50 tx=0x7fda100118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781b85f14d810fc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.807+0000 7fda14ff9640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fda000a39b0 0x7fda000a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.808+0000 7fda14ff9640 1 -- 172.21.15.16:0/3525738781 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fda0c0f22c0 con 0x7fda181066d0 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.808+0000 7fd9fdffb640 1 -- 172.21.15.16:0/1165032572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10004b20 con 0x55e947b9a190 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.808+0000 7fd9fdffb640 1 -- 172.21.15.16:0/1165032572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda10004cc0 con 0x55e947b9a190 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.808+0000 7fd9fdffb640 1 -- 172.21.15.16:0/1165032572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10005280 con 0x55e947b9a190 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.808+0000 7fda177fe640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fda000a39b0 0x7fda000a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.809+0000 7fda209dc740 1 -- 172.21.15.16:0/1165032572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 msgr2=0x55e947b9a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.809+0000 7fda209dc740 1 --2- 172.21.15.16:0/1165032572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947b9a570 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fda10009f50 tx=0x7fda100118d0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.809+0000 7fda177fe640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fda000a39b0 0x7fda000a5e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda08009e00 tx=0x7fda08005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.809+0000 7fda209dc740 1 -- 172.21.15.16:0/1165032572 shutdown_connections 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.809+0000 7fda209dc740 1 --2- 172.21.15.16:0/1165032572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947baa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.809+0000 7fda209dc740 1 --2- 172.21.15.16:0/1165032572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9ab40 0x55e947ba7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.809+0000 7fda209dc740 1 --2- 172.21.15.16:0/1165032572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947b9a570 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.809+0000 7fda209dc740 1 -- 172.21.15.16:0/1165032572 >> 172.21.15.16:0/1165032572 conn(0x55e947b94ee0 msgr2=0x55e947b952c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.809+0000 7fda209dc740 1 -- 172.21.15.16:0/1165032572 shutdown_connections 2024-03-27T06:44:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.809+0000 7fda209dc740 1 -- 172.21.15.16:0/1165032572 wait complete. 2024-03-27T06:44:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.809+0000 7fda209dc740 1 -- start start 2024-03-27T06:44:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.810+0000 7fda209dc740 1 Processor -- start 2024-03-27T06:44:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.810+0000 7fda209dc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9a190 0x55e947ba1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.810+0000 7fda209dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 0x55e947ba1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.810+0000 7fda209dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947ba5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.810+0000 7fda16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 0x55e947ba1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.810+0000 7fda16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 0x55e947ba1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34666/0 (socket says 172.21.15.16:34666) 2024-03-27T06:44:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda177fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9a190 0x55e947ba1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda16ffd640 1 -- 172.21.15.16:0/238153918 learned_addr learned my addr 172.21.15.16:0/238153918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda209dc740 1 -- 172.21.15.16:0/238153918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e947b9b950 con 0x55e947b9ab40 2024-03-27T06:44:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda209dc740 1 -- 172.21.15.16:0/238153918 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e947ab9eb0 con 0x55e947ba80c0 2024-03-27T06:44:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda209dc740 1 -- 172.21.15.16:0/238153918 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e947b9bad0 con 0x55e947b9a190 2024-03-27T06:44:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda17fff640 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947ba5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda177fe640 1 -- 172.21.15.16:0/238153918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 msgr2=0x55e947ba5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda177fe640 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947ba5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda177fe640 1 -- 172.21.15.16:0/238153918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 msgr2=0x55e947ba1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda177fe640 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 0x55e947ba1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda177fe640 1 -- 172.21.15.16:0/238153918 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9478dc7d0 con 0x55e947b9a190 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.811+0000 7fda16ffd640 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 0x55e947ba1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.812+0000 7fda177fe640 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9a190 0x55e947ba1320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda08012cb0 tx=0x7fda08004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b554541d116dee92 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.812+0000 7fda209dc740 10 client.0 _ll_register_callbacks cb 0x55e947af66a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.812+0000 7fd9f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.812+0000 7fda209dc740 1 -- 172.21.15.16:0/238153918 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda10004cc0 con 0x55e947b9a190 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.812+0000 7fd9f6ffd640 1 -- 172.21.15.16:0/238153918 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0802e060 con 0x55e947b9a190 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.812+0000 7fd9f6ffd640 1 -- 172.21.15.16:0/238153918 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fda08018030 con 0x55e947b9a190 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.812+0000 7fd9f6ffd640 1 -- 172.21.15.16:0/238153918 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fda08009e30 con 0x55e947b9a190 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.812+0000 7fda209dc740 10 client.25720 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.812+0000 7fda209dc740 1 -- 172.21.15.16:0/238153918 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55e947bc3bb0 con 0x55e947b9a190 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.812+0000 7fd9f6ffd640 1 -- 172.21.15.16:0/238153918 <== mon.1 v2:172.21.15.170:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fda0800f3c0 con 0x55e947b9a190 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.813+0000 7fda209dc740 10 client.25720 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.813+0000 7fda209dc740 10 client.25720 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.813+0000 7fda209dc740 20 client.25720 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.813+0000 7fda209dc740 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e947bc4f00 0x55e947bc72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.813+0000 7fda209dc740 4 client.25720 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.813+0000 7fda209dc740 1 -- 172.21.15.16:0/238153918 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e947b0bc50 con 0x55e947bc4f00 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.813+0000 7fda16ffd640 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e947bc4f00 0x55e947bc72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.813+0000 7fda16ffd640 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e947bc4f00 0x55e947bc72f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6022984f7df21eb0 server_cookie=d37f203bb0412d21 in_seq=0 out_seq=0 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.813+0000 7fd9f6ffd640 10 client.25720 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.814+0000 7fd9f6ffd640 1 -- 172.21.15.16:0/238153918 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e947b0bc50 con 0x55e947bc4f00 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.814+0000 7fd9f6ffd640 10 client.25720 handle_command_reply: tid=0 2024-03-27T06:44:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.814+0000 7fda209dc740 1 client.25720 shutdown 2024-03-27T06:44:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fda209dc740 1 -- 172.21.15.16:0/238153918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9a190 msgr2=0x55e947ba1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fda209dc740 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9a190 0x55e947ba1320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda08012cb0 tx=0x7fda08004730 comp rx=0 tx=0).stop 2024-03-27T06:44:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fda209dc740 1 -- 172.21.15.16:0/238153918 shutdown_connections 2024-03-27T06:44:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fda209dc740 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e947bc4f00 0x55e947bc72f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fda209dc740 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947ba5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fda209dc740 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 0x55e947ba1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fd9f6ffd640 0 client.25720 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:44:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fda209dc740 1 --2- 172.21.15.16:0/238153918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9a190 0x55e947ba1320 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fda209dc740 1 -- 172.21.15.16:0/238153918 >> 172.21.15.16:0/238153918 conn(0x55e947b94ee0 msgr2=0x55e947b96c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fda209dc740 1 -- 172.21.15.16:0/238153918 shutdown_connections 2024-03-27T06:44:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fda209dc740 1 -- 172.21.15.16:0/238153918 wait complete. 2024-03-27T06:44:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.815+0000 7fda209dc740 20 client.25720 trim_cache size 0 max 16384 2024-03-27T06:44:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.841+0000 7fda209dc740 1 Processor -- start 2024-03-27T06:44:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda209dc740 1 -- start start 2024-03-27T06:44:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda209dc740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947b9a190 0x55e947b96d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda209dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 0x55e947ba0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda209dc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947ba80c0 0x55e947bccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda209dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda181ab6b0 con 0x55e947b9ab40 2024-03-27T06:44:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda209dc740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fda181ad1f0 con 0x55e947b9a190 2024-03-27T06:44:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda209dc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fda181abd80 con 0x55e947ba80c0 2024-03-27T06:44:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda177fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947b9a190 0x55e947b96d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 0x55e947ba0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda177fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947b9a190 0x55e947b96d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60388/0 (socket says 172.21.15.16:60388) 2024-03-27T06:44:44.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 0x55e947ba0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34672/0 (socket says 172.21.15.16:34672) 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda177fe640 1 -- 172.21.15.16:0/352988150 learned_addr learned my addr 172.21.15.16:0/352988150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda17fff640 1 --2- 172.21.15.16:0/352988150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947ba80c0 0x55e947bccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda177fe640 1 -- 172.21.15.16:0/352988150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947ba80c0 msgr2=0x55e947bccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda177fe640 1 --2- 172.21.15.16:0/352988150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947ba80c0 0x55e947bccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda177fe640 1 -- 172.21.15.16:0/352988150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 msgr2=0x55e947ba0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda177fe640 1 --2- 172.21.15.16:0/352988150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 0x55e947ba0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.842+0000 7fda177fe640 1 -- 172.21.15.16:0/352988150 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e947b9a8e0 con 0x55e947b9a190 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.843+0000 7fda177fe640 1 --2- 172.21.15.16:0/352988150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947b9a190 0x55e947b96d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda0800f8c0 tx=0x7fda08023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5fffc0d19fc4aa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.843+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/352988150 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0801ed50 con 0x55e947b9a190 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.843+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/352988150 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda08038020 con 0x55e947b9a190 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.843+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/352988150 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0800fd70 con 0x55e947b9a190 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.844+0000 7fda209dc740 1 -- 172.21.15.16:0/352988150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947b9a190 msgr2=0x55e947b96d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.844+0000 7fda209dc740 1 --2- 172.21.15.16:0/352988150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947b9a190 0x55e947b96d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda0800f8c0 tx=0x7fda08023000 comp rx=0 tx=0).stop 2024-03-27T06:44:44.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.844+0000 7fda209dc740 1 -- 172.21.15.16:0/352988150 shutdown_connections 2024-03-27T06:44:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.844+0000 7fda209dc740 1 --2- 172.21.15.16:0/352988150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947ba80c0 0x55e947bccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.844+0000 7fda209dc740 1 --2- 172.21.15.16:0/352988150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9ab40 0x55e947ba0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.844+0000 7fda209dc740 1 --2- 172.21.15.16:0/352988150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947b9a190 0x55e947b96d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.844+0000 7fda209dc740 1 -- 172.21.15.16:0/352988150 >> 172.21.15.16:0/352988150 conn(0x55e947b94ee0 msgr2=0x55e947b94000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.844+0000 7fda209dc740 1 -- 172.21.15.16:0/352988150 shutdown_connections 2024-03-27T06:44:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.844+0000 7fda209dc740 1 -- 172.21.15.16:0/352988150 wait complete. 2024-03-27T06:44:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.845+0000 7fda209dc740 1 -- start start 2024-03-27T06:44:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.845+0000 7fda209dc740 1 Processor -- start 2024-03-27T06:44:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.845+0000 7fda209dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947bacc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda209dc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9ab40 0x55e947bad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947bacc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947bacc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34678/0 (socket says 172.21.15.16:34678) 2024-03-27T06:44:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda177fe640 1 -- 172.21.15.16:0/24332752 learned_addr learned my addr 172.21.15.16:0/24332752 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda209dc740 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947bad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda209dc740 1 -- 172.21.15.16:0/24332752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda181acb20 con 0x55e947b9a190 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda209dc740 1 -- 172.21.15.16:0/24332752 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fda181ac450 con 0x55e947ba80c0 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda209dc740 1 -- 172.21.15.16:0/24332752 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fda181a9690 con 0x55e947b9ab40 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda16ffd640 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9ab40 0x55e947bad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda17fff640 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947bad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda209dc740 10 client.0 _ll_register_callbacks cb 0x7fda08009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda177fe640 1 -- 172.21.15.16:0/24332752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9ab40 msgr2=0x55e947bad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda177fe640 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9ab40 0x55e947bad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda177fe640 1 -- 172.21.15.16:0/24332752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 msgr2=0x55e947bad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda177fe640 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947bad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda177fe640 1 -- 172.21.15.16:0/24332752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e947c2a2c0 con 0x55e947b9a190 2024-03-27T06:44:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda17fff640 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947bad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda177fe640 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947bacc50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fda080092e0 tx=0x7fda08005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09b59622ba85262 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fd9f6ffd640 10 client.15888 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.846+0000 7fda209dc740 1 -- 172.21.15.16:0/24332752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda08038020 con 0x55e947b9a190 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.847+0000 7fd9f6ffd640 1 -- 172.21.15.16:0/24332752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0802e040 con 0x55e947b9a190 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.847+0000 7fd9f6ffd640 1 -- 172.21.15.16:0/24332752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fda08005250 con 0x55e947b9a190 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.847+0000 7fd9f6ffd640 1 -- 172.21.15.16:0/24332752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fda08005490 con 0x55e947b9a190 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.847+0000 7fda209dc740 10 client.15888 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.847+0000 7fda209dc740 1 -- 172.21.15.16:0/24332752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55e947c2b450 con 0x55e947b9a190 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.847+0000 7fd9f6ffd640 1 -- 172.21.15.16:0/24332752 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fda08004620 con 0x55e947b9a190 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.847+0000 7fda209dc740 10 client.15888 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.848+0000 7fda209dc740 10 client.15888 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-27T06:44:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.848+0000 7fda209dc740 20 client.15888 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.848+0000 7fda209dc740 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e947bc4f00 0x55e947c2d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.848+0000 7fda209dc740 4 client.15888 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.848+0000 7fda209dc740 1 -- 172.21.15.16:0/24332752 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fda08038020 con 0x55e947bc4f00 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.848+0000 7fda16ffd640 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e947bc4f00 0x55e947c2d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.848+0000 7fd9f6ffd640 10 client.15888 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.848+0000 7fda16ffd640 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e947bc4f00 0x55e947c2d970 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=345a0b3f35c977db server_cookie=e5802372e931943 in_seq=0 out_seq=0 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.848+0000 7fd9f6ffd640 1 -- 172.21.15.16:0/24332752 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fda08038020 con 0x55e947bc4f00 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.848+0000 7fd9f6ffd640 10 client.15888 handle_command_reply: tid=0 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.848+0000 7fda209dc740 1 client.15888 shutdown 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fda209dc740 1 -- 172.21.15.16:0/24332752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 msgr2=0x55e947bacc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fda209dc740 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947bacc50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fda080092e0 tx=0x7fda08005860 comp rx=0 tx=0).stop 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fda209dc740 1 -- 172.21.15.16:0/24332752 shutdown_connections 2024-03-27T06:44:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fda209dc740 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e947bc4f00 0x55e947c2d970 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fda209dc740 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e947ba80c0 0x55e947bad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fda209dc740 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e947b9ab40 0x55e947bad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fda209dc740 1 --2- 172.21.15.16:0/24332752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e947b9a190 0x55e947bacc50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fda209dc740 1 -- 172.21.15.16:0/24332752 >> 172.21.15.16:0/24332752 conn(0x55e947b94ee0 msgr2=0x55e947b9fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fd9f6ffd640 0 client.15888 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:44:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fda209dc740 1 -- 172.21.15.16:0/24332752 shutdown_connections 2024-03-27T06:44:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fda209dc740 1 -- 172.21.15.16:0/24332752 wait complete. 2024-03-27T06:44:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.849+0000 7fda209dc740 20 client.15888 trim_cache size 0 max 16384 2024-03-27T06:44:44.850 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:44:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 -- 172.21.15.16:0/3525738781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fda000a39b0 msgr2=0x7fda000a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fda000a39b0 0x7fda000a5e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda08009e00 tx=0x7fda08005f90 comp rx=0 tx=0).stop 2024-03-27T06:44:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 -- 172.21.15.16:0/3525738781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda181066d0 msgr2=0x7fda181989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda181066d0 0x7fda181989e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fda0c005940 tx=0x7fda0c02dc90 comp rx=0 tx=0).stop 2024-03-27T06:44:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 -- 172.21.15.16:0/3525738781 shutdown_connections 2024-03-27T06:44:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda18107090 0x7fda1819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fda000a39b0 0x7fda000a5e70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda181066d0 0x7fda181989e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 --2- 172.21.15.16:0/3525738781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda18069130 0x7fda181984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 -- 172.21.15.16:0/3525738781 >> 172.21.15.16:0/3525738781 conn(0x7fda18077b50 msgr2=0x7fda180fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 -- 172.21.15.16:0/3525738781 shutdown_connections 2024-03-27T06:44:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.850+0000 7fda1dd13640 1 -- 172.21.15.16:0/3525738781 wait complete. 2024-03-27T06:44:44.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-27T06:44:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.974+0000 7f7d975bf640 1 -- 172.21.15.16:0/3546209499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d90068d30 msgr2=0x7f7d900691b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.974+0000 7f7d975bf640 1 --2- 172.21.15.16:0/3546209499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d90068d30 0x7f7d900691b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009930 tx=0x7f7d8002bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.974+0000 7f7d975bf640 1 -- 172.21.15.16:0/3546209499 shutdown_connections 2024-03-27T06:44:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.974+0000 7f7d975bf640 1 --2- 172.21.15.16:0/3546209499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d900696f0 0x7f7d9010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.974+0000 7f7d975bf640 1 --2- 172.21.15.16:0/3546209499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d90068d30 0x7f7d900691b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.974+0000 7f7d975bf640 1 --2- 172.21.15.16:0/3546209499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d90107ad0 0x7f7d90107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.974+0000 7f7d975bf640 1 -- 172.21.15.16:0/3546209499 >> 172.21.15.16:0/3546209499 conn(0x7f7d90077b50 msgr2=0x7f7d90077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.974+0000 7f7d975bf640 1 -- 172.21.15.16:0/3546209499 shutdown_connections 2024-03-27T06:44:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.974+0000 7f7d975bf640 1 -- 172.21.15.16:0/3546209499 wait complete. 2024-03-27T06:44:44.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d975bf640 1 Processor -- start 2024-03-27T06:44:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d975bf640 1 -- start start 2024-03-27T06:44:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d975bf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d90068d30 0x7f7d90100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d975bf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d900696f0 0x7f7d90100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d975bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d90107ad0 0x7f7d90101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d975bf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90111fb0 con 0x7f7d90107ad0 2024-03-27T06:44:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d975bf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90111e30 con 0x7f7d90068d30 2024-03-27T06:44:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d975bf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90112130 con 0x7f7d900696f0 2024-03-27T06:44:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d9535b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d90068d30 0x7f7d90100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d9535b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d90068d30 0x7f7d90100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60408/0 (socket says 172.21.15.16:60408) 2024-03-27T06:44:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d9535b640 1 -- 172.21.15.16:0/3809088615 learned_addr learned my addr 172.21.15.16:0/3809088615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d900696f0 0x7f7d90100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.975+0000 7f7d95b5c640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d90107ad0 0x7f7d90101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d9535b640 1 -- 172.21.15.16:0/3809088615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d900696f0 msgr2=0x7f7d90100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d9535b640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d900696f0 0x7f7d90100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d9535b640 1 -- 172.21.15.16:0/3809088615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d90107ad0 msgr2=0x7f7d90101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d9535b640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d90107ad0 0x7f7d90101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d9535b640 1 -- 172.21.15.16:0/3809088615 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d901a6a10 con 0x7f7d90068d30 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d900696f0 0x7f7d90100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d9535b640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d90068d30 0x7f7d90100a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00e9f0 tx=0x7f7d8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d867fc640 1 -- 172.21.15.16:0/3809088615 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00cd80 con 0x7f7d90068d30 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d867fc640 1 -- 172.21.15.16:0/3809088615 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7d8c010930 con 0x7f7d90068d30 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d975bf640 1 -- 172.21.15.16:0/3809088615 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d901a6d00 con 0x7f7d90068d30 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d867fc640 1 -- 172.21.15.16:0/3809088615 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c018b00 con 0x7f7d90068d30 2024-03-27T06:44:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.976+0000 7f7d975bf640 1 -- 172.21.15.16:0/3809088615 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d901a71c0 con 0x7f7d90068d30 2024-03-27T06:44:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.978+0000 7f7d867fc640 1 -- 172.21.15.16:0/3809088615 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7f7d8c018ca0 con 0x7f7d90068d30 2024-03-27T06:44:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.979+0000 7f7d867fc640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d7c0a39b0 0x7f7d7c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.979+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d7c0a39b0 0x7f7d7c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.979+0000 7f7d867fc640 1 -- 172.21.15.16:0/3809088615 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7d8c0d05c0 con 0x7f7d90068d30 2024-03-27T06:44:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d9a19b740 1 Processor -- start 2024-03-27T06:44:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d9a19b740 1 -- start start 2024-03-27T06:44:44.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d7c0a39b0 0x7f7d7c0a5e70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d80006010 tx=0x7f7d8002c730 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 0x55b01a6a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a30d0 0x55b01a6ab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6abcb0 0x55b01a6ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b01a606710 con 0x55b01a6abcb0 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55b01a5aeaf0 con 0x55b01a6a27b0 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b01a6026c0 con 0x55b01a6a30d0 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d9535b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 0x55b01a6a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d9535b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 0x55b01a6a2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60422/0 (socket says 172.21.15.16:60422) 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d9535b640 1 -- 172.21.15.16:0/429144300 learned_addr learned my addr 172.21.15.16:0/429144300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d95b5c640 1 --2- 172.21.15.16:0/429144300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6abcb0 0x55b01a6ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.980+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/429144300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a30d0 0x55b01a6ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.981+0000 7f7d9535b640 1 -- 172.21.15.16:0/429144300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a30d0 msgr2=0x55b01a6ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.981+0000 7f7d9535b640 1 --2- 172.21.15.16:0/429144300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a30d0 0x55b01a6ab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.981+0000 7f7d9535b640 1 -- 172.21.15.16:0/429144300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6abcb0 msgr2=0x55b01a6ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.981+0000 7f7d9535b640 1 --2- 172.21.15.16:0/429144300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6abcb0 0x55b01a6ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.981+0000 7f7d9535b640 1 -- 172.21.15.16:0/429144300 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b01a31a3f0 con 0x55b01a6a27b0 2024-03-27T06:44:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.981+0000 7f7d95b5c640 1 --2- 172.21.15.16:0/429144300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6abcb0 0x55b01a6ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.981+0000 7f7d9535b640 1 --2- 172.21.15.16:0/429144300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 0x55b01a6a2b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0cdbf0 tx=0x7f7d8c0184e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9e51d3bda32e2ce server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.981+0000 7f7d7b7fe640 1 -- 172.21.15.16:0/429144300 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0a08f0 con 0x55b01a6a27b0 2024-03-27T06:44:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.981+0000 7f7d7b7fe640 1 -- 172.21.15.16:0/429144300 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7d8c0a51b0 con 0x55b01a6a27b0 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.982+0000 7f7d9a19b740 1 -- 172.21.15.16:0/429144300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 msgr2=0x55b01a6a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.982+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/429144300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 0x55b01a6a2b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0cdbf0 tx=0x7f7d8c0184e0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.982+0000 7f7d9a19b740 1 -- 172.21.15.16:0/429144300 shutdown_connections 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.982+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/429144300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6abcb0 0x55b01a6ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.982+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/429144300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a30d0 0x55b01a6ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.982+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/429144300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 0x55b01a6a2b90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.982+0000 7f7d9a19b740 1 -- 172.21.15.16:0/429144300 >> 172.21.15.16:0/429144300 conn(0x55b01a69cee0 msgr2=0x55b01a69d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.982+0000 7f7d9a19b740 1 -- 172.21.15.16:0/429144300 shutdown_connections 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.982+0000 7f7d9a19b740 1 -- 172.21.15.16:0/429144300 wait complete. 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9a19b740 1 -- start start 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9a19b740 1 Processor -- start 2024-03-27T06:44:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a27b0 0x55b01a6a8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a30d0 0x55b01a6a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6abcb0 0x55b01a6a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b01a6aea80 con 0x55b01a6abcb0 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55b01a5c1eb0 con 0x55b01a6a30d0 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b01a6aec00 con 0x55b01a6a27b0 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9535b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a27b0 0x55b01a6a8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9535b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a27b0 0x55b01a6a8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:41456/0 (socket says 172.21.15.16:41456) 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9535b640 1 -- 172.21.15.16:0/2677467176 learned_addr learned my addr 172.21.15.16:0/2677467176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a30d0 0x55b01a6a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d9a19b740 10 client.0 _ll_register_callbacks cb 0x55b01a5fe6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.983+0000 7f7d95b5c640 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6abcb0 0x55b01a6a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d9535b640 1 -- 172.21.15.16:0/2677467176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a30d0 msgr2=0x55b01a6a91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d9535b640 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a30d0 0x55b01a6a91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d9535b640 1 -- 172.21.15.16:0/2677467176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6abcb0 msgr2=0x55b01a6a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d9535b640 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6abcb0 0x55b01a6a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d9535b640 1 -- 172.21.15.16:0/2677467176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b01a3e47d0 con 0x55b01a6a27b0 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d9535b640 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a27b0 0x55b01a6a8c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009250 tx=0x7f7d8c09bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a9a9185e92651bc server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d78ff9640 1 -- 172.21.15.16:0/2677467176 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0b5060 con 0x55b01a6a27b0 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d78ff9640 1 -- 172.21.15.16:0/2677467176 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7d8c0d0110 con 0x55b01a6a27b0 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d9a19b740 1 -- 172.21.15.16:0/2677467176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d8c0a51b0 con 0x55b01a6a27b0 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d78ff9640 1 -- 172.21.15.16:0/2677467176 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d8c0a08f0 con 0x55b01a6a27b0 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d9a19b740 10 client.25738 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.984+0000 7f7d9a19b740 1 -- 172.21.15.16:0/2677467176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b01a6aea80 con 0x55b01a6a27b0 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.985+0000 7f7d78ff9640 1 -- 172.21.15.16:0/2677467176 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7d8c0d0310 con 0x55b01a6a27b0 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.985+0000 7f7d78ff9640 1 -- 172.21.15.16:0/2677467176 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7d8c098320 con 0x55b01a6a27b0 2024-03-27T06:44:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.985+0000 7f7d9a19b740 10 client.25738 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.985+0000 7f7d9a19b740 10 client.25738 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.985+0000 7f7d9a19b740 20 client.25738 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.985+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55b01a6c4690 0x55b01a6c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.985+0000 7f7d9a19b740 4 client.25738 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.985+0000 7f7d9a19b740 1 -- 172.21.15.16:0/2677467176 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7d8c0a51b0 con 0x55b01a6c4690 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.985+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55b01a6c4690 0x55b01a6c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.985+0000 7f7d78ff9640 10 client.25738 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.985+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55b01a6c4690 0x55b01a6c6a80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63f05fa96470d09b server_cookie=34573567418a1f68 in_seq=0 out_seq=0 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d78ff9640 1 -- 172.21.15.16:0/2677467176 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f7d8c0a51b0 con 0x55b01a6c4690 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d78ff9640 10 client.25738 handle_command_reply: tid=0 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d9a19b740 1 client.25738 shutdown 2024-03-27T06:44:44.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d9a19b740 1 -- 172.21.15.16:0/2677467176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a27b0 msgr2=0x55b01a6a8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a27b0 0x55b01a6a8c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009250 tx=0x7f7d8c09bdc0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d9a19b740 1 -- 172.21.15.16:0/2677467176 shutdown_connections 2024-03-27T06:44:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55b01a6c4690 0x55b01a6c6a80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6abcb0 0x55b01a6a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a30d0 0x55b01a6a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/2677467176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6a27b0 0x55b01a6a8c60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d78ff9640 0 client.25738 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:44:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d9a19b740 1 -- 172.21.15.16:0/2677467176 >> 172.21.15.16:0/2677467176 conn(0x55b01a69cee0 msgr2=0x55b01a69ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d9a19b740 1 -- 172.21.15.16:0/2677467176 shutdown_connections 2024-03-27T06:44:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.986+0000 7f7d9a19b740 1 -- 172.21.15.16:0/2677467176 wait complete. 2024-03-27T06:44:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:44.987+0000 7f7d9a19b740 20 client.25738 trim_cache size 0 max 16384 2024-03-27T06:44:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d9a19b740 1 Processor -- start 2024-03-27T06:44:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d9a19b740 1 -- start start 2024-03-27T06:44:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 0x55b01a6ab390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a30d0 0x55b01a6c0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6abcb0 0x55b01a69be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7400e680 con 0x55b01a6a30d0 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f7d740101c0 con 0x55b01a6a27b0 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7400ed50 con 0x55b01a6abcb0 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d94b5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a30d0 0x55b01a6c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d94b5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a30d0 0x55b01a6c0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34714/0 (socket says 172.21.15.16:34714) 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d94b5a640 1 -- 172.21.15.16:0/4077789021 learned_addr learned my addr 172.21.15.16:0/4077789021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d9535b640 1 --2- 172.21.15.16:0/4077789021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 0x55b01a6ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.000+0000 7f7d95b5c640 1 --2- 172.21.15.16:0/4077789021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6abcb0 0x55b01a69be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.001+0000 7f7d94b5a640 1 -- 172.21.15.16:0/4077789021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6abcb0 msgr2=0x55b01a69be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.001+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/4077789021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6abcb0 0x55b01a69be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.001+0000 7f7d94b5a640 1 -- 172.21.15.16:0/4077789021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 msgr2=0x55b01a6ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.001+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/4077789021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 0x55b01a6ab390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.001+0000 7f7d94b5a640 1 -- 172.21.15.16:0/4077789021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b01a6a2f00 con 0x55b01a6a30d0 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.001+0000 7f7d95b5c640 1 --2- 172.21.15.16:0/4077789021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6abcb0 0x55b01a69be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.001+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/4077789021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a30d0 0x55b01a6c0130 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009930 tx=0x7f7d8003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db29b7f8b3370a46 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.001+0000 7f7d7affd640 1 -- 172.21.15.16:0/4077789021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80056070 con 0x55b01a6a30d0 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.001+0000 7f7d7affd640 1 -- 172.21.15.16:0/4077789021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7d8002cab0 con 0x55b01a6a30d0 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.001+0000 7f7d7affd640 1 -- 172.21.15.16:0/4077789021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80051070 con 0x55b01a6a30d0 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.002+0000 7f7d9a19b740 1 -- 172.21.15.16:0/4077789021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a30d0 msgr2=0x55b01a6c0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.002+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/4077789021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a30d0 0x55b01a6c0130 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009930 tx=0x7f7d8003d000 comp rx=0 tx=0).stop 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.002+0000 7f7d9a19b740 1 -- 172.21.15.16:0/4077789021 shutdown_connections 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.002+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/4077789021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6abcb0 0x55b01a69be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.002+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/4077789021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a30d0 0x55b01a6c0130 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.002+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/4077789021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a6a27b0 0x55b01a6ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.002+0000 7f7d9a19b740 1 -- 172.21.15.16:0/4077789021 >> 172.21.15.16:0/4077789021 conn(0x55b01a69cee0 msgr2=0x55b01a69b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.002+0000 7f7d9a19b740 1 -- 172.21.15.16:0/4077789021 shutdown_connections 2024-03-27T06:44:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.002+0000 7f7d9a19b740 1 -- 172.21.15.16:0/4077789021 wait complete. 2024-03-27T06:44:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.002+0000 7f7d9a19b740 1 -- start start 2024-03-27T06:44:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9a19b740 1 Processor -- start 2024-03-27T06:44:45.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a27b0 0x55b01a6a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6abcb0 0x55b01a6a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9a19b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a727750 0x55b01a729b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7400faf0 con 0x55b01a6a27b0 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7400f420 con 0x55b01a727750 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9a19b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7400c660 con 0x55b01a6abcb0 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9a19b740 10 client.0 _ll_register_callbacks cb 0x7f7d88009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9535b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a27b0 0x55b01a6a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9535b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a27b0 0x55b01a6a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34716/0 (socket says 172.21.15.16:34716) 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9535b640 1 -- 172.21.15.16:0/1598620275 learned_addr learned my addr 172.21.15.16:0/1598620275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d95b5c640 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a727750 0x55b01a729b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9535b640 1 -- 172.21.15.16:0/1598620275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6abcb0 msgr2=0x55b01a6a62a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.003+0000 7f7d9535b640 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6abcb0 0x55b01a6a62a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d9535b640 1 -- 172.21.15.16:0/1598620275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a727750 msgr2=0x55b01a729b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d9535b640 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a727750 0x55b01a729b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d9535b640 1 -- 172.21.15.16:0/1598620275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b01a72a080 con 0x55b01a6a27b0 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d9535b640 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a27b0 0x55b01a6a5d60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c09c4d0 tx=0x7f7d8c098de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1f413315690085 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d78ff9640 1 -- 172.21.15.16:0/1598620275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0b5070 con 0x55b01a6a27b0 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d78ff9640 1 -- 172.21.15.16:0/1598620275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7d8c0d0310 con 0x55b01a6a27b0 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d9a19b740 1 -- 172.21.15.16:0/1598620275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d8002cab0 con 0x55b01a6a27b0 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d78ff9640 1 -- 172.21.15.16:0/1598620275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d8c0987e0 con 0x55b01a6a27b0 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d9a19b740 10 client.15906 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d9a19b740 1 -- 172.21.15.16:0/1598620275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b01a72b1a0 con 0x55b01a6a27b0 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.004+0000 7f7d78ff9640 1 -- 172.21.15.16:0/1598620275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7d8c0a51b0 con 0x55b01a6a27b0 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d78ff9640 1 -- 172.21.15.16:0/1598620275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7d8c010c70 con 0x55b01a6a27b0 2024-03-27T06:44:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d9a19b740 10 client.15906 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d9a19b740 10 client.15906 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d9a19b740 20 client.15906 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55b01a72b680 0x55b01a72da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d9a19b740 4 client.15906 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d9a19b740 1 -- 172.21.15.16:0/1598620275 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7d8002cab0 con 0x55b01a72b680 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55b01a72b680 0x55b01a72da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d78ff9640 10 client.15906 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d94b5a640 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55b01a72b680 0x55b01a72da70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a2765e2f8886229 server_cookie=89b5e37961fd466c in_seq=0 out_seq=0 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d78ff9640 1 -- 172.21.15.16:0/1598620275 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7d8002cab0 con 0x55b01a72b680 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d78ff9640 10 client.15906 handle_command_reply: tid=0 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.005+0000 7f7d9a19b740 1 client.15906 shutdown 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d9a19b740 1 -- 172.21.15.16:0/1598620275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a27b0 msgr2=0x55b01a6a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a27b0 0x55b01a6a5d60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c09c4d0 tx=0x7f7d8c098de0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d9a19b740 1 -- 172.21.15.16:0/1598620275 shutdown_connections 2024-03-27T06:44:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x55b01a72b680 0x55b01a72da70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b01a727750 0x55b01a729b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b01a6abcb0 0x55b01a6a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d9a19b740 1 --2- 172.21.15.16:0/1598620275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b01a6a27b0 0x55b01a6a5d60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d9a19b740 1 -- 172.21.15.16:0/1598620275 >> 172.21.15.16:0/1598620275 conn(0x55b01a69cee0 msgr2=0x55b01a69ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d78ff9640 0 client.15906 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:44:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d9a19b740 1 -- 172.21.15.16:0/1598620275 shutdown_connections 2024-03-27T06:44:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d9a19b740 1 -- 172.21.15.16:0/1598620275 wait complete. 2024-03-27T06:44:45.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d9a19b740 20 client.15906 trim_cache size 0 max 16384 2024-03-27T06:44:45.006 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:44:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d975bf640 1 -- 172.21.15.16:0/3809088615 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d7c0a39b0 msgr2=0x7f7d7c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d975bf640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d7c0a39b0 0x7f7d7c0a5e70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d80006010 tx=0x7f7d8002c730 comp rx=0 tx=0).stop 2024-03-27T06:44:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.006+0000 7f7d975bf640 1 -- 172.21.15.16:0/3809088615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d90068d30 msgr2=0x7f7d90100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.007+0000 7f7d975bf640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d90068d30 0x7f7d90100a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00e9f0 tx=0x7f7d8c00eeb0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.007+0000 7f7d975bf640 1 -- 172.21.15.16:0/3809088615 shutdown_connections 2024-03-27T06:44:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.007+0000 7f7d975bf640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d90107ad0 0x7f7d90101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.007+0000 7f7d975bf640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d7c0a39b0 0x7f7d7c0a5e70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.007+0000 7f7d975bf640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d900696f0 0x7f7d90100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.007+0000 7f7d975bf640 1 --2- 172.21.15.16:0/3809088615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d90068d30 0x7f7d90100a10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.007+0000 7f7d975bf640 1 -- 172.21.15.16:0/3809088615 >> 172.21.15.16:0/3809088615 conn(0x7f7d90077b50 msgr2=0x7f7d900fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.007+0000 7f7d975bf640 1 -- 172.21.15.16:0/3809088615 shutdown_connections 2024-03-27T06:44:45.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.007+0000 7f7d975bf640 1 -- 172.21.15.16:0/3809088615 wait complete. 2024-03-27T06:44:45.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-27T06:44:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.126+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/2820278063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b38101030 msgr2=0x7f8b38109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.126+0000 7f8b3d4c9640 1 --2- 172.21.15.16:0/2820278063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b38101030 0x7f8b38109bd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009900 tx=0x7f8b2c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:45.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.126+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/2820278063 shutdown_connections 2024-03-27T06:44:45.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.126+0000 7f8b3d4c9640 1 --2- 172.21.15.16:0/2820278063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b3810a300 0x7f8b38110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.126+0000 7f8b3d4c9640 1 --2- 172.21.15.16:0/2820278063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b38101030 0x7f8b38109bd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.126+0000 7f8b3d4c9640 1 --2- 172.21.15.16:0/2820278063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b38100680 0x7f8b38100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.126+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/2820278063 >> 172.21.15.16:0/2820278063 conn(0x7f8b38075d90 msgr2=0x7f8b380761a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.126+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/2820278063 shutdown_connections 2024-03-27T06:44:45.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.126+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/2820278063 wait complete. 2024-03-27T06:44:45.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.127+0000 7f8b3d4c9640 1 Processor -- start 2024-03-27T06:44:45.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.127+0000 7f8b3d4c9640 1 -- start start 2024-03-27T06:44:45.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.127+0000 7f8b3d4c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b38100680 0x7f8b3819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.127+0000 7f8b3d4c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b3810a300 0x7f8b3819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.127+0000 7f8b3d4c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3819efb0 0x7f8b3819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.127+0000 7f8b3d4c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38114230 con 0x7f8b38100680 2024-03-27T06:44:45.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.127+0000 7f8b3d4c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381140b0 con 0x7f8b3810a300 2024-03-27T06:44:45.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.127+0000 7f8b3d4c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381143b0 con 0x7f8b3819efb0 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b38100680 0x7f8b3819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b367fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b3810a300 0x7f8b3819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b38100680 0x7f8b3819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34736/0 (socket says 172.21.15.16:34736) 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b36ffd640 1 -- 172.21.15.16:0/4067816599 learned_addr learned my addr 172.21.15.16:0/4067816599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b377fe640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3819efb0 0x7f8b3819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b367fc640 1 -- 172.21.15.16:0/4067816599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3819efb0 msgr2=0x7f8b3819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b367fc640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3819efb0 0x7f8b3819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b367fc640 1 -- 172.21.15.16:0/4067816599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b38100680 msgr2=0x7f8b3819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b367fc640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b38100680 0x7f8b3819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b367fc640 1 -- 172.21.15.16:0/4067816599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3819faa0 con 0x7f8b3810a300 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b38100680 0x7f8b3819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b377fe640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3819efb0 0x7f8b3819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b367fc640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b3810a300 0x7f8b3819ac30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c02c170 tx=0x7f8b2c02dbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b1ffff640 1 -- 172.21.15.16:0/4067816599 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c043020 con 0x7f8b3810a300 2024-03-27T06:44:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b1ffff640 1 -- 172.21.15.16:0/4067816599 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b2c004e50 con 0x7f8b3810a300 2024-03-27T06:44:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.128+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/4067816599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3819fd30 con 0x7f8b3810a300 2024-03-27T06:44:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.129+0000 7f8b1ffff640 1 -- 172.21.15.16:0/4067816599 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c003a10 con 0x7f8b3810a300 2024-03-27T06:44:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.129+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/4067816599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b381a74f0 con 0x7f8b3810a300 2024-03-27T06:44:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.131+0000 7f8b1ffff640 1 -- 172.21.15.16:0/4067816599 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7f8b2c046020 con 0x7f8b3810a300 2024-03-27T06:44:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.131+0000 7f8b1ffff640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f8b200a3930 0x7f8b200a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.132+0000 7f8b1ffff640 1 -- 172.21.15.16:0/4067816599 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8b2c0f23d0 con 0x7f8b3810a300 2024-03-27T06:44:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.132+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f8b200a3930 0x7f8b200a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.132+0000 7f8b40193740 1 Processor -- start 2024-03-27T06:44:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.132+0000 7f8b40193740 1 -- start start 2024-03-27T06:44:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.132+0000 7f8b40193740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.132+0000 7f8b40193740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d668e0 0x55edb7d6ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.132+0000 7f8b40193740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d6f4c0 0x55edb7d718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.132+0000 7f8b40193740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55edb7cc1710 con 0x55edb7d6f4c0 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.132+0000 7f8b40193740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55edb7c69af0 con 0x55edb7d668e0 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.132+0000 7f8b40193740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55edb7cbd6c0 con 0x55edb7d65fc0 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d6f4c0 0x55edb7d718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f8b200a3930 0x7f8b200a5df0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8b280058c0 tx=0x7f8b28005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b367fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d668e0 0x55edb7d6ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d6f4c0 0x55edb7d718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34744/0 (socket says 172.21.15.16:34744) 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b377fe640 1 -- 172.21.15.16:0/1640200722 learned_addr learned my addr 172.21.15.16:0/1640200722 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d663a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:41508/0 (socket says 172.21.15.16:41508) 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b36ffd640 1 -- 172.21.15.16:0/1640200722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d668e0 msgr2=0x55edb7d6ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/1640200722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d668e0 0x55edb7d6ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b36ffd640 1 -- 172.21.15.16:0/1640200722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d6f4c0 msgr2=0x55edb7d718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/1640200722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d6f4c0 0x55edb7d718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b36ffd640 1 -- 172.21.15.16:0/1640200722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edb79d53f0 con 0x55edb7d65fc0 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b377fe640 1 --2- 172.21.15.16:0/1640200722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d6f4c0 0x55edb7d718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b367fc640 1 --2- 172.21.15.16:0/1640200722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d668e0 0x55edb7d6ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/1640200722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d663a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b28015000 tx=0x7f8b28014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=face82fcacbd2789 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b1d7fa640 1 -- 172.21.15.16:0/1640200722 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28026440 con 0x55edb7d65fc0 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.133+0000 7f8b1d7fa640 1 -- 172.21.15.16:0/1640200722 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b2801c910 con 0x55edb7d65fc0 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.134+0000 7f8b40193740 1 -- 172.21.15.16:0/1640200722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 msgr2=0x55edb7d663a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.134+0000 7f8b40193740 1 --2- 172.21.15.16:0/1640200722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d663a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b28015000 tx=0x7f8b28014000 comp rx=0 tx=0).stop 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.134+0000 7f8b40193740 1 -- 172.21.15.16:0/1640200722 shutdown_connections 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.134+0000 7f8b40193740 1 --2- 172.21.15.16:0/1640200722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d6f4c0 0x55edb7d718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.134+0000 7f8b40193740 1 --2- 172.21.15.16:0/1640200722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d668e0 0x55edb7d6ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.134+0000 7f8b40193740 1 --2- 172.21.15.16:0/1640200722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d663a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.135+0000 7f8b40193740 1 -- 172.21.15.16:0/1640200722 >> 172.21.15.16:0/1640200722 conn(0x55edb7d606f0 msgr2=0x55edb7d60ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.135+0000 7f8b40193740 1 -- 172.21.15.16:0/1640200722 shutdown_connections 2024-03-27T06:44:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.135+0000 7f8b40193740 1 -- 172.21.15.16:0/1640200722 wait complete. 2024-03-27T06:44:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.135+0000 7f8b40193740 1 -- start start 2024-03-27T06:44:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.135+0000 7f8b40193740 1 Processor -- start 2024-03-27T06:44:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.135+0000 7f8b40193740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d6c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.135+0000 7f8b40193740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 0x55edb7d6c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d6c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d6c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:41514/0 (socket says 172.21.15.16:41514) 2024-03-27T06:44:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b367fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 0x55edb7d6c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b367fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 0x55edb7d6c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34760/0 (socket says 172.21.15.16:34760) 2024-03-27T06:44:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b36ffd640 1 -- 172.21.15.16:0/501028374 learned_addr learned my addr 172.21.15.16:0/501028374 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b40193740 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d6f4c0 0x55edb7d6cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b40193740 1 -- 172.21.15.16:0/501028374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55edb7d72300 con 0x55edb7d668e0 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b40193740 1 -- 172.21.15.16:0/501028374 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55edb7c7ceb0 con 0x55edb7d6f4c0 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b40193740 1 -- 172.21.15.16:0/501028374 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55edb7d72480 con 0x55edb7d65fc0 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b40193740 10 client.0 _ll_register_callbacks cb 0x55edb7cb96a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b36ffd640 1 -- 172.21.15.16:0/501028374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d6f4c0 msgr2=0x55edb7d6cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b377fe640 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d6f4c0 0x55edb7d6cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d6f4c0 0x55edb7d6cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b36ffd640 1 -- 172.21.15.16:0/501028374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 msgr2=0x55edb7d6c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 0x55edb7d6c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b36ffd640 1 -- 172.21.15.16:0/501028374 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edb7a9f7d0 con 0x55edb7d65fc0 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b367fc640 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 0x55edb7d6c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.136+0000 7f8b377fe640 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d6f4c0 0x55edb7d6cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d6c470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800fd90 tx=0x7f8b2800fdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e011c0a1dac8f80 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b1aff5640 1 -- 172.21.15.16:0/501028374 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2801c620 con 0x55edb7d65fc0 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b1aff5640 1 -- 172.21.15.16:0/501028374 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8b2801cac0 con 0x55edb7d65fc0 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b40193740 1 -- 172.21.15.16:0/501028374 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b2801c910 con 0x55edb7d65fc0 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b1aff5640 1 -- 172.21.15.16:0/501028374 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8b2801ccc0 con 0x55edb7d65fc0 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b40193740 10 client.25753 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b40193740 1 -- 172.21.15.16:0/501028374 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55edb7d7e550 con 0x55edb7d65fc0 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b1aff5640 1 -- 172.21.15.16:0/501028374 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b2801c910 con 0x55edb7d65fc0 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b1aff5640 1 -- 172.21.15.16:0/501028374 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8b28024070 con 0x55edb7d65fc0 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b40193740 10 client.25753 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b40193740 10 client.25753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b40193740 20 client.25753 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b40193740 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55edb7d7f8f0 0x55edb7d81ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b40193740 4 client.25753 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.137+0000 7f8b40193740 1 -- 172.21.15.16:0/501028374 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edb7ccec50 con 0x55edb7d7f8f0 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.138+0000 7f8b367fc640 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55edb7d7f8f0 0x55edb7d81ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.138+0000 7f8b1aff5640 10 client.25753 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.138+0000 7f8b367fc640 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55edb7d7f8f0 0x55edb7d81ce0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22bee949bc9384e server_cookie=36f9bda1b03b7285 in_seq=0 out_seq=0 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b1aff5640 1 -- 172.21.15.16:0/501028374 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55edb7ccec50 con 0x55edb7d7f8f0 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b1aff5640 10 client.25753 handle_command_reply: tid=0 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 1 client.25753 shutdown 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 1 -- 172.21.15.16:0/501028374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 msgr2=0x55edb7d6c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d6c470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800fd90 tx=0x7f8b2800fdc0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 1 -- 172.21.15.16:0/501028374 shutdown_connections 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55edb7d7f8f0 0x55edb7d81ce0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b1aff5640 0 client.25753 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d6f4c0 0x55edb7d6cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 0x55edb7d6c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 1 --2- 172.21.15.16:0/501028374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d65fc0 0x55edb7d6c470 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 1 -- 172.21.15.16:0/501028374 >> 172.21.15.16:0/501028374 conn(0x55edb7d606f0 msgr2=0x55edb7d62440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 1 -- 172.21.15.16:0/501028374 shutdown_connections 2024-03-27T06:44:45.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 1 -- 172.21.15.16:0/501028374 wait complete. 2024-03-27T06:44:45.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.139+0000 7f8b40193740 20 client.25753 trim_cache size 0 max 16384 2024-03-27T06:44:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b40193740 1 Processor -- start 2024-03-27T06:44:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b40193740 1 -- start start 2024-03-27T06:44:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b40193740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d65fc0 0x55edb7d6eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b40193740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d668e0 0x55edb7d7b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b40193740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d6f4c0 0x55edb7d6cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b40193740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381ab620 con 0x55edb7d65fc0 2024-03-27T06:44:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b40193740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381ad160 con 0x55edb7d668e0 2024-03-27T06:44:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b40193740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381abcf0 con 0x55edb7d6f4c0 2024-03-27T06:44:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d65fc0 0x55edb7d6eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d6f4c0 0x55edb7d6cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d6f4c0 0x55edb7d6cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:41530/0 (socket says 172.21.15.16:41530) 2024-03-27T06:44:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d65fc0 0x55edb7d6eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34766/0 (socket says 172.21.15.16:34766) 2024-03-27T06:44:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b377fe640 1 -- 172.21.15.16:0/3025955617 learned_addr learned my addr 172.21.15.16:0/3025955617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b367fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d668e0 0x55edb7d7b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b36ffd640 1 -- 172.21.15.16:0/3025955617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d6f4c0 msgr2=0x55edb7d6cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/3025955617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d6f4c0 0x55edb7d6cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b36ffd640 1 -- 172.21.15.16:0/3025955617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d668e0 msgr2=0x55edb7d7b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.153+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/3025955617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d668e0 0x55edb7d7b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.154+0000 7f8b36ffd640 1 -- 172.21.15.16:0/3025955617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edb7d60520 con 0x55edb7d65fc0 2024-03-27T06:44:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.154+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/3025955617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d65fc0 0x55edb7d6eba0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800fe30 tx=0x7f8b2800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1c8cde5c59518d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.154+0000 7f8b1cff9640 1 -- 172.21.15.16:0/3025955617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28020020 con 0x55edb7d65fc0 2024-03-27T06:44:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.154+0000 7f8b1cff9640 1 -- 172.21.15.16:0/3025955617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b28009db0 con 0x55edb7d65fc0 2024-03-27T06:44:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.154+0000 7f8b1cff9640 1 -- 172.21.15.16:0/3025955617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2801ce60 con 0x55edb7d65fc0 2024-03-27T06:44:45.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 -- 172.21.15.16:0/3025955617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d65fc0 msgr2=0x55edb7d6eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 --2- 172.21.15.16:0/3025955617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d65fc0 0x55edb7d6eba0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800fe30 tx=0x7f8b2800f610 comp rx=0 tx=0).stop 2024-03-27T06:44:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 -- 172.21.15.16:0/3025955617 shutdown_connections 2024-03-27T06:44:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 --2- 172.21.15.16:0/3025955617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d6f4c0 0x55edb7d6cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 --2- 172.21.15.16:0/3025955617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d668e0 0x55edb7d7b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 --2- 172.21.15.16:0/3025955617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d65fc0 0x55edb7d6eba0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 -- 172.21.15.16:0/3025955617 >> 172.21.15.16:0/3025955617 conn(0x55edb7d606f0 msgr2=0x55edb7d57000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 -- 172.21.15.16:0/3025955617 shutdown_connections 2024-03-27T06:44:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 -- 172.21.15.16:0/3025955617 wait complete. 2024-03-27T06:44:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 -- start start 2024-03-27T06:44:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 Processor -- start 2024-03-27T06:44:45.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.155+0000 7f8b40193740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 0x55edb7d69870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b40193740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d6f4c0 0x55edb7d69db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b40193740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d7f8f0 0x55edb7d686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b40193740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381aca90 con 0x55edb7d668e0 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b40193740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381ac3c0 con 0x55edb7d6f4c0 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b40193740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381a9620 con 0x55edb7d7f8f0 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b40193740 10 client.0 _ll_register_callbacks cb 0x7f8b30009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 0x55edb7d69870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d7f8f0 0x55edb7d686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d7f8f0 0x55edb7d686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:41542/0 (socket says 172.21.15.16:41542) 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b377fe640 1 -- 172.21.15.16:0/4098975657 learned_addr learned my addr 172.21.15.16:0/4098975657 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b367fc640 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d6f4c0 0x55edb7d69db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b377fe640 1 -- 172.21.15.16:0/4098975657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d6f4c0 msgr2=0x55edb7d69db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b377fe640 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d6f4c0 0x55edb7d69db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b377fe640 1 -- 172.21.15.16:0/4098975657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 msgr2=0x55edb7d69870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b377fe640 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 0x55edb7d69870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b377fe640 1 -- 172.21.15.16:0/4098975657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edb7d77c90 con 0x55edb7d7f8f0 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b377fe640 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d7f8f0 0x55edb7d686e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8b30010f30 tx=0x7f8b3000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53ac64bfe764bbed server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.156+0000 7f8b1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b1aff5640 1 -- 172.21.15.16:0/4098975657 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30003970 con 0x55edb7d7f8f0 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b1aff5640 1 -- 172.21.15.16:0/4098975657 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8b30003b10 con 0x55edb7d7f8f0 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b40193740 1 -- 172.21.15.16:0/4098975657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b28009db0 con 0x55edb7d7f8f0 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b1aff5640 1 -- 172.21.15.16:0/4098975657 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8b3002a050 con 0x55edb7d7f8f0 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b40193740 10 client.25759 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b40193740 1 -- 172.21.15.16:0/4098975657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55edb7de4f50 con 0x55edb7d7f8f0 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b1aff5640 1 -- 172.21.15.16:0/4098975657 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b30003d10 con 0x55edb7d7f8f0 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b1aff5640 1 -- 172.21.15.16:0/4098975657 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8b3001c900 con 0x55edb7d7f8f0 2024-03-27T06:44:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b40193740 10 client.25759 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b40193740 10 client.25759 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b40193740 20 client.25759 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b40193740 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55edb7de5710 0x55edb7de7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b40193740 4 client.25759 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b40193740 1 -- 172.21.15.16:0/4098975657 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8b28009db0 con 0x55edb7de5710 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.157+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55edb7de5710 0x55edb7de7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b1aff5640 10 client.25759 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b36ffd640 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55edb7de5710 0x55edb7de7b00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f4568484e6e02a8 server_cookie=7e2a74abb1fb8761 in_seq=0 out_seq=0 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b1aff5640 1 -- 172.21.15.16:0/4098975657 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8b28009db0 con 0x55edb7de5710 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b1aff5640 10 client.25759 handle_command_reply: tid=0 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 1 client.25759 shutdown 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 1 -- 172.21.15.16:0/4098975657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d7f8f0 msgr2=0x55edb7d686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d7f8f0 0x55edb7d686e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8b30010f30 tx=0x7f8b3000f790 comp rx=0 tx=0).stop 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 1 -- 172.21.15.16:0/4098975657 shutdown_connections 2024-03-27T06:44:45.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55edb7de5710 0x55edb7de7b00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55edb7d7f8f0 0x55edb7d686e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b1aff5640 0 client.25759 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:44:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55edb7d6f4c0 0x55edb7d69db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 1 --2- 172.21.15.16:0/4098975657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edb7d668e0 0x55edb7d69870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 1 -- 172.21.15.16:0/4098975657 >> 172.21.15.16:0/4098975657 conn(0x55edb7d606f0 msgr2=0x55edb7d6ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 1 -- 172.21.15.16:0/4098975657 shutdown_connections 2024-03-27T06:44:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 1 -- 172.21.15.16:0/4098975657 wait complete. 2024-03-27T06:44:45.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.158+0000 7f8b40193740 20 client.25759 trim_cache size 0 max 16384 2024-03-27T06:44:45.158 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:44:45.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/4067816599 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f8b200a3930 msgr2=0x7f8b200a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f8b200a3930 0x7f8b200a5df0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8b280058c0 tx=0x7f8b28005f90 comp rx=0 tx=0).stop 2024-03-27T06:44:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/4067816599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b3810a300 msgr2=0x7f8b3819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b3810a300 0x7f8b3819ac30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c02c170 tx=0x7f8b2c02dbf0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/4067816599 shutdown_connections 2024-03-27T06:44:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f8b200a3930 0x7f8b200a5df0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b3819efb0 0x7f8b3819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b3810a300 0x7f8b3819ac30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 --2- 172.21.15.16:0/4067816599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b38100680 0x7f8b3819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/4067816599 >> 172.21.15.16:0/4067816599 conn(0x7f8b38075d90 msgr2=0x7f8b38107d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/4067816599 shutdown_connections 2024-03-27T06:44:45.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.159+0000 7f8b3d4c9640 1 -- 172.21.15.16:0/4067816599 wait complete. 2024-03-27T06:44:45.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-27T06:44:45.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.279+0000 7f287890d640 1 -- 172.21.15.16:0/3264695886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28741066d0 msgr2=0x7f2874106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.279+0000 7f287890d640 1 --2- 172.21.15.16:0/3264695886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28741066d0 0x7f2874106b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2868009900 tx=0x7f286802bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.279+0000 7f287890d640 1 -- 172.21.15.16:0/3264695886 shutdown_connections 2024-03-27T06:44:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.279+0000 7f287890d640 1 --2- 172.21.15.16:0/3264695886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2874107090 0x7f287410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.279+0000 7f287890d640 1 --2- 172.21.15.16:0/3264695886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28741066d0 0x7f2874106b50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.279+0000 7f287890d640 1 --2- 172.21.15.16:0/3264695886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2874069130 0x7f2874069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.279+0000 7f287890d640 1 -- 172.21.15.16:0/3264695886 >> 172.21.15.16:0/3264695886 conn(0x7f2874077b50 msgr2=0x7f2874077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.279+0000 7f287890d640 1 -- 172.21.15.16:0/3264695886 shutdown_connections 2024-03-27T06:44:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.279+0000 7f287890d640 1 -- 172.21.15.16:0/3264695886 wait complete. 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.280+0000 7f287890d640 1 Processor -- start 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.280+0000 7f287890d640 1 -- start start 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.280+0000 7f287890d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2874069130 0x7f2874100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.280+0000 7f287890d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28741066d0 0x7f2874100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.280+0000 7f287890d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2874107090 0x7f2874101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f287890d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2874111fa0 con 0x7f28741066d0 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f287890d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f2874111e20 con 0x7f2874069130 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f287890d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2874112120 con 0x7f2874107090 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f2871d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28741066d0 0x7f2874100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f2871d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28741066d0 0x7f2874100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34794/0 (socket says 172.21.15.16:34794) 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f287259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2874069130 0x7f2874100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f2871d9b640 1 -- 172.21.15.16:0/1690946192 learned_addr learned my addr 172.21.15.16:0/1690946192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f2872d9d640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2874107090 0x7f2874101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f2871d9b640 1 -- 172.21.15.16:0/1690946192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2874107090 msgr2=0x7f2874101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f2871d9b640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2874107090 0x7f2874101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f2871d9b640 1 -- 172.21.15.16:0/1690946192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2874069130 msgr2=0x7f2874100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f2871d9b640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2874069130 0x7f2874100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f2871d9b640 1 -- 172.21.15.16:0/1690946192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28741a6a40 con 0x7f28741066d0 2024-03-27T06:44:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f2871d9b640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28741066d0 0x7f2874100f50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2868005ab0 tx=0x7f286802dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f28637fe640 1 -- 172.21.15.16:0/1690946192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868043020 con 0x7f28741066d0 2024-03-27T06:44:45.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.281+0000 7f287890d640 1 -- 172.21.15.16:0/1690946192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28741a6cd0 con 0x7f28741066d0 2024-03-27T06:44:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.282+0000 7f28637fe640 1 -- 172.21.15.16:0/1690946192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2868004250 con 0x7f28741066d0 2024-03-27T06:44:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.282+0000 7f28637fe640 1 -- 172.21.15.16:0/1690946192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868003ab0 con 0x7f28741066d0 2024-03-27T06:44:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.282+0000 7f287890d640 1 -- 172.21.15.16:0/1690946192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28741a71c0 con 0x7f28741066d0 2024-03-27T06:44:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.284+0000 7f28637fe640 1 -- 172.21.15.16:0/1690946192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7f2868046020 con 0x7f28741066d0 2024-03-27T06:44:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.285+0000 7f28637fe640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f285c0a39b0 0x7f285c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.285+0000 7f287259c640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f285c0a39b0 0x7f285c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.285+0000 7f28637fe640 1 -- 172.21.15.16:0/1690946192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f28680f1c80 con 0x7f28741066d0 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.285+0000 7f287b5cd740 1 Processor -- start 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.285+0000 7f287b5cd740 1 -- start start 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287259c640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f285c0a39b0 0x7f285c0a5e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f286400bbe0 tx=0x7f286400b390 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c1f70 0x55cac76d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76d34c0 0x55cac76d58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287b5cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cac7625710 con 0x55cac76d34c0 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287b5cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55cac75cdaf0 con 0x55cac76c15c0 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287b5cd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cac76216c0 con 0x55cac76c1f70 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f2871d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c1f70 0x55cac76d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f2872d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76d34c0 0x55cac76d58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76c19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60546/0 (socket says 172.21.15.16:60546) 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f2871d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c1f70 0x55cac76d2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:41568/0 (socket says 172.21.15.16:41568) 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287259c640 1 -- 172.21.15.16:0/1704841 learned_addr learned my addr 172.21.15.16:0/1704841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287259c640 1 -- 172.21.15.16:0/1704841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c1f70 msgr2=0x55cac76d2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287259c640 1 --2- 172.21.15.16:0/1704841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c1f70 0x55cac76d2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287259c640 1 -- 172.21.15.16:0/1704841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76d34c0 msgr2=0x55cac76d58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287259c640 1 --2- 172.21.15.16:0/1704841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76d34c0 0x55cac76d58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287259c640 1 -- 172.21.15.16:0/1704841 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac73393f0 con 0x55cac76c15c0 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.286+0000 7f287259c640 1 --2- 172.21.15.16:0/1704841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76c19a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2864011440 tx=0x7f2864013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9f97f3eee6cb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.287+0000 7f2860ff9640 1 -- 172.21.15.16:0/1704841 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864005220 con 0x55cac76c15c0 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.287+0000 7f2860ff9640 1 -- 172.21.15.16:0/1704841 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2864004ca0 con 0x55cac76c15c0 2024-03-27T06:44:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.287+0000 7f2860ff9640 1 -- 172.21.15.16:0/1704841 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286401eb60 con 0x55cac76c15c0 2024-03-27T06:44:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.288+0000 7f287b5cd740 1 -- 172.21.15.16:0/1704841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 msgr2=0x55cac76c19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.288+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1704841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76c19a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2864011440 tx=0x7f2864013000 comp rx=0 tx=0).stop 2024-03-27T06:44:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.288+0000 7f287b5cd740 1 -- 172.21.15.16:0/1704841 shutdown_connections 2024-03-27T06:44:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.288+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1704841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76d34c0 0x55cac76d58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.288+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1704841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c1f70 0x55cac76d2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.288+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1704841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76c19a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.288+0000 7f287b5cd740 1 -- 172.21.15.16:0/1704841 >> 172.21.15.16:0/1704841 conn(0x55cac76bbee0 msgr2=0x55cac76bc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.288+0000 7f287b5cd740 1 -- 172.21.15.16:0/1704841 shutdown_connections 2024-03-27T06:44:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.288+0000 7f287b5cd740 1 -- 172.21.15.16:0/1704841 wait complete. 2024-03-27T06:44:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.288+0000 7f287b5cd740 1 -- start start 2024-03-27T06:44:45.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.288+0000 7f287b5cd740 1 Processor -- start 2024-03-27T06:44:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76d34c0 0x55cac76d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76d04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:60558/0 (socket says 172.21.15.16:60558) 2024-03-27T06:44:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f2871d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287259c640 1 -- 172.21.15.16:0/3229435121 learned_addr learned my addr 172.21.15.16:0/3229435121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287b5cd740 1 -- 172.21.15.16:0/3229435121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cac76d6280 con 0x55cac76c1f70 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287b5cd740 1 -- 172.21.15.16:0/3229435121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55cac75e0eb0 con 0x55cac76c15c0 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287b5cd740 1 -- 172.21.15.16:0/3229435121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cac76d6400 con 0x55cac76d34c0 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f2872d9d640 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76d34c0 0x55cac76d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287b5cd740 10 client.0 _ll_register_callbacks cb 0x55cac761d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287259c640 1 -- 172.21.15.16:0/3229435121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76d34c0 msgr2=0x55cac76d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287259c640 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76d34c0 0x55cac76d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287259c640 1 -- 172.21.15.16:0/3229435121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 msgr2=0x55cac76d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287259c640 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f287259c640 1 -- 172.21.15.16:0/3229435121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac74037d0 con 0x55cac76c15c0 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.289+0000 7f2871d9b640 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.290+0000 7f287259c640 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76d04e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f286401e740 tx=0x7f286401e530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f40347feb6ce1925 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.290+0000 7f2859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.290+0000 7f2859ffb640 1 -- 172.21.15.16:0/3229435121 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864036060 con 0x55cac76c15c0 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.290+0000 7f2859ffb640 1 -- 172.21.15.16:0/3229435121 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f286401e8a0 con 0x55cac76c15c0 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.290+0000 7f287b5cd740 1 -- 172.21.15.16:0/3229435121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2864004ca0 con 0x55cac76c15c0 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.290+0000 7f2859ffb640 1 -- 172.21.15.16:0/3229435121 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2864004ad0 con 0x55cac76c15c0 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.290+0000 7f287b5cd740 10 client.25772 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.290+0000 7f287b5cd740 1 -- 172.21.15.16:0/3229435121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cac76e2500 con 0x55cac76c15c0 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.290+0000 7f2859ffb640 1 -- 172.21.15.16:0/3229435121 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2864004ca0 con 0x55cac76c15c0 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.290+0000 7f2859ffb640 1 -- 172.21.15.16:0/3229435121 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f286403b510 con 0x55cac76c15c0 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.291+0000 7f287b5cd740 10 client.25772 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.291+0000 7f287b5cd740 10 client.25772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.291+0000 7f287b5cd740 20 client.25772 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.291+0000 7f287b5cd740 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55cac76e3960 0x55cac76e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.291+0000 7f287b5cd740 4 client.25772 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.291+0000 7f287b5cd740 1 -- 172.21.15.16:0/3229435121 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cac7632c50 con 0x55cac76e3960 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.291+0000 7f2871d9b640 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55cac76e3960 0x55cac76e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.291+0000 7f2871d9b640 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55cac76e3960 0x55cac76e5d50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46533972eaadaa6 server_cookie=4d69f94c9b358b9e in_seq=0 out_seq=0 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.291+0000 7f2859ffb640 10 client.25772 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f2859ffb640 1 -- 172.21.15.16:0/3229435121 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cac7632c50 con 0x55cac76e3960 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f2859ffb640 10 client.25772 handle_command_reply: tid=0 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 1 client.25772 shutdown 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 1 -- 172.21.15.16:0/3229435121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 msgr2=0x55cac76d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76d04e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f286401e740 tx=0x7f286401e530 comp rx=0 tx=0).stop 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 1 -- 172.21.15.16:0/3229435121 shutdown_connections 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55cac76e3960 0x55cac76e5d50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76d34c0 0x55cac76d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f2859ffb640 0 client.25772 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 1 --2- 172.21.15.16:0/3229435121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76c15c0 0x55cac76d04e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 1 -- 172.21.15.16:0/3229435121 >> 172.21.15.16:0/3229435121 conn(0x55cac76bbee0 msgr2=0x55cac76bdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 1 -- 172.21.15.16:0/3229435121 shutdown_connections 2024-03-27T06:44:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 1 -- 172.21.15.16:0/3229435121 wait complete. 2024-03-27T06:44:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.292+0000 7f287b5cd740 20 client.25772 trim_cache size 0 max 16384 2024-03-27T06:44:45.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.319+0000 7f287b5cd740 1 Processor -- start 2024-03-27T06:44:45.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.319+0000 7f287b5cd740 1 -- start start 2024-03-27T06:44:45.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.319+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 0x55cac76d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.319+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76df060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.319+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76d34c0 0x55cac76eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.319+0000 7f287b5cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28741ab6d0 con 0x55cac76c1f70 2024-03-27T06:44:45.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.319+0000 7f287b5cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f28741ad210 con 0x55cac76d34c0 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.319+0000 7f287b5cd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28741abda0 con 0x55cac76c15c0 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f287259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 0x55cac76d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f2871d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76df060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f287259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 0x55cac76d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:41594/0 (socket says 172.21.15.16:41594) 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f287259c640 1 -- 172.21.15.16:0/1240234489 learned_addr learned my addr 172.21.15.16:0/1240234489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f2872d9d640 1 --2- 172.21.15.16:0/1240234489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76d34c0 0x55cac76eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f287259c640 1 -- 172.21.15.16:0/1240234489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76d34c0 msgr2=0x55cac76eb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f287259c640 1 --2- 172.21.15.16:0/1240234489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76d34c0 0x55cac76eb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f287259c640 1 -- 172.21.15.16:0/1240234489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 msgr2=0x55cac76df060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f287259c640 1 --2- 172.21.15.16:0/1240234489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76df060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f287259c640 1 -- 172.21.15.16:0/1240234489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac76cea90 con 0x55cac76c15c0 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f2872d9d640 1 --2- 172.21.15.16:0/1240234489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76d34c0 0x55cac76eb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:45.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f2871d9b640 1 --2- 172.21.15.16:0/1240234489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76df060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:45.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f287259c640 1 --2- 172.21.15.16:0/1240234489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 0x55cac76d2c10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f286403b960 tx=0x7f2864010f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d76f246425c29b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f285bfff640 1 -- 172.21.15.16:0/1240234489 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864036070 con 0x55cac76c15c0 2024-03-27T06:44:45.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f285bfff640 1 -- 172.21.15.16:0/1240234489 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f286401ec70 con 0x55cac76c15c0 2024-03-27T06:44:45.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.320+0000 7f285bfff640 1 -- 172.21.15.16:0/1240234489 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864004c70 con 0x55cac76c15c0 2024-03-27T06:44:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.321+0000 7f287b5cd740 1 -- 172.21.15.16:0/1240234489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 msgr2=0x55cac76d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.321+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1240234489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 0x55cac76d2c10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f286403b960 tx=0x7f2864010f30 comp rx=0 tx=0).stop 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.321+0000 7f287b5cd740 1 -- 172.21.15.16:0/1240234489 shutdown_connections 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.321+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1240234489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76d34c0 0x55cac76eb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.321+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1240234489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76df060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.321+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1240234489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 0x55cac76d2c10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.321+0000 7f287b5cd740 1 -- 172.21.15.16:0/1240234489 >> 172.21.15.16:0/1240234489 conn(0x55cac76bbee0 msgr2=0x55cac76b6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.321+0000 7f287b5cd740 1 -- 172.21.15.16:0/1240234489 shutdown_connections 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.321+0000 7f287b5cd740 1 -- 172.21.15.16:0/1240234489 wait complete. 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f287b5cd740 1 -- start start 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f287b5cd740 1 Processor -- start 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 0x55cac76db960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f287b5cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76d34c0 0x55cac76cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f287b5cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28741acb40 con 0x55cac76c1f70 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f287b5cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f28741ac470 con 0x55cac76d34c0 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f287b5cd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28741a96d0 con 0x55cac76c15c0 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f2871d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f2871d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34828/0 (socket says 172.21.15.16:34828) 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f2871d9b640 1 -- 172.21.15.16:0/1916306365 learned_addr learned my addr 172.21.15.16:0/1916306365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f287b5cd740 10 client.0 _ll_register_callbacks cb 0x7f286c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f2872d9d640 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76d34c0 0x55cac76cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.322+0000 7f287259c640 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 0x55cac76db960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.323+0000 7f2871d9b640 1 -- 172.21.15.16:0/1916306365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 msgr2=0x55cac76db960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.323+0000 7f2871d9b640 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 0x55cac76db960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.323+0000 7f2871d9b640 1 -- 172.21.15.16:0/1916306365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76d34c0 msgr2=0x55cac76cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.323+0000 7f2871d9b640 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76d34c0 0x55cac76cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.323+0000 7f2871d9b640 1 -- 172.21.15.16:0/1916306365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac76cdba0 con 0x55cac76c1f70 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.323+0000 7f2871d9b640 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76cce30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f28680ba6e0 tx=0x7f28680de040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcb42ad6be162ff server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.323+0000 7f2859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.323+0000 7f2859ffb640 1 -- 172.21.15.16:0/1916306365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28680c20c0 con 0x55cac76c1f70 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.323+0000 7f287259c640 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 0x55cac76db960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.323+0000 7f2859ffb640 1 -- 172.21.15.16:0/1916306365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f28680de0b0 con 0x55cac76c1f70 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.323+0000 7f287b5cd740 1 -- 172.21.15.16:0/1916306365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f286401ec70 con 0x55cac76c1f70 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.324+0000 7f2859ffb640 1 -- 172.21.15.16:0/1916306365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28680c6980 con 0x55cac76c1f70 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.324+0000 7f287b5cd740 10 client.15942 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.324+0000 7f287b5cd740 1 -- 172.21.15.16:0/1916306365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cac7749b30 con 0x55cac76c1f70 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.324+0000 7f2859ffb640 1 -- 172.21.15.16:0/1916306365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f286401ec70 con 0x55cac76c1f70 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.324+0000 7f2859ffb640 1 -- 172.21.15.16:0/1916306365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f28680dcce0 con 0x55cac76c1f70 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.324+0000 7f287b5cd740 10 client.15942 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.324+0000 7f287b5cd740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.324+0000 7f287b5cd740 20 client.15942 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.324+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55cac76e3960 0x55cac774c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.324+0000 7f287b5cd740 4 client.15942 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.324+0000 7f287b5cd740 1 -- 172.21.15.16:0/1916306365 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cac762cfa0 con 0x55cac76e3960 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f287259c640 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55cac76e3960 0x55cac774c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f2859ffb640 10 client.15942 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f287259c640 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55cac76e3960 0x55cac774c2f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f2cc3b0ab15eedc server_cookie=8c4c95583e918ee4 in_seq=0 out_seq=0 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f2859ffb640 1 -- 172.21.15.16:0/1916306365 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cac762cfa0 con 0x55cac76e3960 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f2859ffb640 10 client.15942 handle_command_reply: tid=0 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f287b5cd740 1 client.15942 shutdown 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f287b5cd740 1 -- 172.21.15.16:0/1916306365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 msgr2=0x55cac76cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76cce30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f28680ba6e0 tx=0x7f28680de040 comp rx=0 tx=0).stop 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f287b5cd740 1 -- 172.21.15.16:0/1916306365 shutdown_connections 2024-03-27T06:44:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55cac76e3960 0x55cac774c2f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cac76d34c0 0x55cac76cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f2859ffb640 0 client.15942 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cac76c1f70 0x55cac76cce30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.325+0000 7f287b5cd740 1 --2- 172.21.15.16:0/1916306365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cac76c15c0 0x55cac76db960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287b5cd740 1 -- 172.21.15.16:0/1916306365 >> 172.21.15.16:0/1916306365 conn(0x55cac76bbee0 msgr2=0x55cac76bd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287b5cd740 1 -- 172.21.15.16:0/1916306365 shutdown_connections 2024-03-27T06:44:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287b5cd740 1 -- 172.21.15.16:0/1916306365 wait complete. 2024-03-27T06:44:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287b5cd740 20 client.15942 trim_cache size 0 max 16384 2024-03-27T06:44:45.326 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:44:45.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287890d640 1 -- 172.21.15.16:0/1690946192 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f285c0a39b0 msgr2=0x7f285c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287890d640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f285c0a39b0 0x7f285c0a5e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f286400bbe0 tx=0x7f286400b390 comp rx=0 tx=0).stop 2024-03-27T06:44:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287890d640 1 -- 172.21.15.16:0/1690946192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28741066d0 msgr2=0x7f2874100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287890d640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28741066d0 0x7f2874100f50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2868005ab0 tx=0x7f286802dd00 comp rx=0 tx=0).stop 2024-03-27T06:44:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287890d640 1 -- 172.21.15.16:0/1690946192 shutdown_connections 2024-03-27T06:44:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287890d640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f285c0a39b0 0x7f285c0a5e70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287890d640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2874107090 0x7f2874101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287890d640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28741066d0 0x7f2874100f50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287890d640 1 --2- 172.21.15.16:0/1690946192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2874069130 0x7f2874100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.326+0000 7f287890d640 1 -- 172.21.15.16:0/1690946192 >> 172.21.15.16:0/1690946192 conn(0x7f2874077b50 msgr2=0x7f28740fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.327+0000 7f287890d640 1 -- 172.21.15.16:0/1690946192 shutdown_connections 2024-03-27T06:44:45.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:45.327+0000 7f287890d640 1 -- 172.21.15.16:0/1690946192 wait complete. 2024-03-27T06:44:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:45 smithi016 ceph-mon[31908]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T06:44:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:45 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1121374244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:44:45.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:45 smithi156 ceph-mon[36261]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T06:44:45.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:45 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1121374244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:44:45.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:45 smithi170 ceph-mon[36438]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T06:44:45.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:45 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1121374244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:44:46.336 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-27T06:44:46.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:44:46.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-27T06:44:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.503+0000 7f9303119640 1 -- 172.21.15.16:0/3166066366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc069130 msgr2=0x7f92fc069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.503+0000 7f9303119640 1 --2- 172.21.15.16:0/3166066366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc069130 0x7f92fc069510 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f92f000fb10 tx=0x7f92f002fdc0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.503+0000 7f9303119640 1 -- 172.21.15.16:0/3166066366 shutdown_connections 2024-03-27T06:44:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.503+0000 7f9303119640 1 --2- 172.21.15.16:0/3166066366 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc107090 0x7f92fc10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.503+0000 7f9303119640 1 --2- 172.21.15.16:0/3166066366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92fc1066d0 0x7f92fc106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.503+0000 7f9303119640 1 --2- 172.21.15.16:0/3166066366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc069130 0x7f92fc069510 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.503+0000 7f9303119640 1 -- 172.21.15.16:0/3166066366 >> 172.21.15.16:0/3166066366 conn(0x7f92fc077b50 msgr2=0x7f92fc077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.503+0000 7f9303119640 1 -- 172.21.15.16:0/3166066366 shutdown_connections 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.503+0000 7f9303119640 1 -- 172.21.15.16:0/3166066366 wait complete. 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f9303119640 1 Processor -- start 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f9303119640 1 -- start start 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f9303119640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92fc1066d0 0x7f92fc1964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f9303119640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc107090 0x7f92fc1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f9303119640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc19ad60 0x7f92fc19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f9303119640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10fe60 con 0x7f92fc107090 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f9303119640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10fce0 con 0x7f92fc19ad60 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f9303119640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10ffe0 con 0x7f92fc1066d0 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc107090 0x7f92fc1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc107090 0x7f92fc1969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46124/0 (socket says 172.21.15.16:46124) 2024-03-27T06:44:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f92fbfff640 1 -- 172.21.15.16:0/3328313413 learned_addr learned my addr 172.21.15.16:0/3328313413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f9300eb5640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92fc1066d0 0x7f92fc1964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.504+0000 7f93016b6640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc19ad60 0x7f92fc19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f92fbfff640 1 -- 172.21.15.16:0/3328313413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92fc1066d0 msgr2=0x7f92fc1964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f92fbfff640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92fc1066d0 0x7f92fc1964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f92fbfff640 1 -- 172.21.15.16:0/3328313413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc19ad60 msgr2=0x7f92fc19b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f92fbfff640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc19ad60 0x7f92fc19b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f92fbfff640 1 -- 172.21.15.16:0/3328313413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc19b850 con 0x7f92fc107090 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f93016b6640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc19ad60 0x7f92fc19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f92fbfff640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc107090 0x7f92fc1969e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f92e800ea60 tx=0x7f92e800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f92f9ffb640 1 -- 172.21.15.16:0/3328313413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e800cde0 con 0x7f92fc107090 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f92f9ffb640 1 -- 172.21.15.16:0/3328313413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f92e80035f0 con 0x7f92fc107090 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f9303119640 1 -- 172.21.15.16:0/3328313413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc19bb40 con 0x7f92fc107090 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f92f9ffb640 1 -- 172.21.15.16:0/3328313413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8010720 con 0x7f92fc107090 2024-03-27T06:44:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.505+0000 7f9303119640 1 -- 172.21.15.16:0/3328313413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc1a3330 con 0x7f92fc107090 2024-03-27T06:44:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.508+0000 7f9305dd6740 1 Processor -- start 2024-03-27T06:44:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.508+0000 7f9305dd6740 1 -- start start 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.508+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888173e50 0x558888174230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.508+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 0x55888817ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.508+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x55888817f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.508+0000 7f9305dd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55888807ec20 con 0x558888174770 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.508+0000 7f9305dd6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5588880d7430 con 0x55888817d350 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.508+0000 7f9305dd6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55888807f900 con 0x558888173e50 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f92f9ffb640 1 -- 172.21.15.16:0/3328313413 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7f92e80037a0 con 0x7f92fc107090 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f9300eb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 0x55888817ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f93016b6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888173e50 0x558888174230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f9300eb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 0x55888817ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46140/0 (socket says 172.21.15.16:46140) 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f93016b6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888173e50 0x558888174230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:49068/0 (socket says 172.21.15.16:49068) 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f9300eb5640 1 -- 172.21.15.16:0/476448458 learned_addr learned my addr 172.21.15.16:0/476448458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f93016b6640 1 -- 172.21.15.16:0/476448458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 msgr2=0x55888817f740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T06:44:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f93016b6640 1 --2- 172.21.15.16:0/476448458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x55888817f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f93016b6640 1 -- 172.21.15.16:0/476448458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 msgr2=0x55888817ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f93016b6640 1 --2- 172.21.15.16:0/476448458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 0x55888817ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f93016b6640 1 -- 172.21.15.16:0/476448458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588880d2180 con 0x558888173e50 2024-03-27T06:44:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f9300eb5640 1 --2- 172.21.15.16:0/476448458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 0x55888817ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.509+0000 7f92f9ffb640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92ec0a3b00 0x7f92ec0a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.510+0000 7f9300eb5640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92ec0a3b00 0x7f92ec0a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.510+0000 7f93016b6640 1 --2- 172.21.15.16:0/476448458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888173e50 0x558888174230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009f10 tx=0x7f92f40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bab80608a44b2652 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.510+0000 7f92f9ffb640 1 -- 172.21.15.16:0/3328313413 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f92e80d0760 con 0x7f92fc107090 2024-03-27T06:44:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.510+0000 7f92e6ffd640 1 -- 172.21.15.16:0/476448458 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f401c680 con 0x558888173e50 2024-03-27T06:44:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.510+0000 7f92e6ffd640 1 -- 172.21.15.16:0/476448458 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f92f401cd20 con 0x558888173e50 2024-03-27T06:44:46.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.511+0000 7f9305dd6740 1 -- 172.21.15.16:0/476448458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888173e50 msgr2=0x558888174230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.511+0000 7f9305dd6740 1 --2- 172.21.15.16:0/476448458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888173e50 0x558888174230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009f10 tx=0x7f92f40107a0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.511+0000 7f9300eb5640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92ec0a3b00 0x7f92ec0a5fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0005a20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.511+0000 7f9305dd6740 1 -- 172.21.15.16:0/476448458 shutdown_connections 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.511+0000 7f9305dd6740 1 --2- 172.21.15.16:0/476448458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x55888817f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.511+0000 7f9305dd6740 1 --2- 172.21.15.16:0/476448458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 0x55888817ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.511+0000 7f9305dd6740 1 --2- 172.21.15.16:0/476448458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888173e50 0x558888174230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.511+0000 7f9305dd6740 1 -- 172.21.15.16:0/476448458 >> 172.21.15.16:0/476448458 conn(0x55888816e580 msgr2=0x55888816e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.511+0000 7f9305dd6740 1 -- 172.21.15.16:0/476448458 shutdown_connections 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.511+0000 7f9305dd6740 1 -- 172.21.15.16:0/476448458 wait complete. 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.511+0000 7f9305dd6740 1 -- start start 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9305dd6740 1 Processor -- start 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558888173e50 0x55888817ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 0x55888817a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55888817d350 0x55888817a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9305dd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558888092cc0 con 0x558888174770 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9305dd6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5588880d33e0 con 0x558888173e50 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9305dd6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5588881801e0 con 0x55888817d350 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9305dd6740 10 client.0 _ll_register_callbacks cb 0x5588880cdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9300eb5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558888173e50 0x55888817ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f93016b6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55888817d350 0x55888817a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f93016b6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55888817d350 0x55888817a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:49070/0 (socket says 172.21.15.16:49070) 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9300eb5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558888173e50 0x55888817ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:52054/0 (socket says 172.21.15.16:52054) 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f93016b6640 1 -- 172.21.15.16:0/1752279778 learned_addr learned my addr 172.21.15.16:0/1752279778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f92fbfff640 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 0x55888817a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9300eb5640 1 -- 172.21.15.16:0/1752279778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55888817d350 msgr2=0x55888817a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9300eb5640 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55888817d350 0x55888817a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.512+0000 7f9300eb5640 1 -- 172.21.15.16:0/1752279778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 msgr2=0x55888817a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f9300eb5640 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 0x55888817a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f9300eb5640 1 -- 172.21.15.16:0/1752279778 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558888099ef0 con 0x558888173e50 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f93016b6640 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55888817d350 0x55888817a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f92fbfff640 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 0x55888817a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f9300eb5640 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558888173e50 0x55888817ccc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f92f0006010 tx=0x7f92f002fdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2846236d0867b11e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f92dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f92dbfff640 1 -- 172.21.15.16:0/1752279778 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0043420 con 0x558888173e50 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f9305dd6740 1 -- 172.21.15.16:0/1752279778 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f401cd20 con 0x558888173e50 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f92dbfff640 1 -- 172.21.15.16:0/1752279778 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f92f003d040 con 0x558888173e50 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f92dbfff640 1 -- 172.21.15.16:0/1752279778 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f92f00435c0 con 0x558888173e50 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f9305dd6740 10 client.25787 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f9305dd6740 1 -- 172.21.15.16:0/1752279778 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558887eb37e0 con 0x558888173e50 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.513+0000 7f92dbfff640 1 -- 172.21.15.16:0/1752279778 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f92f401cd20 con 0x558888173e50 2024-03-27T06:44:46.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.514+0000 7f92dbfff640 1 -- 172.21.15.16:0/1752279778 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f92f0038830 con 0x558888173e50 2024-03-27T06:44:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.514+0000 7f9305dd6740 10 client.25787 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.514+0000 7f9305dd6740 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.514+0000 7f9305dd6740 20 client.25787 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.514+0000 7f9305dd6740 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x558888195ac0 0x558888197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.514+0000 7f9305dd6740 4 client.25787 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.514+0000 7f9305dd6740 1 -- 172.21.15.16:0/1752279778 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588880e4970 con 0x558888195ac0 2024-03-27T06:44:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.514+0000 7f92fbfff640 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x558888195ac0 0x558888197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f92dbfff640 10 client.25787 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:46.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f92fbfff640 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x558888195ac0 0x558888197ed0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26c6e4aefcab4d4 server_cookie=5faf57846279828c in_seq=0 out_seq=0 2024-03-27T06:44:46.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f92dbfff640 1 -- 172.21.15.16:0/1752279778 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5588880e4970 con 0x558888195ac0 2024-03-27T06:44:46.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f92dbfff640 10 client.25787 handle_command_reply: tid=0 2024-03-27T06:44:46.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f9305dd6740 1 client.25787 shutdown 2024-03-27T06:44:46.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f9305dd6740 1 -- 172.21.15.16:0/1752279778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558888173e50 msgr2=0x55888817ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f9305dd6740 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558888173e50 0x55888817ccc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f92f0006010 tx=0x7f92f002fdc0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f9305dd6740 1 -- 172.21.15.16:0/1752279778 shutdown_connections 2024-03-27T06:44:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f9305dd6740 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x558888195ac0 0x558888197ed0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f9305dd6740 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55888817d350 0x55888817a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f9305dd6740 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888174770 0x55888817a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f92dbfff640 0 client.25787 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f9305dd6740 1 --2- 172.21.15.16:0/1752279778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558888173e50 0x55888817ccc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.515+0000 7f9305dd6740 1 -- 172.21.15.16:0/1752279778 >> 172.21.15.16:0/1752279778 conn(0x55888816e580 msgr2=0x5588881702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.516+0000 7f9305dd6740 1 -- 172.21.15.16:0/1752279778 shutdown_connections 2024-03-27T06:44:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.516+0000 7f9305dd6740 1 -- 172.21.15.16:0/1752279778 wait complete. 2024-03-27T06:44:46.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.516+0000 7f9305dd6740 20 client.25787 trim_cache size 0 max 16384 2024-03-27T06:44:46.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f9305dd6740 1 Processor -- start 2024-03-27T06:44:46.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f9305dd6740 1 -- start start 2024-03-27T06:44:46.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x558888188e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888174770 0x55888819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x55888817c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f9305dd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1abc40 con 0x558888173e50 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f9305dd6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1ad780 con 0x55888817d350 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f9305dd6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1ac310 con 0x558888174770 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f9300eb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x558888188e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f9300eb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x558888188e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46160/0 (socket says 172.21.15.16:46160) 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f9300eb5640 1 -- 172.21.15.16:0/1546876826 learned_addr learned my addr 172.21.15.16:0/1546876826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f92fbfff640 1 --2- 172.21.15.16:0/1546876826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888174770 0x55888819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.534+0000 7f93016b6640 1 --2- 172.21.15.16:0/1546876826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x55888817c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.535+0000 7f9300eb5640 1 -- 172.21.15.16:0/1546876826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888174770 msgr2=0x55888819d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.535+0000 7f9300eb5640 1 --2- 172.21.15.16:0/1546876826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888174770 0x55888819d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.535+0000 7f9300eb5640 1 -- 172.21.15.16:0/1546876826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 msgr2=0x55888817c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.535+0000 7f9300eb5640 1 --2- 172.21.15.16:0/1546876826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x55888817c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.535+0000 7f9300eb5640 1 -- 172.21.15.16:0/1546876826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55888817a920 con 0x558888173e50 2024-03-27T06:44:46.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.535+0000 7f93016b6640 1 --2- 172.21.15.16:0/1546876826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x55888817c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:44:46.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.535+0000 7f9300eb5640 1 --2- 172.21.15.16:0/1546876826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x558888188e50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f92f0057040 tx=0x7f92f0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65cd9c97a4c7d1e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.535+0000 7f92e67fc640 1 -- 172.21.15.16:0/1546876826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0043420 con 0x558888173e50 2024-03-27T06:44:46.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.535+0000 7f92e67fc640 1 -- 172.21.15.16:0/1546876826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f92f0034050 con 0x558888173e50 2024-03-27T06:44:46.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.535+0000 7f92e67fc640 1 -- 172.21.15.16:0/1546876826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0043e00 con 0x558888173e50 2024-03-27T06:44:46.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.536+0000 7f9305dd6740 1 -- 172.21.15.16:0/1546876826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 msgr2=0x558888188e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.536+0000 7f9305dd6740 1 --2- 172.21.15.16:0/1546876826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x558888188e50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f92f0057040 tx=0x7f92f0005bd0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.536+0000 7f9305dd6740 1 -- 172.21.15.16:0/1546876826 shutdown_connections 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.536+0000 7f9305dd6740 1 --2- 172.21.15.16:0/1546876826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x55888817c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.536+0000 7f9305dd6740 1 --2- 172.21.15.16:0/1546876826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888174770 0x55888819d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.536+0000 7f9305dd6740 1 --2- 172.21.15.16:0/1546876826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x558888188e50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.536+0000 7f9305dd6740 1 -- 172.21.15.16:0/1546876826 >> 172.21.15.16:0/1546876826 conn(0x55888816e580 msgr2=0x558888170240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.536+0000 7f9305dd6740 1 -- 172.21.15.16:0/1546876826 shutdown_connections 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.536+0000 7f9305dd6740 1 -- 172.21.15.16:0/1546876826 wait complete. 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.536+0000 7f9305dd6740 1 -- start start 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f9305dd6740 1 Processor -- start 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x5588881813b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888174770 0x5588881818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f9305dd6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x5588881fadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f9305dd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1ad0b0 con 0x558888173e50 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f9305dd6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1ac9e0 con 0x55888817d350 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f9305dd6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1a9c20 con 0x558888174770 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f9305dd6740 10 client.0 _ll_register_callbacks cb 0x7f92f0047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f9300eb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x5588881813b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f9300eb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x5588881813b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46176/0 (socket says 172.21.15.16:46176) 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f9300eb5640 1 -- 172.21.15.16:0/2087629887 learned_addr learned my addr 172.21.15.16:0/2087629887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f92fbfff640 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888174770 0x5588881818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f93016b6640 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x5588881fadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f92fbfff640 1 -- 172.21.15.16:0/2087629887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 msgr2=0x5588881fadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f92fbfff640 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x5588881fadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.537+0000 7f92fbfff640 1 -- 172.21.15.16:0/2087629887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 msgr2=0x5588881813b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f92fbfff640 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x5588881813b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f92fbfff640 1 -- 172.21.15.16:0/2087629887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558887f13f70 con 0x558888174770 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f9300eb5640 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x5588881813b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f92fbfff640 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888174770 0x5588881818f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f92e80cdca0 tx=0x7f92e800c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec7a37f3e5f9cd3c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f92dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f92dbfff640 1 -- 172.21.15.16:0/2087629887 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e809cde0 con 0x558888174770 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f9305dd6740 1 -- 172.21.15.16:0/2087629887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f0034050 con 0x558888174770 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f92dbfff640 1 -- 172.21.15.16:0/2087629887 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f92e80bb320 con 0x558888174770 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f92dbfff640 1 -- 172.21.15.16:0/2087629887 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f92e809c6e0 con 0x558888174770 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f9305dd6740 10 client.25792 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f9305dd6740 1 -- 172.21.15.16:0/2087629887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558887dab170 con 0x558888174770 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.538+0000 7f92dbfff640 1 -- 172.21.15.16:0/2087629887 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f92e80a0a90 con 0x558888174770 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.539+0000 7f92dbfff640 1 -- 172.21.15.16:0/2087629887 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f92e8098430 con 0x558888174770 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.539+0000 7f9305dd6740 10 client.25792 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.539+0000 7f9305dd6740 10 client.25792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:46.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.539+0000 7f9305dd6740 20 client.25792 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:46.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.539+0000 7f9305dd6740 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5588881fc8e0 0x5588881fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.539+0000 7f9305dd6740 4 client.25792 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-27T06:44:46.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.539+0000 7f9305dd6740 1 -- 172.21.15.16:0/2087629887 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f92f0034050 con 0x5588881fc8e0 2024-03-27T06:44:46.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.539+0000 7f9300eb5640 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5588881fc8e0 0x5588881fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.539+0000 7f92dbfff640 10 client.25792 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:46.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.539+0000 7f9300eb5640 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5588881fc8e0 0x5588881fecd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a06d292c3bad650 server_cookie=f3e1de85942c6f63 in_seq=0 out_seq=0 2024-03-27T06:44:46.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f92dbfff640 1 -- 172.21.15.16:0/2087629887 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f92f0034050 con 0x5588881fc8e0 2024-03-27T06:44:46.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f92dbfff640 10 client.25792 handle_command_reply: tid=0 2024-03-27T06:44:46.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 1 client.25792 shutdown 2024-03-27T06:44:46.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 1 -- 172.21.15.16:0/2087629887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888174770 msgr2=0x5588881818f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888174770 0x5588881818f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f92e80cdca0 tx=0x7f92e800c890 comp rx=0 tx=0).stop 2024-03-27T06:44:46.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 1 -- 172.21.15.16:0/2087629887 shutdown_connections 2024-03-27T06:44:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5588881fc8e0 0x5588881fecd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f92dbfff640 0 client.25792 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55888817d350 0x5588881fadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558888174770 0x5588881818f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 1 --2- 172.21.15.16:0/2087629887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558888173e50 0x5588881813b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 1 -- 172.21.15.16:0/2087629887 >> 172.21.15.16:0/2087629887 conn(0x55888816e580 msgr2=0x5588881787f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 1 -- 172.21.15.16:0/2087629887 shutdown_connections 2024-03-27T06:44:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 1 -- 172.21.15.16:0/2087629887 wait complete. 2024-03-27T06:44:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.541+0000 7f9305dd6740 20 client.25792 trim_cache size 0 max 16384 2024-03-27T06:44:46.540 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:44:46.540 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-03-27T06:44:46.541 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "d43d8381-4565-4e97-9809-2033946e8948", 2024-03-27T06:44:46.541 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-03-27T06:44:46.541 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:44:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 -- 172.21.15.16:0/3328313413 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92ec0a3b00 msgr2=0x7f92ec0a5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92ec0a3b00 0x7f92ec0a5fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0005a20 comp rx=0 tx=0).stop 2024-03-27T06:44:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 -- 172.21.15.16:0/3328313413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc107090 msgr2=0x7f92fc1969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc107090 0x7f92fc1969e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f92e800ea60 tx=0x7f92e800ef20 comp rx=0 tx=0).stop 2024-03-27T06:44:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 -- 172.21.15.16:0/3328313413 shutdown_connections 2024-03-27T06:44:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92fc19ad60 0x7f92fc19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc107090 0x7f92fc1969e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92ec0a3b00 0x7f92ec0a5fc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 --2- 172.21.15.16:0/3328313413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92fc1066d0 0x7f92fc1964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 -- 172.21.15.16:0/3328313413 >> 172.21.15.16:0/3328313413 conn(0x7f92fc077b50 msgr2=0x7f92fc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 -- 172.21.15.16:0/3328313413 shutdown_connections 2024-03-27T06:44:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.542+0000 7f9303119640 1 -- 172.21.15.16:0/3328313413 wait complete. 2024-03-27T06:44:46.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T06:44:46.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T06:44:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.663+0000 7fcea6655640 1 -- 172.21.15.16:0/1503749380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea0101380 msgr2=0x7fcea010d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.663+0000 7fcea6655640 1 --2- 172.21.15.16:0/1503749380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea0101380 0x7fcea010d890 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fce98009c10 tx=0x7fce9802bef0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.663+0000 7fcea6655640 1 -- 172.21.15.16:0/1503749380 shutdown_connections 2024-03-27T06:44:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.663+0000 7fcea6655640 1 --2- 172.21.15.16:0/1503749380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea0101380 0x7fcea010d890 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.663+0000 7fcea6655640 1 --2- 172.21.15.16:0/1503749380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcea01009c0 0x7fcea0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.663+0000 7fcea6655640 1 --2- 172.21.15.16:0/1503749380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea01069d0 0x7fcea0106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.663+0000 7fcea6655640 1 -- 172.21.15.16:0/1503749380 >> 172.21.15.16:0/1503749380 conn(0x7fcea0077b50 msgr2=0x7fcea0077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.663+0000 7fcea6655640 1 -- 172.21.15.16:0/1503749380 shutdown_connections 2024-03-27T06:44:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.663+0000 7fcea6655640 1 -- 172.21.15.16:0/1503749380 wait complete. 2024-03-27T06:44:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.663+0000 7fcea6655640 1 Processor -- start 2024-03-27T06:44:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fcea6655640 1 -- start start 2024-03-27T06:44:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fcea6655640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea01009c0 0x7fcea0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fcea6655640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcea0101380 0x7fcea0197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fcea6655640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea01069d0 0x7fcea019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fcea6655640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcea0110ed0 con 0x7fcea01009c0 2024-03-27T06:44:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fcea6655640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fcea0110d50 con 0x7fcea0101380 2024-03-27T06:44:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fcea6655640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcea0111050 con 0x7fcea01069d0 2024-03-27T06:44:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fce9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea01009c0 0x7fcea0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fce9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea01009c0 0x7fcea0197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46186/0 (socket says 172.21.15.16:46186) 2024-03-27T06:44:46.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fce9ffff640 1 -- 172.21.15.16:0/2715000900 learned_addr learned my addr 172.21.15.16:0/2715000900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcea0101380 0x7fcea0197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.664+0000 7fcea4bf2640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea01069d0 0x7fcea019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.665+0000 7fce9ffff640 1 -- 172.21.15.16:0/2715000900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea01069d0 msgr2=0x7fcea019bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.665+0000 7fce9ffff640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea01069d0 0x7fcea019bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.665+0000 7fce9ffff640 1 -- 172.21.15.16:0/2715000900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcea0101380 msgr2=0x7fcea0197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.665+0000 7fce9ffff640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcea0101380 0x7fcea0197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.665+0000 7fce9ffff640 1 -- 172.21.15.16:0/2715000900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea019c340 con 0x7fcea01009c0 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.665+0000 7fce9ffff640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea01009c0 0x7fcea0197350 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fce9000e9f0 tx=0x7fce9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.665+0000 7fce9d7fa640 1 -- 172.21.15.16:0/2715000900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9000cd80 con 0x7fcea01009c0 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.665+0000 7fcea6655640 1 -- 172.21.15.16:0/2715000900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea019c630 con 0x7fcea01009c0 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.665+0000 7fce9d7fa640 1 -- 172.21.15.16:0/2715000900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fce90010930 con 0x7fcea01009c0 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.665+0000 7fcea6655640 1 -- 172.21.15.16:0/2715000900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea01a3ec0 con 0x7fcea01009c0 2024-03-27T06:44:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.665+0000 7fce9d7fa640 1 -- 172.21.15.16:0/2715000900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90018b00 con 0x7fcea01009c0 2024-03-27T06:44:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.668+0000 7fce9d7fa640 1 -- 172.21.15.16:0/2715000900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7fce90018ca0 con 0x7fcea01009c0 2024-03-27T06:44:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.668+0000 7fce9d7fa640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fce880a39b0 0x7fce880a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.668+0000 7fce9d7fa640 1 -- 172.21.15.16:0/2715000900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fce900d0670 con 0x7fcea01009c0 2024-03-27T06:44:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.668+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fce880a39b0 0x7fce880a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fcea9240740 1 Processor -- start 2024-03-27T06:44:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fce880a39b0 0x7fce880a5e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcea0104fc0 tx=0x7fce94009260 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fcea9240740 1 -- start start 2024-03-27T06:44:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fcea9240740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807226630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fcea9240740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x55780722f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fcea9240740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x557807231b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fcea9240740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55780718a860 con 0x557807226250 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fcea9240740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5578071328d0 con 0x557807226b70 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fcea9240740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557807186810 con 0x55780722f750 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fce9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807226630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fce9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807226630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46198/0 (socket says 172.21.15.16:46198) 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.669+0000 7fce9ffff640 1 -- 172.21.15.16:0/4013334446 learned_addr learned my addr 172.21.15.16:0/4013334446 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fcea4bf2640 1 --2- 172.21.15.16:0/4013334446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x557807231b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fce9ffff640 1 -- 172.21.15.16:0/4013334446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 msgr2=0x557807231b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fce9ffff640 1 --2- 172.21.15.16:0/4013334446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x557807231b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/4013334446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x55780722f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fce9ffff640 1 -- 172.21.15.16:0/4013334446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 msgr2=0x55780722f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fce9ffff640 1 --2- 172.21.15.16:0/4013334446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x55780722f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fce9ffff640 1 -- 172.21.15.16:0/4013334446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55780713d120 con 0x557807226250 2024-03-27T06:44:46.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/4013334446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x55780722f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:44:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fce9ffff640 1 --2- 172.21.15.16:0/4013334446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807226630 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fce9000e9c0 tx=0x7fce9009c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ca8d2610ca09b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fce867fc640 1 -- 172.21.15.16:0/4013334446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce900a5260 con 0x557807226250 2024-03-27T06:44:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fce867fc640 1 -- 172.21.15.16:0/4013334446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fce90098d10 con 0x557807226250 2024-03-27T06:44:46.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.670+0000 7fce867fc640 1 -- 172.21.15.16:0/4013334446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9009baf0 con 0x557807226250 2024-03-27T06:44:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.671+0000 7fcea9240740 1 -- 172.21.15.16:0/4013334446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 msgr2=0x557807226630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.671+0000 7fcea9240740 1 --2- 172.21.15.16:0/4013334446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807226630 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fce9000e9c0 tx=0x7fce9009c0c0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.671+0000 7fcea9240740 1 -- 172.21.15.16:0/4013334446 shutdown_connections 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.671+0000 7fcea9240740 1 --2- 172.21.15.16:0/4013334446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x557807231b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.671+0000 7fcea9240740 1 --2- 172.21.15.16:0/4013334446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x55780722f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.671+0000 7fcea9240740 1 --2- 172.21.15.16:0/4013334446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807226630 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.671+0000 7fcea9240740 1 -- 172.21.15.16:0/4013334446 >> 172.21.15.16:0/4013334446 conn(0x557807220980 msgr2=0x557807220d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.671+0000 7fcea9240740 1 -- 172.21.15.16:0/4013334446 shutdown_connections 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.671+0000 7fcea9240740 1 -- 172.21.15.16:0/4013334446 wait complete. 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.672+0000 7fcea9240740 1 -- start start 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.672+0000 7fcea9240740 1 Processor -- start 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.672+0000 7fcea9240740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x55780722c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fcea9240740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557807226b70 0x55780722cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fce9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x55780722c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fce9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x55780722c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46214/0 (socket says 172.21.15.16:46214) 2024-03-27T06:44:46.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fce9ffff640 1 -- 172.21.15.16:0/169632562 learned_addr learned my addr 172.21.15.16:0/169632562 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fcea9240740 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55780722f750 0x55780722d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fcea9240740 1 -- 172.21.15.16:0/169632562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5578072324e0 con 0x557807226250 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557807226b70 0x55780722cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fcea9240740 1 -- 172.21.15.16:0/169632562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x557807145cd0 con 0x55780722f750 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fcea9240740 1 -- 172.21.15.16:0/169632562 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557807232660 con 0x557807226b70 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fcea4bf2640 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55780722f750 0x55780722d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fce9ffff640 1 -- 172.21.15.16:0/169632562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557807226b70 msgr2=0x55780722cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fce9ffff640 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557807226b70 0x55780722cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fce9ffff640 1 -- 172.21.15.16:0/169632562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55780722f750 msgr2=0x55780722d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fce9ffff640 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55780722f750 0x55780722d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.673+0000 7fce9ffff640 1 -- 172.21.15.16:0/169632562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557806f687d0 con 0x557807226250 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fce9ffff640 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x55780722c720 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fce900cdcf0 tx=0x7fce90087170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e641026fd6d448 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fce7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fcea9240740 10 client.0 _ll_register_callbacks cb 0x557807137ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:46.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fce7f7fe640 1 -- 172.21.15.16:0/169632562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce900a5260 con 0x557807226250 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fce7f7fe640 1 -- 172.21.15.16:0/169632562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fce900bb2f0 con 0x557807226250 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fcea9240740 1 -- 172.21.15.16:0/169632562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce90098d10 con 0x557807226250 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fce7f7fe640 1 -- 172.21.15.16:0/169632562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce9009baf0 con 0x557807226250 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fcea9240740 10 client.15972 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fcea9240740 1 -- 172.21.15.16:0/169632562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) -- 0x557806e9e3f0 con 0x557807226250 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fce7f7fe640 1 -- 172.21.15.16:0/169632562 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fce9009bc90 con 0x557807226250 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fcea9240740 10 client.15972 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.674+0000 7fcea9240740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.675+0000 7fcea9240740 20 client.15972 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.675+0000 7fcea9240740 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55780723f400 0x5578072417f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.675+0000 7fcea9240740 4 client.15972 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.675+0000 7fcea9240740 1 -- 172.21.15.16:0/169632562 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fce90098d10 con 0x55780723f400 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.675+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55780723f400 0x5578072417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.675+0000 7fce7f7fe640 10 client.15972 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.675+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55780723f400 0x5578072417f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24df38046230e646 server_cookie=a99f64f898c11483 in_seq=0 out_seq=0 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.676+0000 7fce7f7fe640 1 -- 172.21.15.16:0/169632562 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fce90098d10 con 0x55780723f400 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.676+0000 7fce7f7fe640 10 client.15972 handle_command_reply: tid=0 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.676+0000 7fcea9240740 1 client.15972 shutdown 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.676+0000 7fcea9240740 1 -- 172.21.15.16:0/169632562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 msgr2=0x55780722c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.676+0000 7fcea9240740 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x55780722c720 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fce900cdcf0 tx=0x7fce90087170 comp rx=0 tx=0).stop 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.676+0000 7fcea9240740 1 -- 172.21.15.16:0/169632562 shutdown_connections 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.676+0000 7fcea9240740 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55780723f400 0x5578072417f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.677+0000 7fcea9240740 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55780722f750 0x55780722d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.677+0000 7fcea9240740 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557807226b70 0x55780722cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.677+0000 7fcea9240740 1 --2- 172.21.15.16:0/169632562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x55780722c720 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.677+0000 7fcea9240740 1 -- 172.21.15.16:0/169632562 >> 172.21.15.16:0/169632562 conn(0x557807220980 msgr2=0x5578072226d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.677+0000 7fce7f7fe640 0 client.15972 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.677+0000 7fcea9240740 1 -- 172.21.15.16:0/169632562 shutdown_connections 2024-03-27T06:44:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.677+0000 7fcea9240740 1 -- 172.21.15.16:0/169632562 wait complete. 2024-03-27T06:44:46.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.677+0000 7fcea9240740 20 client.15972 trim_cache size 0 max 16384 2024-03-27T06:44:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.712+0000 7fcea9240740 1 Processor -- start 2024-03-27T06:44:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.712+0000 7fcea9240740 1 -- start start 2024-03-27T06:44:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fcea9240740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x55780722b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fcea9240740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807241410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fcea9240740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x55780723b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fcea9240740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcea01ab640 con 0x557807226250 2024-03-27T06:44:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fcea9240740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fcea01ad180 con 0x557807226b70 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fcea9240740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcea01abd10 con 0x55780722f750 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fce9f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807241410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fce9f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807241410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:52106/0 (socket says 172.21.15.16:52106) 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fce9f7fe640 1 -- 172.21.15.16:0/2429119731 learned_addr learned my addr 172.21.15.16:0/2429119731 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fcea4bf2640 1 --2- 172.21.15.16:0/2429119731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x55780723b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fce9ffff640 1 --2- 172.21.15.16:0/2429119731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x55780722b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fce9f7fe640 1 -- 172.21.15.16:0/2429119731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 msgr2=0x55780723b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/2429119731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x55780723b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fce9f7fe640 1 -- 172.21.15.16:0/2429119731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 msgr2=0x55780722b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/2429119731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x55780722b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fce9f7fe640 1 -- 172.21.15.16:0/2429119731 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578072269a0 con 0x557807226b70 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.713+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/2429119731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807241410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fce94009530 tx=0x7fce94014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c848d39f295ae6e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.714+0000 7fce85ffb640 1 -- 172.21.15.16:0/2429119731 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94028430 con 0x557807226b70 2024-03-27T06:44:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.714+0000 7fce85ffb640 1 -- 172.21.15.16:0/2429119731 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fce94005520 con 0x557807226b70 2024-03-27T06:44:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.714+0000 7fce85ffb640 1 -- 172.21.15.16:0/2429119731 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94004da0 con 0x557807226b70 2024-03-27T06:44:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.715+0000 7fcea9240740 1 -- 172.21.15.16:0/2429119731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 msgr2=0x557807241410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.715+0000 7fcea9240740 1 --2- 172.21.15.16:0/2429119731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807241410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fce94009530 tx=0x7fce94014000 comp rx=0 tx=0).stop 2024-03-27T06:44:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.715+0000 7fcea9240740 1 -- 172.21.15.16:0/2429119731 shutdown_connections 2024-03-27T06:44:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.715+0000 7fcea9240740 1 --2- 172.21.15.16:0/2429119731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x55780723b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.715+0000 7fcea9240740 1 --2- 172.21.15.16:0/2429119731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807241410 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.715+0000 7fcea9240740 1 --2- 172.21.15.16:0/2429119731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x55780722b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.715+0000 7fcea9240740 1 -- 172.21.15.16:0/2429119731 >> 172.21.15.16:0/2429119731 conn(0x557807220980 msgr2=0x55780721f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.715+0000 7fcea9240740 1 -- 172.21.15.16:0/2429119731 shutdown_connections 2024-03-27T06:44:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.715+0000 7fcea9240740 1 -- 172.21.15.16:0/2429119731 wait complete. 2024-03-27T06:44:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.715+0000 7fcea9240740 1 -- start start 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.715+0000 7fcea9240740 1 Processor -- start 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fcea9240740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807237cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fcea9240740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807233040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fce9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807237cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fce9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807237cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46232/0 (socket says 172.21.15.16:46232) 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fce9f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807233040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fce9f7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807233040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:52108/0 (socket says 172.21.15.16:52108) 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fcea9240740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x557807233780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fce9ffff640 1 -- 172.21.15.16:0/3084890808 learned_addr learned my addr 172.21.15.16:0/3084890808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fcea9240740 1 -- 172.21.15.16:0/3084890808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcea01acab0 con 0x557807226250 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fcea9240740 1 -- 172.21.15.16:0/3084890808 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fcea01ac3e0 con 0x557807226b70 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fcea9240740 1 -- 172.21.15.16:0/3084890808 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcea01a9620 con 0x55780722f750 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fcea4bf2640 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x557807233780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fcea9240740 10 client.0 _ll_register_callbacks cb 0x7fce94005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:46.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fce9ffff640 1 -- 172.21.15.16:0/3084890808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 msgr2=0x557807233780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fce9ffff640 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x557807233780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fce9ffff640 1 -- 172.21.15.16:0/3084890808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 msgr2=0x557807233040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fce9ffff640 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807233040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.716+0000 7fce9ffff640 1 -- 172.21.15.16:0/3084890808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557806e5e0f0 con 0x557807226250 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.717+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807233040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.717+0000 7fce9ffff640 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807237cd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fce90098ab0 tx=0x7fce900b9f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ae9242c350a46a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.717+0000 7fce7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.717+0000 7fce7f7fe640 1 -- 172.21.15.16:0/3084890808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce900a5260 con 0x557807226250 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.717+0000 7fce7f7fe640 1 -- 172.21.15.16:0/3084890808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fce9009a9a0 con 0x557807226250 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.717+0000 7fcea9240740 1 -- 172.21.15.16:0/3084890808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce94005520 con 0x557807226250 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.717+0000 7fce7f7fe640 1 -- 172.21.15.16:0/3084890808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce9008f050 con 0x557807226250 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.717+0000 7fcea9240740 10 client.15978 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.717+0000 7fcea9240740 1 -- 172.21.15.16:0/3084890808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578072a54f0 con 0x557807226250 2024-03-27T06:44:46.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.717+0000 7fce7f7fe640 1 -- 172.21.15.16:0/3084890808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fce90087090 con 0x557807226250 2024-03-27T06:44:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.718+0000 7fce7f7fe640 1 -- 172.21.15.16:0/3084890808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fce900cdec0 con 0x557807226250 2024-03-27T06:44:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.718+0000 7fcea9240740 10 client.15978 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.718+0000 7fcea9240740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.718+0000 7fcea9240740 20 client.15978 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.718+0000 7fcea9240740 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5578072a5a20 0x5578072a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.718+0000 7fcea9240740 4 client.15978 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T06:44:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.718+0000 7fcea9240740 1 -- 172.21.15.16:0/3084890808 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fce94005520 con 0x5578072a5a20 2024-03-27T06:44:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.718+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5578072a5a20 0x5578072a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.718+0000 7fce7f7fe640 10 client.15978 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:46.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.719+0000 7fce9f7fe640 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5578072a5a20 0x5578072a7e10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed1a82a91a8aca3 server_cookie=40a0b2b4e9e74e90 in_seq=0 out_seq=0 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.719+0000 7fce7f7fe640 1 -- 172.21.15.16:0/3084890808 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fce94005520 con 0x5578072a5a20 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.719+0000 7fce7f7fe640 10 client.15978 handle_command_reply: tid=0 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.719+0000 7fcea9240740 1 client.15978 shutdown 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.719+0000 7fcea9240740 1 -- 172.21.15.16:0/3084890808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 msgr2=0x557807237cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.719+0000 7fcea9240740 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807237cd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fce90098ab0 tx=0x7fce900b9f90 comp rx=0 tx=0).stop 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea9240740 1 -- 172.21.15.16:0/3084890808 shutdown_connections 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea9240740 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5578072a5a20 0x5578072a7e10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea9240740 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55780722f750 0x557807233780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea9240740 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557807226b70 0x557807233040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea9240740 1 --2- 172.21.15.16:0/3084890808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557807226250 0x557807237cd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea9240740 1 -- 172.21.15.16:0/3084890808 >> 172.21.15.16:0/3084890808 conn(0x557807220980 msgr2=0x557807221d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fce7f7fe640 0 client.15978 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea9240740 1 -- 172.21.15.16:0/3084890808 shutdown_connections 2024-03-27T06:44:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea9240740 1 -- 172.21.15.16:0/3084890808 wait complete. 2024-03-27T06:44:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea9240740 20 client.15978 trim_cache size 0 max 16384 2024-03-27T06:44:46.719 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:44:46.719 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-27T06:44:46.719 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-03-27T06:44:46.719 INFO:teuthology.orchestra.run.smithi016.stdout: "d43d8381-4565-4e97-9809-2033946e8948": { 2024-03-27T06:44:46.719 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-03-27T06:44:46.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "d43d8381-4565-4e97-9809-2033946e8948", 2024-03-27T06:44:46.719 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-03-27T06:44:46.719 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:44:46.720 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:44:46.720 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:44:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea6655640 1 -- 172.21.15.16:0/2715000900 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fce880a39b0 msgr2=0x7fce880a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea6655640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fce880a39b0 0x7fce880a5e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcea0104fc0 tx=0x7fce94009260 comp rx=0 tx=0).stop 2024-03-27T06:44:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea6655640 1 -- 172.21.15.16:0/2715000900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea01009c0 msgr2=0x7fcea0197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.720+0000 7fcea6655640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea01009c0 0x7fcea0197350 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fce9000e9f0 tx=0x7fce9000eeb0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.721+0000 7fcea6655640 1 -- 172.21.15.16:0/2715000900 shutdown_connections 2024-03-27T06:44:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.721+0000 7fcea6655640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fce880a39b0 0x7fce880a5e70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.721+0000 7fcea6655640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea01069d0 0x7fcea019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.721+0000 7fcea6655640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcea0101380 0x7fcea0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.721+0000 7fcea6655640 1 --2- 172.21.15.16:0/2715000900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea01009c0 0x7fcea0197350 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.721+0000 7fcea6655640 1 -- 172.21.15.16:0/2715000900 >> 172.21.15.16:0/2715000900 conn(0x7fcea0077b50 msgr2=0x7fcea01030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.721+0000 7fcea6655640 1 -- 172.21.15.16:0/2715000900 shutdown_connections 2024-03-27T06:44:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:44:46.721+0000 7fcea6655640 1 -- 172.21.15.16:0/2715000900 wait complete. 2024-03-27T06:44:46.730 INFO:tasks.cephfs.filesystem:scrub status for tag:d43d8381-4565-4e97-9809-2033946e8948 - {'path': '/', 'tag': 'd43d8381-4565-4e97-9809-2033946e8948', 'options': 'recursive,force'} 2024-03-27T06:44:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:47 smithi156 ceph-mon[36261]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T06:44:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:47 smithi156 ceph-mon[36261]: scrub queued for path: / 2024-03-27T06:44:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:47 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:47.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:47 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:44:47.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:47 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:47 smithi170 ceph-mon[36438]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T06:44:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:47 smithi170 ceph-mon[36438]: scrub queued for path: / 2024-03-27T06:44:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:47 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:47 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:44:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:47 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:47 smithi016 ceph-mon[31908]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-27T06:44:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:47 smithi016 ceph-mon[31908]: scrub queued for path: / 2024-03-27T06:44:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:47 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:47 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:44:47.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:47 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.376+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/1243042669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0101b40 msgr2=0x7fd2f0101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.376+0000 7fd2f7b3e640 1 --2- 172.21.15.16:0/1243042669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0101b40 0x7fd2f0101fc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009900 tx=0x7fd2e002bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.376+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/1243042669 shutdown_connections 2024-03-27T06:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.376+0000 7fd2f7b3e640 1 --2- 172.21.15.16:0/1243042669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f0102500 0x7fd2f010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.376+0000 7fd2f7b3e640 1 --2- 172.21.15.16:0/1243042669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0101b40 0x7fd2f0101fc0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.376+0000 7fd2f7b3e640 1 --2- 172.21.15.16:0/1243042669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2f0107c00 0x7fd2f0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.376+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/1243042669 >> 172.21.15.16:0/1243042669 conn(0x7fd2f00fd650 msgr2=0x7fd2f00ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.376+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/1243042669 shutdown_connections 2024-03-27T06:44:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.376+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/1243042669 wait complete. 2024-03-27T06:44:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.377+0000 7fd2f7b3e640 1 Processor -- start 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.377+0000 7fd2f7b3e640 1 -- start start 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.377+0000 7fd2f7b3e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2f0101b40 0x7fd2f0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.377+0000 7fd2f7b3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0102500 0x7fd2f01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.377+0000 7fd2f7b3e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f0107c00 0x7fd2f019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.377+0000 7fd2f7b3e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0112060 con 0x7fd2f0102500 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.377+0000 7fd2f7b3e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0111ee0 con 0x7fd2f0101b40 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.377+0000 7fd2f7b3e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01121e0 con 0x7fd2f0107c00 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.377+0000 7fd2f58da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2f0101b40 0x7fd2f0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f50d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0102500 0x7fd2f01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f58da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2f0101b40 0x7fd2f0198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:52136/0 (socket says 172.21.15.16:52136) 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f50d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0102500 0x7fd2f01988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46252/0 (socket says 172.21.15.16:46252) 2024-03-27T06:44:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f58da640 1 -- 172.21.15.16:0/3412458589 learned_addr learned my addr 172.21.15.16:0/3412458589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f60db640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f0107c00 0x7fd2f019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f50d9640 1 -- 172.21.15.16:0/3412458589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f0107c00 msgr2=0x7fd2f019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f0107c00 0x7fd2f019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f50d9640 1 -- 172.21.15.16:0/3412458589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2f0101b40 msgr2=0x7fd2f0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2f0101b40 0x7fd2f0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f50d9640 1 -- 172.21.15.16:0/3412458589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f019d370 con 0x7fd2f0102500 2024-03-27T06:44:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f58da640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2f0101b40 0x7fd2f0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f60db640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f0107c00 0x7fd2f019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.378+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0102500 0x7fd2f01988c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0036040 tx=0x7fd2e002dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.379+0000 7fd2e6ffd640 1 -- 172.21.15.16:0/3412458589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0043020 con 0x7fd2f0102500 2024-03-27T06:44:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.379+0000 7fd2e6ffd640 1 -- 172.21.15.16:0/3412458589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd2e0004e50 con 0x7fd2f0102500 2024-03-27T06:44:48.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.379+0000 7fd2e6ffd640 1 -- 172.21.15.16:0/3412458589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0003970 con 0x7fd2f0102500 2024-03-27T06:44:48.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.379+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/3412458589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f019d600 con 0x7fd2f0102500 2024-03-27T06:44:48.376 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.379+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/3412458589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f01a4ef0 con 0x7fd2f0102500 2024-03-27T06:44:48.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.382+0000 7fd2e6ffd640 1 -- 172.21.15.16:0/3412458589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7fd2e0046020 con 0x7fd2f0102500 2024-03-27T06:44:48.377 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.382+0000 7fd2fa803740 1 Processor -- start 2024-03-27T06:44:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2fa803740 1 -- start start 2024-03-27T06:44:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 0x5627682ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 0x5627682d32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682d37e0 0x5627682d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2fa803740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562768290cd0 con 0x5627682ca0f0 2024-03-27T06:44:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2fa803740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5627681d5950 con 0x5627682d37e0 2024-03-27T06:44:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2fa803740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56276822de50 con 0x5627682caa10 2024-03-27T06:44:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2f60db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 0x5627682ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2f60db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 0x5627682ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46264/0 (socket says 172.21.15.16:46264) 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2f60db640 1 -- 172.21.15.16:0/168814210 learned_addr learned my addr 172.21.15.16:0/168814210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2e6ffd640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd2dc0a3a50 0x7fd2dc0a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/168814210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682d37e0 0x5627682d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2f58da640 1 --2- 172.21.15.16:0/168814210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 0x5627682d32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2f60db640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd2dc0a3a50 0x7fd2dc0a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2f58da640 1 -- 172.21.15.16:0/168814210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682d37e0 msgr2=0x5627682d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2f58da640 1 --2- 172.21.15.16:0/168814210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682d37e0 0x5627682d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2f58da640 1 -- 172.21.15.16:0/168814210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 msgr2=0x5627682ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.383+0000 7fd2f58da640 1 --2- 172.21.15.16:0/168814210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 0x5627682ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.384+0000 7fd2f58da640 1 -- 172.21.15.16:0/168814210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562767f42210 con 0x5627682caa10 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.384+0000 7fd2e6ffd640 1 -- 172.21.15.16:0/3412458589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd2e00f1d20 con 0x7fd2f0102500 2024-03-27T06:44:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.384+0000 7fd2f58da640 1 --2- 172.21.15.16:0/168814210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 0x5627682d32a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0114f0 tx=0x7fd2ec0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daa6dee898511b20 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.384+0000 7fd2dbfff640 1 -- 172.21.15.16:0/168814210 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec01e070 con 0x5627682caa10 2024-03-27T06:44:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.384+0000 7fd2dbfff640 1 -- 172.21.15.16:0/168814210 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd2ec0035f0 con 0x5627682caa10 2024-03-27T06:44:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.384+0000 7fd2dbfff640 1 -- 172.21.15.16:0/168814210 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00f8f0 con 0x5627682caa10 2024-03-27T06:44:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.385+0000 7fd2fa803740 1 -- 172.21.15.16:0/168814210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 msgr2=0x5627682d32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.385+0000 7fd2f60db640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd2dc0a3a50 0x7fd2dc0a5f10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009fd0 tx=0x7fd2e80134c0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.385+0000 7fd2fa803740 1 --2- 172.21.15.16:0/168814210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 0x5627682d32a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0114f0 tx=0x7fd2ec0119b0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.385+0000 7fd2fa803740 1 -- 172.21.15.16:0/168814210 shutdown_connections 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.385+0000 7fd2fa803740 1 --2- 172.21.15.16:0/168814210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682d37e0 0x5627682d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.385+0000 7fd2fa803740 1 --2- 172.21.15.16:0/168814210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 0x5627682d32a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.385+0000 7fd2fa803740 1 --2- 172.21.15.16:0/168814210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 0x5627682ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.385+0000 7fd2fa803740 1 -- 172.21.15.16:0/168814210 >> 172.21.15.16:0/168814210 conn(0x5627682c4a10 msgr2=0x5627682c4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.385+0000 7fd2fa803740 1 -- 172.21.15.16:0/168814210 shutdown_connections 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.385+0000 7fd2fa803740 1 -- 172.21.15.16:0/168814210 wait complete. 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.386+0000 7fd2fa803740 1 -- start start 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.386+0000 7fd2fa803740 1 Processor -- start 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.387+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 0x5627682d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.387+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682caa10 0x5627682d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.387+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682d37e0 0x5627682d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.387+0000 7fd2fa803740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562768229e00 con 0x5627682caa10 2024-03-27T06:44:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.387+0000 7fd2f58da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 0x5627682d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.387+0000 7fd2f50d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682caa10 0x5627682d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.387+0000 7fd2f58da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 0x5627682d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:52158/0 (socket says 172.21.15.16:52158) 2024-03-27T06:44:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.387+0000 7fd2f58da640 1 -- 172.21.15.16:0/4256520422 learned_addr learned my addr 172.21.15.16:0/4256520422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.387+0000 7fd2fa803740 1 -- 172.21.15.16:0/4256520422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5627681d6520 con 0x5627682ca0f0 2024-03-27T06:44:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.387+0000 7fd2f60db640 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682d37e0 0x5627682d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2fa803740 1 -- 172.21.15.16:0/4256520422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5627681e98e0 con 0x5627682d37e0 2024-03-27T06:44:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2fa803740 10 client.0 _ll_register_callbacks cb 0x562768224a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2f58da640 1 -- 172.21.15.16:0/4256520422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682d37e0 msgr2=0x5627682d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2f58da640 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682d37e0 0x5627682d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2f58da640 1 -- 172.21.15.16:0/4256520422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682caa10 msgr2=0x5627682d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2f58da640 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682caa10 0x5627682d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2f58da640 1 -- 172.21.15.16:0/4256520422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56276800e620 con 0x5627682ca0f0 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2f58da640 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 0x5627682d0830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec004630 tx=0x7fd2ec0044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bceb88f7237510 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2d97fa640 1 -- 172.21.15.16:0/4256520422 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec01e070 con 0x5627682ca0f0 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2d97fa640 1 -- 172.21.15.16:0/4256520422 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd2ec0039b0 con 0x5627682ca0f0 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.388+0000 7fd2fa803740 1 -- 172.21.15.16:0/4256520422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2ec0035f0 con 0x5627682ca0f0 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2d97fa640 1 -- 172.21.15.16:0/4256520422 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd2ec031050 con 0x5627682ca0f0 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2fa803740 10 client.25820 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2fa803740 1 -- 172.21.15.16:0/4256520422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5627682e2140 con 0x5627682ca0f0 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2d97fa640 1 -- 172.21.15.16:0/4256520422 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd2ec0035f0 con 0x5627682ca0f0 2024-03-27T06:44:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2d97fa640 1 -- 172.21.15.16:0/4256520422 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd2ec019450 con 0x5627682ca0f0 2024-03-27T06:44:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2fa803740 10 client.25820 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2fa803740 10 client.25820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2fa803740 20 client.25820 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2fa803740 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5627682e34f0 0x5627682e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2fa803740 4 client.25820 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:44:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2fa803740 1 -- 172.21.15.16:0/4256520422 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56276823b390 con 0x5627682e34f0 2024-03-27T06:44:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.389+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5627682e34f0 0x5627682e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.390+0000 7fd2d97fa640 10 client.25820 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.390+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5627682e34f0 0x5627682e58e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c858a0c747cec3 server_cookie=6194ddbde281bd9d in_seq=0 out_seq=0 2024-03-27T06:44:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2d97fa640 1 -- 172.21.15.16:0/4256520422 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56276823b390 con 0x5627682e34f0 2024-03-27T06:44:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2d97fa640 10 client.25820 handle_command_reply: tid=0 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 1 client.25820 shutdown 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 1 -- 172.21.15.16:0/4256520422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 msgr2=0x5627682d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 0x5627682d0830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec004630 tx=0x7fd2ec0044f0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 1 -- 172.21.15.16:0/4256520422 shutdown_connections 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5627682e34f0 0x5627682e58e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682d37e0 0x5627682d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682caa10 0x5627682d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2d97fa640 0 client.25820 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 1 --2- 172.21.15.16:0/4256520422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 0x5627682d0830 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 1 -- 172.21.15.16:0/4256520422 >> 172.21.15.16:0/4256520422 conn(0x5627682c4a10 msgr2=0x5627682c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 1 -- 172.21.15.16:0/4256520422 shutdown_connections 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 1 -- 172.21.15.16:0/4256520422 wait complete. 2024-03-27T06:44:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.391+0000 7fd2fa803740 20 client.25820 trim_cache size 0 max 16384 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.421+0000 7fd2fa803740 1 Processor -- start 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2fa803740 1 -- start start 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 0x5627682d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 0x5627682deeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682d37e0 0x5627682eb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2fa803740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01afa20 con 0x5627682d37e0 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2fa803740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01b1560 con 0x5627682ca0f0 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2fa803740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01b00f0 con 0x5627682caa10 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2f58da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 0x5627682d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2f50d9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 0x5627682deeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2f58da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 0x5627682d2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:52170/0 (socket says 172.21.15.16:52170) 2024-03-27T06:44:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2f58da640 1 -- 172.21.15.16:0/2332922666 learned_addr learned my addr 172.21.15.16:0/2332922666 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2f60db640 1 --2- 172.21.15.16:0/2332922666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682d37e0 0x5627682eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2f50d9640 1 -- 172.21.15.16:0/2332922666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 msgr2=0x5627682d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.422+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/2332922666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 0x5627682d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.423+0000 7fd2f50d9640 1 -- 172.21.15.16:0/2332922666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682d37e0 msgr2=0x5627682eb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.423+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/2332922666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682d37e0 0x5627682eb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.423+0000 7fd2f50d9640 1 -- 172.21.15.16:0/2332922666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627682ced80 con 0x5627682caa10 2024-03-27T06:44:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.423+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/2332922666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 0x5627682deeb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00ef370 tx=0x7fd2e002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c02eccbf8062b89 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.423+0000 7fd2db7fe640 1 -- 172.21.15.16:0/2332922666 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e00d7110 con 0x5627682caa10 2024-03-27T06:44:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.423+0000 7fd2db7fe640 1 -- 172.21.15.16:0/2332922666 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd2e00be4b0 con 0x5627682caa10 2024-03-27T06:44:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.423+0000 7fd2db7fe640 1 -- 172.21.15.16:0/2332922666 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e00de240 con 0x5627682caa10 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.424+0000 7fd2fa803740 1 -- 172.21.15.16:0/2332922666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 msgr2=0x5627682deeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.424+0000 7fd2fa803740 1 --2- 172.21.15.16:0/2332922666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 0x5627682deeb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00ef370 tx=0x7fd2e002bc30 comp rx=0 tx=0).stop 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.424+0000 7fd2fa803740 1 -- 172.21.15.16:0/2332922666 shutdown_connections 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.424+0000 7fd2fa803740 1 --2- 172.21.15.16:0/2332922666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682d37e0 0x5627682eb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.424+0000 7fd2fa803740 1 --2- 172.21.15.16:0/2332922666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682caa10 0x5627682deeb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.424+0000 7fd2fa803740 1 --2- 172.21.15.16:0/2332922666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682ca0f0 0x5627682d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.424+0000 7fd2fa803740 1 -- 172.21.15.16:0/2332922666 >> 172.21.15.16:0/2332922666 conn(0x5627682c4a10 msgr2=0x5627682cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.424+0000 7fd2fa803740 1 -- 172.21.15.16:0/2332922666 shutdown_connections 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.424+0000 7fd2fa803740 1 -- 172.21.15.16:0/2332922666 wait complete. 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.424+0000 7fd2fa803740 1 -- start start 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2fa803740 1 Processor -- start 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 0x5627682db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682caa10 0x5627682cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2fa803740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682d37e0 0x5627682cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2f58da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 0x5627682db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2f50d9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682caa10 0x5627682cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2f58da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 0x5627682db7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46296/0 (socket says 172.21.15.16:46296) 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2f50d9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682caa10 0x5627682cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:52176/0 (socket says 172.21.15.16:52176) 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2f58da640 1 -- 172.21.15.16:0/3027498041 learned_addr learned my addr 172.21.15.16:0/3027498041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2f60db640 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682d37e0 0x5627682cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2fa803740 1 -- 172.21.15.16:0/3027498041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01b0e90 con 0x5627682ca0f0 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2fa803740 1 -- 172.21.15.16:0/3027498041 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01b07c0 con 0x5627682caa10 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.425+0000 7fd2fa803740 1 -- 172.21.15.16:0/3027498041 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01ada20 con 0x5627682d37e0 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2fa803740 10 client.0 _ll_register_callbacks cb 0x7fd2ec01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2f50d9640 1 -- 172.21.15.16:0/3027498041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682d37e0 msgr2=0x5627682cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682d37e0 0x5627682cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2f50d9640 1 -- 172.21.15.16:0/3027498041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 msgr2=0x5627682db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 0x5627682db7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2f50d9640 1 -- 172.21.15.16:0/3027498041 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627682cdef0 con 0x5627682caa10 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2f58da640 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 0x5627682db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2f50d9640 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682caa10 0x5627682cd160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00bdd40 tx=0x7fd2e00bd9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=856922c7dd32788e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2d97fa640 1 -- 172.21.15.16:0/3027498041 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e00a1060 con 0x5627682caa10 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2d97fa640 1 -- 172.21.15.16:0/3027498041 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd2e0005000 con 0x5627682caa10 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.426+0000 7fd2fa803740 1 -- 172.21.15.16:0/3027498041 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2e00be4b0 con 0x5627682caa10 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2d97fa640 1 -- 172.21.15.16:0/3027498041 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd2e00d7110 con 0x5627682caa10 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2fa803740 10 client.25826 fetch_fsmap learned FSMap version 25 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2fa803740 1 -- 172.21.15.16:0/3027498041 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562768349700 con 0x5627682caa10 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2d97fa640 1 -- 172.21.15.16:0/3027498041 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd2e00be4b0 con 0x5627682caa10 2024-03-27T06:44:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2d97fa640 1 -- 172.21.15.16:0/3027498041 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd2e00a8040 con 0x5627682caa10 2024-03-27T06:44:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2fa803740 10 client.25826 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:44:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2fa803740 10 client.25826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:44:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2fa803740 20 client.25826 populate_metadata read hostname 'smithi016' 2024-03-27T06:44:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2fa803740 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562768349d20 0x56276834c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:44:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2fa803740 4 client.25826 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:44:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2fa803740 1 -- 172.21.15.16:0/3027498041 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627682356e0 con 0x562768349d20 2024-03-27T06:44:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.427+0000 7fd2f58da640 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562768349d20 0x56276834c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:44:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.428+0000 7fd2d97fa640 10 client.25826 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.428+0000 7fd2f58da640 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562768349d20 0x56276834c140 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d1085657b0a315d server_cookie=a28a4aabc0324db in_seq=0 out_seq=0 2024-03-27T06:44:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2d97fa640 1 -- 172.21.15.16:0/3027498041 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627682356e0 con 0x562768349d20 2024-03-27T06:44:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2d97fa640 10 client.25826 handle_command_reply: tid=0 2024-03-27T06:44:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2fa803740 1 client.25826 shutdown 2024-03-27T06:44:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2fa803740 1 -- 172.21.15.16:0/3027498041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682caa10 msgr2=0x5627682cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2fa803740 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682caa10 0x5627682cd160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00bdd40 tx=0x7fd2e00bd9f0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2fa803740 1 -- 172.21.15.16:0/3027498041 shutdown_connections 2024-03-27T06:44:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2fa803740 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562768349d20 0x56276834c140 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2fa803740 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627682d37e0 0x5627682cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2fa803740 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5627682caa10 0x5627682cd160 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2fa803740 1 --2- 172.21.15.16:0/3027498041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627682ca0f0 0x5627682db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2fa803740 1 -- 172.21.15.16:0/3027498041 >> 172.21.15.16:0/3027498041 conn(0x5627682c4a10 msgr2=0x5627682c31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2d97fa640 0 client.25826 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:44:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2fa803740 1 -- 172.21.15.16:0/3027498041 shutdown_connections 2024-03-27T06:44:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.431+0000 7fd2fa803740 1 -- 172.21.15.16:0/3027498041 wait complete. 2024-03-27T06:44:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.432+0000 7fd2fa803740 20 client.25826 trim_cache size 0 max 16384 2024-03-27T06:44:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:44:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:44:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:44:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:44:48.431 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.432+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/3412458589 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd2dc0a3a50 msgr2=0x7fd2dc0a5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.432+0000 7fd2f7b3e640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd2dc0a3a50 0x7fd2dc0a5f10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009fd0 tx=0x7fd2e80134c0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.432+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/3412458589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0102500 msgr2=0x7fd2f01988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:44:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.432+0000 7fd2f7b3e640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0102500 0x7fd2f01988c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0036040 tx=0x7fd2e002dc00 comp rx=0 tx=0).stop 2024-03-27T06:44:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.432+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/3412458589 shutdown_connections 2024-03-27T06:44:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.433+0000 7fd2f7b3e640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd2dc0a3a50 0x7fd2dc0a5f10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.433+0000 7fd2f7b3e640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f0107c00 0x7fd2f019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.433+0000 7fd2f7b3e640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f0102500 0x7fd2f01988c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.433+0000 7fd2f7b3e640 1 --2- 172.21.15.16:0/3412458589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2f0101b40 0x7fd2f0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:44:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.433+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/3412458589 >> 172.21.15.16:0/3412458589 conn(0x7fd2f00fd650 msgr2=0x7fd2f00feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:44:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.433+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/3412458589 shutdown_connections 2024-03-27T06:44:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:44:48.433+0000 7fd2f7b3e640 1 -- 172.21.15.16:0/3412458589 wait complete. 2024-03-27T06:44:49.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:49 smithi016 ceph-mon[31908]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T06:44:49.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:49 smithi016 ceph-mon[31908]: scrub summary: idle 2024-03-27T06:44:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:49 smithi156 ceph-mon[36261]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T06:44:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:49 smithi156 ceph-mon[36261]: scrub summary: idle 2024-03-27T06:44:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:49 smithi170 ceph-mon[36438]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-27T06:44:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:49 smithi170 ceph-mon[36438]: scrub summary: idle 2024-03-27T06:44:51.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:51 smithi156 ceph-mon[36261]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 511 B/s wr, 14 op/s 2024-03-27T06:44:51.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:51 smithi170 ceph-mon[36438]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 511 B/s wr, 14 op/s 2024-03-27T06:44:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:51 smithi016 ceph-mon[31908]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 511 B/s wr, 14 op/s 2024-03-27T06:44:53.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:53 smithi156 ceph-mon[36261]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-27T06:44:53.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:53 smithi170 ceph-mon[36438]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-27T06:44:53.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:53 smithi016 ceph-mon[31908]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-27T06:44:55.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:55 smithi156 ceph-mon[36261]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T06:44:55.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:55 smithi170 ceph-mon[36438]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T06:44:55.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:55 smithi016 ceph-mon[31908]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T06:44:57.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:57 smithi156 ceph-mon[36261]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T06:44:57.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:57 smithi170 ceph-mon[36438]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T06:44:57.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:57 smithi016 ceph-mon[31908]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T06:44:59.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:44:59 smithi156 ceph-mon[36261]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T06:44:59.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:44:59 smithi170 ceph-mon[36438]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T06:44:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:44:59 smithi016 ceph-mon[31908]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-27T06:45:00.476 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 96% (11872/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-27T06:45:01.106 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to '155268c4e432a12433aa833f174f9fe3b1016ae0'. 2024-03-27T06:45:01.106 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-03-27T06:45:01.106 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-27T06:45:01.107 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-03-27T06:45:01.108 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at 155268c4e43 Merge PR #51993 into wip-rishabh-testing-20240326.131558 2024-03-27T06:45:01.112 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-27T06:45:01.172 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-27T06:45:01.174 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-27T06:45:01.174 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-27T06:45:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 -- 172.21.15.16:0/554135018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf7c1024c0 msgr2=0x7fcf7c10e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 --2- 172.21.15.16:0/554135018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf7c1024c0 0x7fcf7c10e9d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009cd0 tx=0x7fcf7402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 -- 172.21.15.16:0/554135018 shutdown_connections 2024-03-27T06:45:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 --2- 172.21.15.16:0/554135018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf7c1024c0 0x7fcf7c10e9d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 --2- 172.21.15.16:0/554135018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf7c101b00 0x7fcf7c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 --2- 172.21.15.16:0/554135018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf7c107ba0 0x7fcf7c107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 -- 172.21.15.16:0/554135018 >> 172.21.15.16:0/554135018 conn(0x7fcf7c0fd610 msgr2=0x7fcf7c0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 -- 172.21.15.16:0/554135018 shutdown_connections 2024-03-27T06:45:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 -- 172.21.15.16:0/554135018 wait complete. 2024-03-27T06:45:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 Processor -- start 2024-03-27T06:45:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 -- start start 2024-03-27T06:45:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf7c101b00 0x7fcf7c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf7c1024c0 0x7fcf7c074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.524+0000 7fcf82ced640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf7c107ba0 0x7fcf7c0748d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf82ced640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c112040 con 0x7fcf7c1024c0 2024-03-27T06:45:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf82ced640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c111ec0 con 0x7fcf7c107ba0 2024-03-27T06:45:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf82ced640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1121c0 con 0x7fcf7c101b00 2024-03-27T06:45:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf7c1024c0 0x7fcf7c074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf7c1024c0 0x7fcf7c074390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47794/0 (socket says 172.21.15.16:47794) 2024-03-27T06:45:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf7bfff640 1 -- 172.21.15.16:0/2275978802 learned_addr learned my addr 172.21.15.16:0/2275978802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf80a89640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf7c101b00 0x7fcf7c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf8128a640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf7c107ba0 0x7fcf7c0748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf7bfff640 1 -- 172.21.15.16:0/2275978802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf7c101b00 msgr2=0x7fcf7c073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf7bfff640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf7c101b00 0x7fcf7c073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf7bfff640 1 -- 172.21.15.16:0/2275978802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf7c107ba0 msgr2=0x7fcf7c0748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf7bfff640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf7c107ba0 0x7fcf7c0748d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf7bfff640 1 -- 172.21.15.16:0/2275978802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c1a68e0 con 0x7fcf7c1024c0 2024-03-27T06:45:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf7bfff640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf7c1024c0 0x7fcf7c074390 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800e9f0 tx=0x7fcf6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf79ffb640 1 -- 172.21.15.16:0/2275978802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800cd80 con 0x7fcf7c1024c0 2024-03-27T06:45:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf82ced640 1 -- 172.21.15.16:0/2275978802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c1a6bd0 con 0x7fcf7c1024c0 2024-03-27T06:45:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf79ffb640 1 -- 172.21.15.16:0/2275978802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcf68005520 con 0x7fcf7c1024c0 2024-03-27T06:45:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.525+0000 7fcf79ffb640 1 -- 172.21.15.16:0/2275978802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68010690 con 0x7fcf7c1024c0 2024-03-27T06:45:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.526+0000 7fcf82ced640 1 -- 172.21.15.16:0/2275978802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c1a70d0 con 0x7fcf7c1024c0 2024-03-27T06:45:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf79ffb640 1 -- 172.21.15.16:0/2275978802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138654+0+0 (secure 0 0 0) 0x7fcf680046c0 con 0x7fcf7c1024c0 2024-03-27T06:45:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf859b5740 1 Processor -- start 2024-03-27T06:45:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf859b5740 1 -- start start 2024-03-27T06:45:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf79ffb640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf6c0a39d0 0x7fcf6c0a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 0x5643124354d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564312435a10 0x56431243e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 0x564312440bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf859b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5643123fbcd0 con 0x56431243e7e0 2024-03-27T06:45:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf8128a640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf6c0a39d0 0x7fcf6c0a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf859b5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x564312340950 con 0x564312435a10 2024-03-27T06:45:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf859b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564312398e50 con 0x5643124350f0 2024-03-27T06:45:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.528+0000 7fcf79ffb640 1 -- 172.21.15.16:0/2275978802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fcf680d07e0 con 0x7fcf7c1024c0 2024-03-27T06:45:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf80a89640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 0x5643124354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf8128a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 0x564312440bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf80a89640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 0x5643124354d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:46366/0 (socket says 172.21.15.16:46366) 2024-03-27T06:45:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf80a89640 1 -- 172.21.15.16:0/306672123 learned_addr learned my addr 172.21.15.16:0/306672123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf7bfff640 1 --2- 172.21.15.16:0/306672123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564312435a10 0x56431243e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf80a89640 1 -- 172.21.15.16:0/306672123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564312435a10 msgr2=0x56431243e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf80a89640 1 --2- 172.21.15.16:0/306672123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564312435a10 0x56431243e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf80a89640 1 -- 172.21.15.16:0/306672123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 msgr2=0x564312440bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf80a89640 1 --2- 172.21.15.16:0/306672123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 0x564312440bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf80a89640 1 -- 172.21.15.16:0/306672123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643120ad210 con 0x5643124350f0 2024-03-27T06:45:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf8128a640 1 --2- 172.21.15.16:0/306672123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 0x564312440bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:45:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf8128a640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf6c0a39d0 0x7fcf6c0a5e90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcf7402c630 tx=0x7fcf740323d0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf80a89640 1 --2- 172.21.15.16:0/306672123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 0x5643124354d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009c70 tx=0x7fcf70011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad411abb65826b9b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf66ffd640 1 -- 172.21.15.16:0/306672123 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70004230 con 0x5643124350f0 2024-03-27T06:45:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf66ffd640 1 -- 172.21.15.16:0/306672123 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcf70009e20 con 0x5643124350f0 2024-03-27T06:45:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.529+0000 7fcf66ffd640 1 -- 172.21.15.16:0/306672123 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70019760 con 0x5643124350f0 2024-03-27T06:45:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.530+0000 7fcf859b5740 1 -- 172.21.15.16:0/306672123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 msgr2=0x5643124354d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.530+0000 7fcf859b5740 1 --2- 172.21.15.16:0/306672123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 0x5643124354d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009c70 tx=0x7fcf70011810 comp rx=0 tx=0).stop 2024-03-27T06:45:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.530+0000 7fcf859b5740 1 -- 172.21.15.16:0/306672123 shutdown_connections 2024-03-27T06:45:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.530+0000 7fcf859b5740 1 --2- 172.21.15.16:0/306672123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 0x564312440bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.530+0000 7fcf859b5740 1 --2- 172.21.15.16:0/306672123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564312435a10 0x56431243e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.530+0000 7fcf859b5740 1 --2- 172.21.15.16:0/306672123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 0x5643124354d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.530+0000 7fcf859b5740 1 -- 172.21.15.16:0/306672123 >> 172.21.15.16:0/306672123 conn(0x56431242fa10 msgr2=0x56431242fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.530+0000 7fcf859b5740 1 -- 172.21.15.16:0/306672123 shutdown_connections 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.530+0000 7fcf859b5740 1 -- 172.21.15.16:0/306672123 wait complete. 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.530+0000 7fcf859b5740 1 -- start start 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.530+0000 7fcf859b5740 1 Processor -- start 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643124350f0 0x56431243b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564312435a10 0x56431243bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 0x56431243c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf859b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564312394e00 con 0x56431243e7e0 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf859b5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x564312341520 con 0x5643124350f0 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf859b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5643123548e0 con 0x564312435a10 2024-03-27T06:45:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf859b5740 10 client.0 _ll_register_callbacks cb 0x56431238fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf8128a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 0x56431243c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf80a89640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643124350f0 0x56431243b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf7bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564312435a10 0x56431243bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf8128a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 0x56431243c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47810/0 (socket says 172.21.15.16:47810) 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf8128a640 1 -- 172.21.15.16:0/1041058565 learned_addr learned my addr 172.21.15.16:0/1041058565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf7bfff640 1 -- 172.21.15.16:0/1041058565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643124350f0 msgr2=0x56431243b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf7bfff640 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643124350f0 0x56431243b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf7bfff640 1 -- 172.21.15.16:0/1041058565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 msgr2=0x56431243c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf7bfff640 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 0x56431243c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf7bfff640 1 -- 172.21.15.16:0/1041058565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564312179620 con 0x564312435a10 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf8128a640 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 0x56431243c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf7bfff640 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564312435a10 0x56431243bcf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcf680cdd20 tx=0x7fcf6809c180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a971c20bb130b4d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.531+0000 7fcf5bfff640 1 -- 172.21.15.16:0/1041058565 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680b5020 con 0x564312435a10 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf5bfff640 1 -- 172.21.15.16:0/1041058565 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fcf680a5140 con 0x564312435a10 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf859b5740 1 -- 172.21.15.16:0/1041058565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf70009e20 con 0x564312435a10 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf5bfff640 1 -- 172.21.15.16:0/1041058565 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf680a0b10 con 0x564312435a10 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf859b5740 10 client.25840 fetch_fsmap learned FSMap version 25 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf859b5740 1 -- 172.21.15.16:0/1041058565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56431244d370 con 0x564312435a10 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf5bfff640 1 -- 172.21.15.16:0/1041058565 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcf70009e20 con 0x564312435a10 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf5bfff640 1 -- 172.21.15.16:0/1041058565 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcf680ce030 con 0x564312435a10 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf859b5740 10 client.25840 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf859b5740 10 client.25840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf859b5740 20 client.25840 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf859b5740 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x56431244e850 0x564312450c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf859b5740 4 client.25840 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf859b5740 1 -- 172.21.15.16:0/1041058565 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643123a6390 con 0x56431244e850 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.532+0000 7fcf80a89640 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x56431244e850 0x564312450c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.533+0000 7fcf5bfff640 10 client.25840 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.533+0000 7fcf80a89640 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x56431244e850 0x564312450c40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84107da09d0fc4e7 server_cookie=89008841dfe38127 in_seq=0 out_seq=0 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf5bfff640 1 -- 172.21.15.16:0/1041058565 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5643123a6390 con 0x56431244e850 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf5bfff640 10 client.25840 handle_command_reply: tid=0 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 1 client.25840 shutdown 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 1 -- 172.21.15.16:0/1041058565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564312435a10 msgr2=0x56431243bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564312435a10 0x56431243bcf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcf680cdd20 tx=0x7fcf6809c180 comp rx=0 tx=0).stop 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 1 -- 172.21.15.16:0/1041058565 shutdown_connections 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x56431244e850 0x564312450c40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431243e7e0 0x56431243c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564312435a10 0x56431243bcf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 1 --2- 172.21.15.16:0/1041058565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643124350f0 0x56431243b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf5bfff640 0 client.25840 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 1 -- 172.21.15.16:0/1041058565 >> 172.21.15.16:0/1041058565 conn(0x56431242fa10 msgr2=0x564312431760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 1 -- 172.21.15.16:0/1041058565 shutdown_connections 2024-03-27T06:45:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 1 -- 172.21.15.16:0/1041058565 wait complete. 2024-03-27T06:45:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.534+0000 7fcf859b5740 20 client.25840 trim_cache size 0 max 16384 2024-03-27T06:45:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf859b5740 1 Processor -- start 2024-03-27T06:45:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf859b5740 1 -- start start 2024-03-27T06:45:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643124350f0 0x564312449eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564312435a10 0x564312456580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56431243e7e0 0x56431243b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf859b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1af990 con 0x5643124350f0 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf859b5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1b14d0 con 0x564312435a10 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf859b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1b0060 con 0x56431243e7e0 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf80a89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643124350f0 0x564312449eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf80a89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643124350f0 0x564312449eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47816/0 (socket says 172.21.15.16:47816) 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf80a89640 1 -- 172.21.15.16:0/3436369075 learned_addr learned my addr 172.21.15.16:0/3436369075 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf7bfff640 1 --2- 172.21.15.16:0/3436369075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564312435a10 0x564312456580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.549+0000 7fcf8128a640 1 --2- 172.21.15.16:0/3436369075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56431243e7e0 0x56431243b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf80a89640 1 -- 172.21.15.16:0/3436369075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56431243e7e0 msgr2=0x56431243b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf80a89640 1 --2- 172.21.15.16:0/3436369075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56431243e7e0 0x56431243b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf80a89640 1 -- 172.21.15.16:0/3436369075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564312435a10 msgr2=0x564312456580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf80a89640 1 --2- 172.21.15.16:0/3436369075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564312435a10 0x564312456580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf80a89640 1 -- 172.21.15.16:0/3436369075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564312450fb0 con 0x5643124350f0 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf80a89640 1 --2- 172.21.15.16:0/3436369075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643124350f0 0x564312449eb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009c70 tx=0x7fcf70003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b222d0ed4efcb111 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf667fc640 1 -- 172.21.15.16:0/3436369075 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70005c50 con 0x5643124350f0 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf667fc640 1 -- 172.21.15.16:0/3436369075 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcf70003af0 con 0x5643124350f0 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf667fc640 1 -- 172.21.15.16:0/3436369075 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf700196d0 con 0x5643124350f0 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf859b5740 1 -- 172.21.15.16:0/3436369075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643124350f0 msgr2=0x564312449eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf859b5740 1 --2- 172.21.15.16:0/3436369075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643124350f0 0x564312449eb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009c70 tx=0x7fcf70003f50 comp rx=0 tx=0).stop 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.550+0000 7fcf859b5740 1 -- 172.21.15.16:0/3436369075 shutdown_connections 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 --2- 172.21.15.16:0/3436369075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56431243e7e0 0x56431243b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 --2- 172.21.15.16:0/3436369075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564312435a10 0x564312456580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 --2- 172.21.15.16:0/3436369075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643124350f0 0x564312449eb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 -- 172.21.15.16:0/3436369075 >> 172.21.15.16:0/3436369075 conn(0x56431242fa10 msgr2=0x564312436ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 -- 172.21.15.16:0/3436369075 shutdown_connections 2024-03-27T06:45:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 -- 172.21.15.16:0/3436369075 wait complete. 2024-03-27T06:45:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 -- start start 2024-03-27T06:45:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 Processor -- start 2024-03-27T06:45:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 0x564312446930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564312435a10 0x564312441ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431243e7e0 0x564312442390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1b0e00 con 0x564312435a10 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1b0730 con 0x56431243e7e0 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1ad970 con 0x5643124350f0 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf859b5740 10 client.0 _ll_register_callbacks cb 0x7fcf68098a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564312435a10 0x564312441ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564312435a10 0x564312441ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47832/0 (socket says 172.21.15.16:47832) 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf7bfff640 1 -- 172.21.15.16:0/2210103062 learned_addr learned my addr 172.21.15.16:0/2210103062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf80a89640 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 0x564312446930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.551+0000 7fcf8128a640 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431243e7e0 0x564312442390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf7bfff640 1 -- 172.21.15.16:0/2210103062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 msgr2=0x564312446930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf7bfff640 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 0x564312446930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf7bfff640 1 -- 172.21.15.16:0/2210103062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431243e7e0 msgr2=0x564312442390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf7bfff640 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431243e7e0 0x564312442390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf7bfff640 1 -- 172.21.15.16:0/2210103062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643124b3bb0 con 0x564312435a10 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf7bfff640 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564312435a10 0x564312441ca0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800e9c0 tx=0x7fcf6809c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c94e3075338d26a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf5bfff640 1 -- 172.21.15.16:0/2210103062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680b5040 con 0x564312435a10 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf5bfff640 1 -- 172.21.15.16:0/2210103062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fcf680bb2b0 con 0x564312435a10 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf859b5740 1 -- 172.21.15.16:0/2210103062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf70003af0 con 0x564312435a10 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf5bfff640 1 -- 172.21.15.16:0/2210103062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf6809c2a0 con 0x564312435a10 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf859b5740 10 client.16014 fetch_fsmap learned FSMap version 25 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf859b5740 1 -- 172.21.15.16:0/2210103062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643124b4d60 con 0x564312435a10 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf5bfff640 1 -- 172.21.15.16:0/2210103062 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcf70003af0 con 0x564312435a10 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf5bfff640 1 -- 172.21.15.16:0/2210103062 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcf680a4f40 con 0x564312435a10 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf859b5740 10 client.16014 fetch_fsmap finished waiting for FSMap version 25 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf859b5740 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf859b5740 20 client.16014 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf859b5740 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5643124b52c0 0x5643124b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf859b5740 4 client.16014 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:45:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.552+0000 7fcf859b5740 1 -- 172.21.15.16:0/2210103062 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643123a06e0 con 0x5643124b52c0 2024-03-27T06:45:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.553+0000 7fcf80a89640 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5643124b52c0 0x5643124b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.553+0000 7fcf80a89640 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5643124b52c0 0x5643124b76e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87cb0b7827920f85 server_cookie=de35b2db360f7a5f in_seq=0 out_seq=0 2024-03-27T06:45:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:01.553+0000 7fcf5bfff640 10 client.16014 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:01.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:01 smithi156 ceph-mon[36261]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T06:45:01.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:01 smithi170 ceph-mon[36438]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T06:45:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:01 smithi016 ceph-mon[31908]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-27T06:45:01.933 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-27T06:45:01.998 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-27T06:45:02.038 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-27T06:45:02.039 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-27T06:45:02.039 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-27T06:45:02.077 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-27T06:45:02.080 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:45:02.081 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-27T06:45:02.136 INFO:tasks.workunit:Running workunits matching fs/norstats on client.0... 2024-03-27T06:45:02.137 INFO:tasks.workunit:Running workunit fs/norstats/kernel_untar_tar.sh... 2024-03-27T06:45:02.137 DEBUG:teuthology.orchestra.run.smithi016:workunit test fs/norstats/kernel_untar_tar.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=155268c4e432a12433aa833f174f9fe3b1016ae0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh 2024-03-27T06:45:02.197 INFO:tasks.workunit.client.0.smithi016.stderr:+ KERNEL=linux-4.0.5 2024-03-27T06:45:02.197 INFO:tasks.workunit.client.0.smithi016.stderr:+ wget -q http://download.ceph.com/qa/linux-4.0.5.tar.xz 2024-03-27T06:45:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf5bfff640 1 -- 172.21.15.16:0/2210103062 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643123a06e0 con 0x5643124b52c0 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf5bfff640 10 client.16014 handle_command_reply: tid=0 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 1 client.16014 shutdown 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 1 -- 172.21.15.16:0/2210103062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564312435a10 msgr2=0x564312441ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564312435a10 0x564312441ca0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800e9c0 tx=0x7fcf6809c1f0 comp rx=0 tx=0).stop 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 1 -- 172.21.15.16:0/2210103062 shutdown_connections 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5643124b52c0 0x5643124b76e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56431243e7e0 0x564312442390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564312435a10 0x564312441ca0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf5bfff640 0 client.16014 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 1 --2- 172.21.15.16:0/2210103062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5643124350f0 0x564312446930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 1 -- 172.21.15.16:0/2210103062 >> 172.21.15.16:0/2210103062 conn(0x56431242fa10 msgr2=0x56431243ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 1 -- 172.21.15.16:0/2210103062 shutdown_connections 2024-03-27T06:45:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 1 -- 172.21.15.16:0/2210103062 wait complete. 2024-03-27T06:45:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.589+0000 7fcf859b5740 20 client.16014 trim_cache size 0 max 16384 2024-03-27T06:45:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:45:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:45:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:45:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:45:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 -- 172.21.15.16:0/2275978802 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf6c0a39d0 msgr2=0x7fcf6c0a5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf6c0a39d0 0x7fcf6c0a5e90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcf7402c630 tx=0x7fcf740323d0 comp rx=0 tx=0).stop 2024-03-27T06:45:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 -- 172.21.15.16:0/2275978802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf7c1024c0 msgr2=0x7fcf7c074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf7c1024c0 0x7fcf7c074390 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800e9f0 tx=0x7fcf6800edf0 comp rx=0 tx=0).stop 2024-03-27T06:45:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 -- 172.21.15.16:0/2275978802 shutdown_connections 2024-03-27T06:45:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf7c107ba0 0x7fcf7c0748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf7c1024c0 0x7fcf7c074390 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcf6c0a39d0 0x7fcf6c0a5e90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 --2- 172.21.15.16:0/2275978802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf7c101b00 0x7fcf7c073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 -- 172.21.15.16:0/2275978802 >> 172.21.15.16:0/2275978802 conn(0x7fcf7c0fd610 msgr2=0x7fcf7c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 -- 172.21.15.16:0/2275978802 shutdown_connections 2024-03-27T06:45:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:02.590+0000 7fcf82ced640 1 -- 172.21.15.16:0/2275978802 wait complete. 2024-03-27T06:45:03.058 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: *** Caught signal (Segmentation fault) ** 2024-03-27T06:45:03.058 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: in thread 7f19fd221700 thread_name:md_log_replay 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev) 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1a0e5e2d20] 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 2: /usr/bin/ceph-mds(+0x5484cf) [0x562551bdd4cf] 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x562551be0c63] 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 4: (MDLog::_replay_thread()+0x617) [0x562551c3a6c7] 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 5: (MDLog::ReplayThread::entry()+0x11) [0x562551c43db1] 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1a0e5d81ca] 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 7: clone() 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 2024-03-27T06:45:02.705+0000 7f19fd221700 -1 *** Caught signal (Segmentation fault) ** 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: in thread 7f19fd221700 thread_name:md_log_replay 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev) 2024-03-27T06:45:03.059 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1a0e5e2d20] 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 2: /usr/bin/ceph-mds(+0x5484cf) [0x562551bdd4cf] 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x562551be0c63] 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 4: (MDLog::_replay_thread()+0x617) [0x562551c3a6c7] 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 5: (MDLog::ReplayThread::entry()+0x11) [0x562551c43db1] 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1a0e5d81ca] 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 7: clone() 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 0> 2024-03-27T06:45:02.705+0000 7f19fd221700 -1 *** Caught signal (Segmentation fault) ** 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: in thread 7f19fd221700 thread_name:md_log_replay 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev) 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1a0e5e2d20] 2024-03-27T06:45:03.060 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 2: /usr/bin/ceph-mds(+0x5484cf) [0x562551bdd4cf] 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x562551be0c63] 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 4: (MDLog::_replay_thread()+0x617) [0x562551c3a6c7] 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 5: (MDLog::ReplayThread::entry()+0x11) [0x562551c43db1] 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1a0e5d81ca] 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 7: clone() 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 0> 2024-03-27T06:45:02.705+0000 7f19fd221700 -1 *** Caught signal (Segmentation fault) ** 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: in thread 7f19fd221700 thread_name:md_log_replay 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: ceph version 19.0.0-2478-g155268c4 (155268c4e432a12433aa833f174f9fe3b1016ae0) squid (dev) 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1a0e5e2d20] 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 2: /usr/bin/ceph-mds(+0x5484cf) [0x562551bdd4cf] 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x562551be0c63] 2024-03-27T06:45:03.061 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 4: (MDLog::_replay_thread()+0x617) [0x562551c3a6c7] 2024-03-27T06:45:03.062 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 5: (MDLog::ReplayThread::entry()+0x11) [0x562551c43db1] 2024-03-27T06:45:03.062 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f1a0e5d81ca] 2024-03-27T06:45:03.062 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 7: clone() 2024-03-27T06:45:03.062 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-27T06:45:03.062 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[67255]: 2024-03-27T06:45:03.062 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:02 smithi170 podman[70905]: 2024-03-27 06:45:02.781081663 +0000 UTC m=+0.020391732 container died dac4d49d7a3889a2bc8e5b55eff1131cceb73633c22795a6fd1d19d73978f6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T06:45:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:03 smithi156 ceph-mon[36261]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:45:03.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:03 smithi170 ceph-mon[36438]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:45:03.751 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:03 smithi170 podman[70905]: 2024-03-27 06:45:03.492570726 +0000 UTC m=+0.731880794 container remove dac4d49d7a3889a2bc8e5b55eff1131cceb73633c22795a6fd1d19d73978f6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:45:03.751 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:03 smithi170 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-03-27T06:45:03.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:03 smithi016 ceph-mon[31908]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:45:04.095 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:03 smithi170 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.f.service: Failed with result 'exit-code'. 2024-03-27T06:45:05.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:05 smithi156 ceph-mon[36261]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:45:05.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:05 smithi156 ceph-mon[36261]: mgrmap e25: x(active, since 90s), standbys: y, z 2024-03-27T06:45:05.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:05 smithi170 ceph-mon[36438]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:45:05.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:05 smithi170 ceph-mon[36438]: mgrmap e25: x(active, since 90s), standbys: y, z 2024-03-27T06:45:05.858 INFO:tasks.workunit.client.0.smithi016.stderr:+ mkdir untar_tar 2024-03-27T06:45:05.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:05 smithi016 ceph-mon[31908]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:45:05.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:05 smithi016 ceph-mon[31908]: mgrmap e25: x(active, since 90s), standbys: y, z 2024-03-27T06:45:05.862 INFO:tasks.workunit.client.0.smithi016.stderr:+ cd untar_tar 2024-03-27T06:45:05.862 INFO:tasks.workunit.client.0.smithi016.stderr:+ tar Jxvf ../linux-4.0.5.tar.xz linux-4.0.5/Documentation/ 2024-03-27T06:45:05.885 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ 2024-03-27T06:45:05.887 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/00-INDEX 2024-03-27T06:45:05.888 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/ 2024-03-27T06:45:05.889 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/README 2024-03-27T06:45:05.890 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/obsolete/ 2024-03-27T06:45:05.891 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/obsolete/proc-sys-vm-nr_pdflush_threads 2024-03-27T06:45:05.892 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-bus-usb 2024-03-27T06:45:05.893 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-class-rfkill 2024-03-27T06:45:05.894 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-koneplus 2024-03-27T06:45:05.895 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-kovaplus 2024-03-27T06:45:05.896 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra 2024-03-27T06:45:05.897 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/removed/ 2024-03-27T06:45:05.898 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/removed/devfs 2024-03-27T06:45:05.899 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/removed/dv1394 2024-03-27T06:45:05.901 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/removed/ip_queue 2024-03-27T06:45:05.901 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/removed/net_dma 2024-03-27T06:45:05.903 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/removed/o2cb 2024-03-27T06:45:05.905 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/removed/raw1394 2024-03-27T06:45:05.907 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/removed/video1394 2024-03-27T06:45:05.908 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/ 2024-03-27T06:45:05.910 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/firewire-cdev 2024-03-27T06:45:05.911 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/o2cb 2024-03-27T06:45:05.913 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/syscalls 2024-03-27T06:45:05.915 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-acpi-pmprofile 2024-03-27T06:45:05.916 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-firewire 2024-03-27T06:45:05.918 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-usb 2024-03-27T06:45:05.919 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-xen-backend 2024-03-27T06:45:05.920 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-backlight 2024-03-27T06:45:05.921 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-rfkill 2024-03-27T06:45:05.922 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-tpm 2024-03-27T06:45:05.923 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-ubi 2024-03-27T06:45:05.924 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-udc 2024-03-27T06:45:05.925 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-node 2024-03-27T06:45:05.926 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-cpu 2024-03-27T06:45:05.927 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-xen_memory 2024-03-27T06:45:05.928 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-ib_srp 2024-03-27T06:45:05.929 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-qla2xxx 2024-03-27T06:45:05.930 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-usb-usbtmc 2024-03-27T06:45:05.931 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-w1_ds28e04 2024-03-27T06:45:05.932 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-efi-vars 2024-03-27T06:45:05.933 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-dump 2024-03-27T06:45:05.934 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-elog 2024-03-27T06:45:05.935 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-module 2024-03-27T06:45:05.936 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-transport-srp 2024-03-27T06:45:05.937 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/thermal-notification 2024-03-27T06:45:05.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/stable/vdso 2024-03-27T06:45:05.943 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/ 2024-03-27T06:45:05.943 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-spear-pcie-gadget 2024-03-27T06:45:05.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget 2024-03-27T06:45:05.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-acm 2024-03-27T06:45:05.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ecm 2024-03-27T06:45:05.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-eem 2024-03-27T06:45:05.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ffs 2024-03-27T06:45:05.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-hid 2024-03-27T06:45:05.947 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-loopback 2024-03-27T06:45:05.948 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-mass-storage 2024-03-27T06:45:05.949 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-midi 2024-03-27T06:45:05.950 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ncm 2024-03-27T06:45:05.952 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-obex 2024-03-27T06:45:05.953 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-phonet 2024-03-27T06:45:05.955 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-rndis 2024-03-27T06:45:05.956 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-serial 2024-03-27T06:45:05.958 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-sourcesink 2024-03-27T06:45:05.959 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-subset 2024-03-27T06:45:05.961 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac1 2024-03-27T06:45:05.963 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac2 2024-03-27T06:45:05.964 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uvc 2024-03-27T06:45:05.966 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-driver-genwqe 2024-03-27T06:45:05.967 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ec 2024-03-27T06:45:05.969 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ideapad 2024-03-27T06:45:05.970 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-olpc 2024-03-27T06:45:05.972 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pfo-nx-crypto 2024-03-27T06:45:05.974 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pktcdvd 2024-03-27T06:45:05.975 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/dev-kmsg 2024-03-27T06:45:05.977 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/evm 2024-03-27T06:45:05.978 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/ima_policy 2024-03-27T06:45:05.980 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/procfs-diskstats 2024-03-27T06:45:05.981 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/pstore 2024-03-27T06:45:05.983 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ata 2024-03-27T06:45:05.985 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block 2024-03-27T06:45:05.986 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-bcache 2024-03-27T06:45:05.988 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-dm 2024-03-27T06:45:05.989 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-rssd 2024-03-27T06:45:05.991 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-zram 2024-03-27T06:45:05.993 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-acpi 2024-03-27T06:45:05.994 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-amba 2024-03-27T06:45:05.996 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-bcma 2024-03-27T06:45:05.998 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10 2024-03-27T06:45:05.999 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm3x 2024-03-27T06:45:06.001 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-funnel 2024-03-27T06:45:06.002 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc 2024-03-27T06:45:06.003 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-css 2024-03-27T06:45:06.004 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-events 2024-03-27T06:45:06.005 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-format 2024-03-27T06:45:06.006 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 2024-03-27T06:45:06.007 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_gpci 2024-03-27T06:45:06.008 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-fcoe 2024-03-27T06:45:06.009 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-hsi 2024-03-27T06:45:06.010 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-fsa9480 2024-03-27T06:45:06.011 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 2024-03-27T06:45:06.012 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-lm3533 2024-03-27T06:45:06.013 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio 2024-03-27T06:45:06.014 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-accel-bmc150 2024-03-27T06:45:06.015 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-ad9523 2024-03-27T06:45:06.016 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4350 2024-03-27T06:45:06.017 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-gyro-bmg160 2024-03-27T06:45:06.018 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-light-lm3533-als 2024-03-27T06:45:06.019 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-mpu6050 2024-03-27T06:45:06.020 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 2024-03-27T06:45:06.021 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-trigger-sysfs 2024-03-27T06:45:06.022 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mdio 2024-03-27T06:45:06.023 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-media 2024-03-27T06:45:06.024 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mei 2024-03-27T06:45:06.025 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci 2024-03-27T06:45:06.026 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss 2024-03-27T06:45:06.027 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-drivers-ehci_hcd 2024-03-27T06:45:06.028 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-platform 2024-03-27T06:45:06.029 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rbd 2024-03-27T06:45:06.030 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rpmsg 2024-03-27T06:45:06.031 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-umc 2024-03-27T06:45:06.033 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb 2024-03-27T06:45:06.033 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg 2024-03-27T06:45:06.034 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-lvstest 2024-03-27T06:45:06.035 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-c2port 2024-03-27T06:45:06.036 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-cfq-target-latency 2024-03-27T06:45:06.037 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class 2024-03-27T06:45:06.038 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-adp8870 2024-03-27T06:45:06.039 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 2024-03-27T06:45:06.040 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-bdi 2024-03-27T06:45:06.041 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-cxl 2024-03-27T06:45:06.042 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-devfreq 2024-03-27T06:45:06.044 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-extcon 2024-03-27T06:45:06.045 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu 2024-03-27T06:45:06.046 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-amd-iommu 2024-03-27T06:45:06.047 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-intel-iommu 2024-03-27T06:45:06.048 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-lcd 2024-03-27T06:45:06.049 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led 2024-03-27T06:45:06.050 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 2024-03-27T06:45:06.052 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-leds-gt683r 2024-03-27T06:45:06.054 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mei 2024-03-27T06:45:06.055 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mic.txt 2024-03-27T06:45:06.057 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mtd 2024-03-27T06:45:06.059 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net 2024-03-27T06:45:06.060 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-batman-adv 2024-03-27T06:45:06.062 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-cdc_ncm 2024-03-27T06:45:06.064 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-grcan 2024-03-27T06:45:06.065 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-mesh 2024-03-27T06:45:06.067 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-queues 2024-03-27T06:45:06.069 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-statistics 2024-03-27T06:45:06.070 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pktcdvd 2024-03-27T06:45:06.072 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-power 2024-03-27T06:45:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-powercap 2024-03-27T06:45:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pwm 2024-03-27T06:45:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rc 2024-03-27T06:45:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-regulator 2024-03-27T06:45:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rtc-rtc0-device-rtc_calibration 2024-03-27T06:45:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-scsi_host 2024-03-27T06:45:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc 2024-03-27T06:45:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc-wusbhc 2024-03-27T06:45:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-dev 2024-03-27T06:45:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices 2024-03-27T06:45:06.090 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-edac 2024-03-27T06:45:06.092 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-firmware_node 2024-03-27T06:45:06.093 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-lpss_ltr 2024-03-27T06:45:06.094 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-memory 2024-03-27T06:45:06.095 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-mmc 2024-03-27T06:45:06.096 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-online 2024-03-27T06:45:06.097 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-_UDC_-gadget 2024-03-27T06:45:06.099 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-docg3 2024-03-27T06:45:06.100 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-sh_mobile_lcdc_fb 2024-03-27T06:45:06.102 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power 2024-03-27T06:45:06.103 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D0 2024-03-27T06:45:06.105 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D1 2024-03-27T06:45:06.107 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D2 2024-03-27T06:45:06.108 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D3hot 2024-03-27T06:45:06.110 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_wakeup 2024-03-27T06:45:06.111 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_state 2024-03-27T06:45:06.113 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-real_power_state 2024-03-27T06:45:06.115 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-resource_in_use 2024-03-27T06:45:06.117 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-soc 2024-03-27T06:45:06.118 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-sun 2024-03-27T06:45:06.120 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-cpu 2024-03-27T06:45:06.121 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-ibm-rtl 2024-03-27T06:45:06.123 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-xen_cpu 2024-03-27T06:45:06.124 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-genwqe 2024-03-27T06:45:06.126 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid 2024-03-27T06:45:06.128 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-lenovo 2024-03-27T06:45:06.129 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-logitech-lg4ff 2024-03-27T06:45:06.131 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-multitouch 2024-03-27T06:45:06.132 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-picolcd 2024-03-27T06:45:06.133 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-prodikeys 2024-03-27T06:45:06.134 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-arvo 2024-03-27T06:45:06.135 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-isku 2024-03-27T06:45:06.136 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone 2024-03-27T06:45:06.137 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-koneplus 2024-03-27T06:45:06.138 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-konepure 2024-03-27T06:45:06.139 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kovaplus 2024-03-27T06:45:06.140 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-lua 2024-03-27T06:45:06.141 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-pyra 2024-03-27T06:45:07.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:07 smithi156 ceph-mon[36261]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-03-27T06:45:07.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:07 smithi170 ceph-mon[36438]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-03-27T06:45:07.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:07 smithi016 ceph-mon[31908]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-03-27T06:45:08.171 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-ryos 2024-03-27T06:45:08.173 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-savu 2024-03-27T06:45:08.174 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-srws1 2024-03-27T06:45:08.175 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-wiimote 2024-03-27T06:45:08.176 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-input-axp-pek 2024-03-27T06:45:08.177 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-intel-rapid-start 2024-03-27T06:45:08.178 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-pciback 2024-03-27T06:45:08.179 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-ppi 2024-03-27T06:45:08.180 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-samsung-laptop 2024-03-27T06:45:08.181 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-sunxi-sid 2024-03-27T06:45:08.182 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-tegra-fuse 2024-03-27T06:45:08.183 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-toshiba_acpi 2024-03-27T06:45:08.184 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-wacom 2024-03-27T06:45:08.185 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkback 2024-03-27T06:45:08.186 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkfront 2024-03-27T06:45:08.187 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-acpi 2024-03-27T06:45:08.188 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-dmi 2024-03-27T06:45:08.190 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi 2024-03-27T06:45:08.194 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi-runtime-map 2024-03-27T06:45:08.194 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-gsmi 2024-03-27T06:45:08.194 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-log 2024-03-27T06:45:08.196 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-memmap 2024-03-27T06:45:08.197 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-ofw 2024-03-27T06:45:08.199 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sfi 2024-03-27T06:45:08.201 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sgi_uv 2024-03-27T06:45:08.203 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-ext4 2024-03-27T06:45:08.204 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-f2fs 2024-03-27T06:45:08.207 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-nilfs2 2024-03-27T06:45:08.208 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-xfs 2024-03-27T06:45:08.210 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-gpio 2024-03-27T06:45:08.211 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-i2c-bmp085 2024-03-27T06:45:08.212 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ibft 2024-03-27T06:45:08.214 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-boot_params 2024-03-27T06:45:08.216 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-fscaps 2024-03-27T06:45:08.218 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-iommu_groups 2024-03-27T06:45:08.219 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-livepatch 2024-03-27T06:45:08.221 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm 2024-03-27T06:45:08.222 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-hugepages 2024-03-27T06:45:08.223 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-ksm 2024-03-27T06:45:08.225 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-slab 2024-03-27T06:45:08.226 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-uids 2024-03-27T06:45:08.228 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo 2024-03-27T06:45:08.229 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-memory-page-offline 2024-03-27T06:45:08.231 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-module 2024-03-27T06:45:08.232 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ocfs2 2024-03-27T06:45:08.233 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-laptop 2024-03-27T06:45:08.234 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-wmi 2024-03-27T06:45:08.235 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-at91 2024-03-27T06:45:08.236 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-brcmstb-gisb-arb 2024-03-27T06:45:08.237 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-chipidea-usb-otg 2024-03-27T06:45:08.238 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-eeepc-laptop 2024-03-27T06:45:08.239 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ideapad-laptop 2024-03-27T06:45:08.240 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-kim 2024-03-27T06:45:08.241 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-msi-laptop 2024-03-27T06:45:08.243 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-tahvo-usb 2024-03-27T06:45:08.244 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ts5500 2024-03-27T06:45:08.245 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-power 2024-03-27T06:45:08.247 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-pps 2024-03-27T06:45:08.248 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-profiling 2024-03-27T06:45:08.250 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ptp 2024-03-27T06:45:08.251 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-tty 2024-03-27T06:45:08.252 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-wusb_cbaf 2024-03-27T06:45:08.253 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/BUG-HUNTING 2024-03-27T06:45:08.254 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/Changes 2024-03-27T06:45:08.255 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/CodeOfConflict 2024-03-27T06:45:08.258 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/CodingStyle 2024-03-27T06:45:08.259 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DMA-API-HOWTO.txt 2024-03-27T06:45:08.259 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DMA-API.txt 2024-03-27T06:45:08.259 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DMA-ISA-LPC.txt 2024-03-27T06:45:08.260 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DMA-attributes.txt 2024-03-27T06:45:08.261 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/ 2024-03-27T06:45:08.264 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/.gitignore 2024-03-27T06:45:08.264 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/80211.tmpl 2024-03-27T06:45:08.264 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/Makefile 2024-03-27T06:45:08.265 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/alsa-driver-api.tmpl 2024-03-27T06:45:08.266 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/crypto-API.tmpl 2024-03-27T06:45:08.267 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/debugobjects.tmpl 2024-03-27T06:45:08.268 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/device-drivers.tmpl 2024-03-27T06:45:08.269 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/deviceiobook.tmpl 2024-03-27T06:45:08.270 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/drm.tmpl 2024-03-27T06:45:08.272 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/filesystems.tmpl 2024-03-27T06:45:08.273 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/gadget.tmpl 2024-03-27T06:45:08.274 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/genericirq.tmpl 2024-03-27T06:45:08.275 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/kernel-api.tmpl 2024-03-27T06:45:08.276 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/kernel-hacking.tmpl 2024-03-27T06:45:08.277 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/kernel-locking.tmpl 2024-03-27T06:45:08.278 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/kgdb.tmpl 2024-03-27T06:45:08.279 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/libata.tmpl 2024-03-27T06:45:08.280 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/librs.tmpl 2024-03-27T06:45:08.281 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/lsm.tmpl 2024-03-27T06:45:08.282 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/ 2024-03-27T06:45:08.283 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/Makefile 2024-03-27T06:45:08.283 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/bayer.png.b64 2024-03-27T06:45:08.284 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/constraints.png.b64 2024-03-27T06:45:08.286 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/crop.gif.b64 2024-03-27T06:45:08.287 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ 2024-03-27T06:45:08.288 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/.gitignore 2024-03-27T06:45:08.289 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/audio.xml 2024-03-27T06:45:08.290 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ca.xml 2024-03-27T06:45:08.291 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/demux.xml 2024-03-27T06:45:08.292 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbapi.xml 2024-03-27T06:45:08.293 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbproperty.xml 2024-03-27T06:45:08.294 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbstb.pdf 2024-03-27T06:45:08.295 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/examples.xml 2024-03-27T06:45:08.296 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/frontend.xml 2024-03-27T06:45:08.297 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/intro.xml 2024-03-27T06:45:08.298 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/kdapi.xml 2024-03-27T06:45:08.299 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/net.xml 2024-03-27T06:45:08.300 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/video.xml 2024-03-27T06:45:08.301 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/dvbstb.png.b64 2024-03-27T06:45:08.302 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_bt.gif.b64 2024-03-27T06:45:08.304 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_tb.gif.b64 2024-03-27T06:45:08.305 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt.gif.b64 2024-03-27T06:45:08.306 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt_example.gif.b64 2024-03-27T06:45:08.307 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/pipeline.png.b64 2024-03-27T06:45:08.308 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/selection.png.b64 2024-03-27T06:45:08.309 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/ 2024-03-27T06:45:08.310 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/.gitignore 2024-03-27T06:45:08.311 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/biblio.xml 2024-03-27T06:45:08.312 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/capture.c.xml 2024-03-27T06:45:08.313 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/common.xml 2024-03-27T06:45:08.314 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/compat.xml 2024-03-27T06:45:08.316 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/controls.xml 2024-03-27T06:45:08.318 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/crop.pdf 2024-03-27T06:45:08.320 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-capture.xml 2024-03-27T06:45:08.321 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-codec.xml 2024-03-27T06:45:08.322 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-effect.xml 2024-03-27T06:45:08.323 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-event.xml 2024-03-27T06:45:08.324 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-osd.xml 2024-03-27T06:45:08.325 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-output.xml 2024-03-27T06:45:08.326 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-overlay.xml 2024-03-27T06:45:08.327 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-radio.xml 2024-03-27T06:45:08.328 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-raw-vbi.xml 2024-03-27T06:45:08.329 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-rds.xml 2024-03-27T06:45:08.330 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sdr.xml 2024-03-27T06:45:08.331 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sliced-vbi.xml 2024-03-27T06:45:08.332 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-subdev.xml 2024-03-27T06:45:08.333 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-teletext.xml 2024-03-27T06:45:08.334 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/driver.xml 2024-03-27T06:45:08.335 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fdl-appendix.xml 2024-03-27T06:45:08.336 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_bt.pdf 2024-03-27T06:45:08.337 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_tb.pdf 2024-03-27T06:45:08.338 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-close.xml 2024-03-27T06:45:08.339 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-ioctl.xml 2024-03-27T06:45:08.340 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-mmap.xml 2024-03-27T06:45:08.341 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-munmap.xml 2024-03-27T06:45:08.342 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-open.xml 2024-03-27T06:45:08.343 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-poll.xml 2024-03-27T06:45:08.344 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-read.xml 2024-03-27T06:45:08.345 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-select.xml 2024-03-27T06:45:08.346 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-write.xml 2024-03-27T06:45:08.347 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/gen-errors.xml 2024-03-27T06:45:08.348 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/io.xml 2024-03-27T06:45:08.349 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/keytable.c.xml 2024-03-27T06:45:08.350 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/libv4l.xml 2024-03-27T06:45:08.351 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/lirc_device_interface.xml 2024-03-27T06:45:08.352 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-controller.xml 2024-03-27T06:45:08.353 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-close.xml 2024-03-27T06:45:08.354 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-ioctl.xml 2024-03-27T06:45:08.355 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-open.xml 2024-03-27T06:45:08.356 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-device-info.xml 2024-03-27T06:45:08.357 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-entities.xml 2024-03-27T06:45:08.358 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-links.xml 2024-03-27T06:45:08.359 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-setup-link.xml 2024-03-27T06:45:08.360 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pipeline.pdf 2024-03-27T06:45:08.361 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-grey.xml 2024-03-27T06:45:08.362 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-m420.xml 2024-03-27T06:45:08.363 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12.xml 2024-03-27T06:45:08.364 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12m.xml 2024-03-27T06:45:08.365 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12mt.xml 2024-03-27T06:45:08.366 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16.xml 2024-03-27T06:45:08.367 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml 2024-03-27T06:45:08.368 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv24.xml 2024-03-27T06:45:08.369 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-rgb.xml 2024-03-27T06:45:08.370 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-yuv.xml 2024-03-27T06:45:08.371 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr16.xml 2024-03-27T06:45:08.372 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr8.xml 2024-03-27T06:45:08.373 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs08.xml 2024-03-27T06:45:08.374 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs14le.xml 2024-03-27T06:45:08.375 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu08.xml 2024-03-27T06:45:08.376 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu16le.xml 2024-03-27T06:45:08.377 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-ru12le.xml 2024-03-27T06:45:08.378 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgbrg8.xml 2024-03-27T06:45:08.379 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgrbg8.xml 2024-03-27T06:45:08.380 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10.xml 2024-03-27T06:45:08.381 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10alaw8.xml 2024-03-27T06:45:08.382 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10dpcm8.xml 2024-03-27T06:45:08.383 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10p.xml 2024-03-27T06:45:08.384 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb12.xml 2024-03-27T06:45:08.386 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb8.xml 2024-03-27T06:45:08.387 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uv8.xml 2024-03-27T06:45:08.389 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uyvy.xml 2024-03-27T06:45:08.390 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-vyuy.xml 2024-03-27T06:45:08.392 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10.xml 2024-03-27T06:45:08.393 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10b.xml 2024-03-27T06:45:08.394 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y12.xml 2024-03-27T06:45:08.395 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y16.xml 2024-03-27T06:45:08.396 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y41p.xml 2024-03-27T06:45:08.397 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv410.xml 2024-03-27T06:45:08.398 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv411p.xml 2024-03-27T06:45:08.403 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420.xml 2024-03-27T06:45:08.403 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420m.xml 2024-03-27T06:45:08.403 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv422p.xml 2024-03-27T06:45:08.403 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuyv.xml 2024-03-27T06:45:08.403 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvu420m.xml 2024-03-27T06:45:08.404 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvyu.xml 2024-03-27T06:45:08.405 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt.xml 2024-03-27T06:45:08.407 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/planar-apis.xml 2024-03-27T06:45:08.408 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/remote_controllers.xml 2024-03-27T06:45:08.410 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selection-api.xml 2024-03-27T06:45:08.411 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selections-common.xml 2024-03-27T06:45:08.413 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-formats.xml 2024-03-27T06:45:08.415 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.dia 2024-03-27T06:45:08.416 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.svg 2024-03-27T06:45:08.418 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.dia 2024-03-27T06:45:08.419 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.svg 2024-03-27T06:45:08.420 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.dia 2024-03-27T06:45:08.421 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.svg 2024-03-27T06:45:08.423 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2.xml 2024-03-27T06:45:08.423 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2grab.c.xml 2024-03-27T06:45:08.424 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_525.pdf 2024-03-27T06:45:08.425 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_625.pdf 2024-03-27T06:45:08.426 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_hsync.pdf 2024-03-27T06:45:08.427 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml 2024-03-27T06:45:08.428 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-cropcap.xml 2024-03-27T06:45:08.429 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-chip-info.xml 2024-03-27T06:45:08.430 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-register.xml 2024-03-27T06:45:08.432 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-decoder-cmd.xml 2024-03-27T06:45:08.432 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dqevent.xml 2024-03-27T06:45:08.433 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml 2024-03-27T06:45:08.434 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-encoder-cmd.xml 2024-03-27T06:45:08.435 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml 2024-03-27T06:45:08.436 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-fmt.xml 2024-03-27T06:45:08.437 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-frameintervals.xml 2024-03-27T06:45:08.439 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-framesizes.xml 2024-03-27T06:45:08.440 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml 2024-03-27T06:45:08.441 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudio.xml 2024-03-27T06:45:08.442 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudioout.xml 2024-03-27T06:45:08.443 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enuminput.xml 2024-03-27T06:45:08.444 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumoutput.xml 2024-03-27T06:45:08.445 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumstd.xml 2024-03-27T06:45:08.446 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-expbuf.xml 2024-03-27T06:45:08.447 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audio.xml 2024-03-27T06:45:08.448 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audioout.xml 2024-03-27T06:45:08.449 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-crop.xml 2024-03-27T06:45:08.450 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ctrl.xml 2024-03-27T06:45:08.451 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-dv-timings.xml 2024-03-27T06:45:08.452 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-edid.xml 2024-03-27T06:45:08.453 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-enc-index.xml 2024-03-27T06:45:08.454 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ext-ctrls.xml 2024-03-27T06:45:08.455 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fbuf.xml 2024-03-27T06:45:08.456 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fmt.xml 2024-03-27T06:45:08.458 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-frequency.xml 2024-03-27T06:45:08.460 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-input.xml 2024-03-27T06:45:08.461 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-jpegcomp.xml 2024-03-27T06:45:08.463 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-modulator.xml 2024-03-27T06:45:08.464 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-output.xml 2024-03-27T06:45:08.466 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-parm.xml 2024-03-27T06:45:08.468 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-priority.xml 2024-03-27T06:45:08.469 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-selection.xml 2024-03-27T06:45:08.471 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-sliced-vbi-cap.xml 2024-03-27T06:45:08.472 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-std.xml 2024-03-27T06:45:08.474 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-tuner.xml 2024-03-27T06:45:08.475 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-log-status.xml 2024-03-27T06:45:08.477 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-overlay.xml 2024-03-27T06:45:08.479 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml 2024-03-27T06:45:08.481 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-qbuf.xml 2024-03-27T06:45:08.482 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml 2024-03-27T06:45:08.484 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querybuf.xml 2024-03-27T06:45:09.465 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querycap.xml 2024-03-27T06:45:09.466 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-queryctrl.xml 2024-03-27T06:45:09.467 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querystd.xml 2024-03-27T06:45:09.468 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-reqbufs.xml 2024-03-27T06:45:09.470 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-s-hw-freq-seek.xml 2024-03-27T06:45:09.471 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-streamon.xml 2024-03-27T06:45:09.472 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml 2024-03-27T06:45:09.474 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml 2024-03-27T06:45:09.476 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml 2024-03-27T06:45:09.477 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-crop.xml 2024-03-27T06:45:09.479 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml 2024-03-27T06:45:09.481 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml 2024-03-27T06:45:09.483 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml 2024-03-27T06:45:09.484 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml 2024-03-27T06:45:09.486 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_525.gif.b64 2024-03-27T06:45:09.488 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_625.gif.b64 2024-03-27T06:45:09.490 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_hsync.gif.b64 2024-03-27T06:45:09.492 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/media_api.tmpl 2024-03-27T06:45:09.493 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/mtdnand.tmpl 2024-03-27T06:45:09.495 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/networking.tmpl 2024-03-27T06:45:09.497 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/rapidio.tmpl 2024-03-27T06:45:09.499 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/regulator.tmpl 2024-03-27T06:45:09.501 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/s390-drivers.tmpl 2024-03-27T06:45:09.502 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/scsi.tmpl 2024-03-27T06:45:09.504 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/sh.tmpl 2024-03-27T06:45:09.506 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/stylesheet.xsl 2024-03-27T06:45:09.508 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/tracepoint.tmpl 2024-03-27T06:45:09.509 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/uio-howto.tmpl 2024-03-27T06:45:09.511 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/usb.tmpl 2024-03-27T06:45:09.513 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/w1.tmpl 2024-03-27T06:45:09.515 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/writing-an-alsa-driver.tmpl 2024-03-27T06:45:09.518 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/writing_musb_glue_layer.tmpl 2024-03-27T06:45:09.520 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/writing_usb_driver.tmpl 2024-03-27T06:45:09.521 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/DocBook/z8530book.tmpl 2024-03-27T06:45:09.523 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/EDID/ 2024-03-27T06:45:09.524 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/EDID/1024x768.S 2024-03-27T06:45:09.525 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/EDID/1280x1024.S 2024-03-27T06:45:09.526 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/EDID/1600x1200.S 2024-03-27T06:45:09.527 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/EDID/1680x1050.S 2024-03-27T06:45:09.528 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/EDID/1920x1080.S 2024-03-27T06:45:09.529 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/EDID/800x600.S 2024-03-27T06:45:09.530 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/EDID/HOWTO.txt 2024-03-27T06:45:09.531 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/EDID/Makefile 2024-03-27T06:45:09.533 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/EDID/edid.S 2024-03-27T06:45:09.534 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/EDID/hex 2024-03-27T06:45:09.536 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/HOWTO 2024-03-27T06:45:09.537 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/IPMI.txt 2024-03-27T06:45:09.539 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/IRQ-affinity.txt 2024-03-27T06:45:09.541 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/IRQ-domain.txt 2024-03-27T06:45:09.542 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/IRQ.txt 2024-03-27T06:45:09.543 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/Intel-IOMMU.txt 2024-03-27T06:45:09.545 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/Makefile 2024-03-27T06:45:09.547 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ManagementStyle 2024-03-27T06:45:09.549 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/PCI/ 2024-03-27T06:45:09.550 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/PCI/00-INDEX 2024-03-27T06:45:09.552 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/PCI/MSI-HOWTO.txt 2024-03-27T06:45:09.554 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/PCI/PCIEBUS-HOWTO.txt 2024-03-27T06:45:09.556 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/PCI/pci-error-recovery.txt 2024-03-27T06:45:09.558 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/PCI/pci-iov-howto.txt 2024-03-27T06:45:09.559 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/PCI/pci.txt 2024-03-27T06:45:09.560 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/PCI/pcieaer-howto.txt 2024-03-27T06:45:09.562 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/ 2024-03-27T06:45:09.563 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/00-INDEX 2024-03-27T06:45:09.565 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/NMI-RCU.txt 2024-03-27T06:45:09.566 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/RTFP.txt 2024-03-27T06:45:09.567 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/UP.txt 2024-03-27T06:45:09.568 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/arrayRCU.txt 2024-03-27T06:45:09.570 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/checklist.txt 2024-03-27T06:45:09.571 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/listRCU.txt 2024-03-27T06:45:09.572 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/lockdep-splat.txt 2024-03-27T06:45:09.574 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/lockdep.txt 2024-03-27T06:45:09.574 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/rcu.txt 2024-03-27T06:45:09.576 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/rcu_dereference.txt 2024-03-27T06:45:09.577 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/rcubarrier.txt 2024-03-27T06:45:09.579 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/rculist_nulls.txt 2024-03-27T06:45:09.580 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/rcuref.txt 2024-03-27T06:45:09.582 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/stallwarn.txt 2024-03-27T06:45:09.584 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/torture.txt 2024-03-27T06:45:09.585 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/trace.txt 2024-03-27T06:45:09.586 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/RCU/whatisRCU.txt 2024-03-27T06:45:09.587 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/SAK.txt 2024-03-27T06:45:09.588 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/SM501.txt 2024-03-27T06:45:09.589 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/SecurityBugs 2024-03-27T06:45:09.591 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/SubmitChecklist 2024-03-27T06:45:09.592 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/SubmittingDrivers 2024-03-27T06:45:09.593 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/SubmittingPatches 2024-03-27T06:45:09.594 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/VGA-softcursor.txt 2024-03-27T06:45:09.595 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/accounting/ 2024-03-27T06:45:09.596 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/accounting/.gitignore 2024-03-27T06:45:09.597 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/accounting/Makefile 2024-03-27T06:45:09.598 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/accounting/cgroupstats.txt 2024-03-27T06:45:09.599 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/accounting/delay-accounting.txt 2024-03-27T06:45:09.600 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/accounting/getdelays.c 2024-03-27T06:45:09.601 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/accounting/taskstats-struct.txt 2024-03-27T06:45:09.602 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/accounting/taskstats.txt 2024-03-27T06:45:09.603 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/ 2024-03-27T06:45:09.605 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/apei/ 2024-03-27T06:45:09.605 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/apei/einj.txt 2024-03-27T06:45:09.606 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/apei/output_format.txt 2024-03-27T06:45:09.607 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/debug.txt 2024-03-27T06:45:09.608 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/dsdt-override.txt 2024-03-27T06:45:09.609 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/enumeration.txt 2024-03-27T06:45:09.610 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/gpio-properties.txt 2024-03-27T06:45:09.611 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/initrd_table_override.txt 2024-03-27T06:45:09.612 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/method-customizing.txt 2024-03-27T06:45:09.613 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/method-tracing.txt 2024-03-27T06:45:09.614 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/namespace.txt 2024-03-27T06:45:09.615 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/scan_handlers.txt 2024-03-27T06:45:09.616 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/acpi/video_extension.txt 2024-03-27T06:45:09.618 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/aoe/ 2024-03-27T06:45:09.619 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/aoe/aoe.txt 2024-03-27T06:45:09.620 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/aoe/autoload.sh 2024-03-27T06:45:09.621 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/aoe/status.sh 2024-03-27T06:45:09.622 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/aoe/todo.txt 2024-03-27T06:45:09.627 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/aoe/udev-install.sh 2024-03-27T06:45:09.627 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/aoe/udev.txt 2024-03-27T06:45:09.627 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/applying-patches.txt 2024-03-27T06:45:09.627 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/ 2024-03-27T06:45:09.627 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/00-INDEX 2024-03-27T06:45:09.628 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Atmel/ 2024-03-27T06:45:09.629 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Atmel/README 2024-03-27T06:45:09.630 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Booting 2024-03-27T06:45:09.631 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/CCN.txt 2024-03-27T06:45:09.632 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/IXP4xx 2024-03-27T06:45:09.633 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Interrupts 2024-03-27T06:45:09.634 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Makefile 2024-03-27T06:45:09.635 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Marvell/ 2024-03-27T06:45:09.636 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Marvell/README 2024-03-27T06:45:09.637 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Netwinder 2024-03-27T06:45:09.638 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/OMAP/ 2024-03-27T06:45:09.639 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/OMAP/DSS 2024-03-27T06:45:09.640 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/OMAP/omap_pm 2024-03-27T06:45:09.641 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Porting 2024-03-27T06:45:09.642 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/README 2024-03-27T06:45:09.643 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/ 2024-03-27T06:45:09.644 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/ADSBitsy 2024-03-27T06:45:09.645 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/Assabet 2024-03-27T06:45:09.646 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/Brutus 2024-03-27T06:45:09.647 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/CERF 2024-03-27T06:45:09.648 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/FreeBird 2024-03-27T06:45:09.649 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsClient 2024-03-27T06:45:09.650 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsMaster 2024-03-27T06:45:09.651 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/HUW_WEBPANEL 2024-03-27T06:45:09.652 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/Itsy 2024-03-27T06:45:09.653 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/LART 2024-03-27T06:45:09.654 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/PLEB 2024-03-27T06:45:09.655 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/Pangolin 2024-03-27T06:45:09.656 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/Tifon 2024-03-27T06:45:09.657 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/Victor 2024-03-27T06:45:09.658 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/Yopy 2024-03-27T06:45:09.660 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/empeg 2024-03-27T06:45:09.660 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/nanoEngine 2024-03-27T06:45:09.661 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SA1100/serial_UART 2024-03-27T06:45:09.662 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/ 2024-03-27T06:45:09.663 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/.gitignore 2024-03-27T06:45:09.664 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/Makefile 2024-03-27T06:45:09.665 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/vrl4.c 2024-03-27T06:45:09.666 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-mmcif.txt 2024-03-27T06:45:09.667 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-sdhi.txt 2024-03-27T06:45:09.668 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SPEAr/ 2024-03-27T06:45:09.669 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/SPEAr/overview.txt 2024-03-27T06:45:09.670 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/ 2024-03-27T06:45:09.671 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/CPUfreq.txt 2024-03-27T06:45:09.673 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/EB2410ITX.txt 2024-03-27T06:45:09.674 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/GPIO.txt 2024-03-27T06:45:09.676 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/H1940.txt 2024-03-27T06:45:09.678 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/NAND.txt 2024-03-27T06:45:09.679 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Overview.txt 2024-03-27T06:45:09.681 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2412.txt 2024-03-27T06:45:09.683 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2413.txt 2024-03-27T06:45:09.684 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/SMDK2440.txt 2024-03-27T06:45:09.686 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Suspend.txt 2024-03-27T06:45:09.688 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/USB-Host.txt 2024-03-27T06:45:09.689 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung/ 2024-03-27T06:45:09.690 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung/GPIO.txt 2024-03-27T06:45:09.691 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung/Overview.txt 2024-03-27T06:45:09.692 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Samsung/clksrc-change-registers.awk 2024-03-27T06:45:09.693 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/Setup 2024-03-27T06:45:09.694 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/VFP/ 2024-03-27T06:45:09.696 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/VFP/release-notes.txt 2024-03-27T06:45:09.697 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/cluster-pm-race-avoidance.txt 2024-03-27T06:45:09.699 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/firmware.txt 2024-03-27T06:45:09.700 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/kernel_mode_neon.txt 2024-03-27T06:45:09.701 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/kernel_user_helpers.txt 2024-03-27T06:45:09.702 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/mem_alignment 2024-03-27T06:45:09.703 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/memory.txt 2024-03-27T06:45:09.704 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/msm/ 2024-03-27T06:45:09.705 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/msm/gpiomux.txt 2024-03-27T06:45:09.707 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/nwfpe/ 2024-03-27T06:45:09.708 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/nwfpe/NOTES 2024-03-27T06:45:09.709 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/nwfpe/README 2024-03-27T06:45:09.711 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/nwfpe/README.FPE 2024-03-27T06:45:09.712 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/nwfpe/TODO 2024-03-27T06:45:09.713 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/pxa/ 2024-03-27T06:45:09.716 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/pxa/mfp.txt 2024-03-27T06:45:09.717 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/sti/ 2024-03-27T06:45:09.718 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/sti/overview.txt 2024-03-27T06:45:09.720 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/sti/stih407-overview.txt 2024-03-27T06:45:09.721 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/sti/stih415-overview.txt 2024-03-27T06:45:09.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:09 smithi156 ceph-mon[36261]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-03-27T06:45:09.723 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/sti/stih416-overview.txt 2024-03-27T06:45:09.724 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/sti/stih418-overview.txt 2024-03-27T06:45:09.726 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/sunxi/ 2024-03-27T06:45:09.728 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/sunxi/README 2024-03-27T06:45:09.729 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/sunxi/clocks.txt 2024-03-27T06:45:09.731 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/swp_emulation 2024-03-27T06:45:09.732 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/tcm.txt 2024-03-27T06:45:09.734 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/uefi.txt 2024-03-27T06:45:09.735 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm/vlocks.txt 2024-03-27T06:45:09.737 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm64/ 2024-03-27T06:45:09.739 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm64/booting.txt 2024-03-27T06:45:09.740 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm64/legacy_instructions.txt 2024-03-27T06:45:09.742 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm64/memory.txt 2024-03-27T06:45:09.743 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/arm64/tagged-pointers.txt 2024-03-27T06:45:09.745 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/assoc_array.txt 2024-03-27T06:45:09.747 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/atomic_ops.txt 2024-03-27T06:45:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/auxdisplay/ 2024-03-27T06:45:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/auxdisplay/.gitignore 2024-03-27T06:45:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/auxdisplay/Makefile 2024-03-27T06:45:09.753 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b 2024-03-27T06:45:09.755 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b-example.c 2024-03-27T06:45:09.756 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/auxdisplay/ks0108 2024-03-27T06:45:09.758 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/backlight/ 2024-03-27T06:45:09.760 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/backlight/lp855x-driver.txt 2024-03-27T06:45:09.761 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/bad_memory.txt 2024-03-27T06:45:09.763 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/basic_profiling.txt 2024-03-27T06:45:09.765 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/bcache.txt 2024-03-27T06:45:09.766 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/binfmt_misc.txt 2024-03-27T06:45:09.768 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blackfin/ 2024-03-27T06:45:09.769 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blackfin/00-INDEX 2024-03-27T06:45:09.771 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blackfin/Makefile 2024-03-27T06:45:09.772 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blackfin/bfin-gpio-notes.txt 2024-03-27T06:45:09.774 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blackfin/bfin-spi-notes.txt 2024-03-27T06:45:09.776 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blackfin/gptimers-example.c 2024-03-27T06:45:09.778 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/ 2024-03-27T06:45:09.779 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/00-INDEX 2024-03-27T06:45:09.781 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/biodoc.txt 2024-03-27T06:45:09.782 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/biovecs.txt 2024-03-27T06:45:09.784 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/capability.txt 2024-03-27T06:45:09.785 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/cfq-iosched.txt 2024-03-27T06:45:09.787 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/cmdline-partition.txt 2024-03-27T06:45:09.788 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/data-integrity.txt 2024-03-27T06:45:09.790 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/deadline-iosched.txt 2024-03-27T06:45:09.791 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/ioprio.txt 2024-03-27T06:45:09.792 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/null_blk.txt 2024-03-27T06:45:09.793 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/queue-sysfs.txt 2024-03-27T06:45:09.794 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/request.txt 2024-03-27T06:45:09.796 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/stat.txt 2024-03-27T06:45:09.797 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/switching-sched.txt 2024-03-27T06:45:09.799 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/block/writeback_cache_control.txt 2024-03-27T06:45:09.800 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/ 2024-03-27T06:45:09.802 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/00-INDEX 2024-03-27T06:45:09.803 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/README.DAC960 2024-03-27T06:45:09.804 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/cciss.txt 2024-03-27T06:45:09.805 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/cpqarray.txt 2024-03-27T06:45:09.807 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/drbd/ 2024-03-27T06:45:09.808 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-8.3-data-packets.svg 2024-03-27T06:45:09.809 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-data-packets.svg 2024-03-27T06:45:09.811 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/drbd/README.txt 2024-03-27T06:45:09.812 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/drbd/conn-states-8.dot 2024-03-27T06:45:09.814 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/drbd/data-structure-v9.txt 2024-03-27T06:45:09.815 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/drbd/disk-states-8.dot 2024-03-27T06:45:09.817 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/drbd/drbd-connection-state-overview.dot 2024-03-27T06:45:09.819 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/drbd/node-states-8.dot 2024-03-27T06:45:09.820 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/floppy.txt 2024-03-27T06:45:09.822 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/mflash.txt 2024-03-27T06:45:09.824 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/nbd.txt 2024-03-27T06:45:09.825 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/paride.txt 2024-03-27T06:45:09.827 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/ramdisk.txt 2024-03-27T06:45:09.829 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/blockdev/zram.txt 2024-03-27T06:45:09.835 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/braille-console.txt 2024-03-27T06:45:09.836 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/bt8xxgpio.txt 2024-03-27T06:45:09.836 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/btmrvl.txt 2024-03-27T06:45:09.836 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/bus-devices/ 2024-03-27T06:45:09.837 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt 2024-03-27T06:45:09.839 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/bus-virt-phys-mapping.txt 2024-03-27T06:45:09.840 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cachetlb.txt 2024-03-27T06:45:09.842 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cdrom/ 2024-03-27T06:45:09.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:09 smithi170 ceph-mon[36438]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-03-27T06:45:09.844 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cdrom/00-INDEX 2024-03-27T06:45:09.845 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cdrom/Makefile 2024-03-27T06:45:09.847 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cdrom/cdrom-standard.tex 2024-03-27T06:45:09.848 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cdrom/ide-cd 2024-03-27T06:45:09.850 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cdrom/packet-writing.txt 2024-03-27T06:45:09.852 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/ 2024-03-27T06:45:09.853 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/00-INDEX 2024-03-27T06:45:09.855 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/blkio-controller.txt 2024-03-27T06:45:09.856 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/cgroups.txt 2024-03-27T06:45:09.857 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/cpuacct.txt 2024-03-27T06:45:09.859 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/cpusets.txt 2024-03-27T06:45:09.860 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/devices.txt 2024-03-27T06:45:09.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:09 smithi016 ceph-mon[31908]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-03-27T06:45:09.862 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/freezer-subsystem.txt 2024-03-27T06:45:09.864 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/hugetlb.txt 2024-03-27T06:45:09.865 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/memcg_test.txt 2024-03-27T06:45:09.867 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/memory.txt 2024-03-27T06:45:09.868 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/net_cls.txt 2024-03-27T06:45:09.870 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/net_prio.txt 2024-03-27T06:45:09.872 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cgroups/unified-hierarchy.txt 2024-03-27T06:45:09.873 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/circular-buffers.txt 2024-03-27T06:45:09.875 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/clk.txt 2024-03-27T06:45:09.877 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/coccinelle.txt 2024-03-27T06:45:09.878 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/connector/ 2024-03-27T06:45:09.880 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/connector/.gitignore 2024-03-27T06:45:09.882 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/connector/Makefile 2024-03-27T06:45:09.883 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/connector/cn_test.c 2024-03-27T06:45:09.885 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/connector/connector.txt 2024-03-27T06:45:09.886 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/connector/ucon.c 2024-03-27T06:45:09.888 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/console/ 2024-03-27T06:45:09.889 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/console/console.txt 2024-03-27T06:45:09.891 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/ 2024-03-27T06:45:09.892 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/amd-powernow.txt 2024-03-27T06:45:09.894 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/boost.txt 2024-03-27T06:45:09.895 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/core.txt 2024-03-27T06:45:09.897 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/cpu-drivers.txt 2024-03-27T06:45:09.898 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-nforce2.txt 2024-03-27T06:45:09.899 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-stats.txt 2024-03-27T06:45:09.900 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/governors.txt 2024-03-27T06:45:09.901 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/index.txt 2024-03-27T06:45:09.902 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/intel-pstate.txt 2024-03-27T06:45:09.903 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/pcc-cpufreq.txt 2024-03-27T06:45:09.904 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-freq/user-guide.txt 2024-03-27T06:45:09.905 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-hotplug.txt 2024-03-27T06:45:09.907 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpu-load.txt 2024-03-27T06:45:09.908 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpuidle/ 2024-03-27T06:45:09.909 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpuidle/core.txt 2024-03-27T06:45:09.911 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpuidle/driver.txt 2024-03-27T06:45:09.912 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpuidle/governor.txt 2024-03-27T06:45:09.914 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cpuidle/sysfs.txt 2024-03-27T06:45:09.915 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cputopology.txt 2024-03-27T06:45:09.917 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/crc32.txt 2024-03-27T06:45:09.919 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cris/ 2024-03-27T06:45:09.920 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/cris/README 2024-03-27T06:45:09.922 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/crypto/ 2024-03-27T06:45:09.923 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/crypto/api-intro.txt 2024-03-27T06:45:09.925 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/crypto/asymmetric-keys.txt 2024-03-27T06:45:09.926 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/crypto/async-tx-api.txt 2024-03-27T06:45:09.928 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/crypto/crypto-API-userspace.txt 2024-03-27T06:45:09.930 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/crypto/descore-readme.txt 2024-03-27T06:45:09.931 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dcdbas.txt 2024-03-27T06:45:09.933 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/debugging-modules.txt 2024-03-27T06:45:09.935 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/debugging-via-ohci1394.txt 2024-03-27T06:45:09.937 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dell_rbu.txt 2024-03-27T06:45:09.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/development-process/ 2024-03-27T06:45:09.940 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/development-process/1.Intro 2024-03-27T06:45:09.941 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/development-process/2.Process 2024-03-27T06:45:09.943 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/development-process/3.Early-stage 2024-03-27T06:45:09.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/development-process/4.Coding 2024-03-27T06:45:09.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/development-process/5.Posting 2024-03-27T06:45:09.948 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/development-process/6.Followthrough 2024-03-27T06:45:09.949 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/development-process/7.AdvancedTopics 2024-03-27T06:45:09.951 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/development-process/8.Conclusion 2024-03-27T06:45:09.952 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/ 2024-03-27T06:45:09.954 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/cache-policies.txt 2024-03-27T06:45:09.955 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/cache.txt 2024-03-27T06:45:09.957 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/delay.txt 2024-03-27T06:45:09.958 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/dm-crypt.txt 2024-03-27T06:45:09.960 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/dm-flakey.txt 2024-03-27T06:45:09.961 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/dm-io.txt 2024-03-27T06:45:09.963 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/dm-log.txt 2024-03-27T06:45:09.964 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/dm-queue-length.txt 2024-03-27T06:45:09.966 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/dm-raid.txt 2024-03-27T06:45:09.968 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/dm-service-time.txt 2024-03-27T06:45:09.969 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/dm-uevent.txt 2024-03-27T06:45:09.971 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/era.txt 2024-03-27T06:45:09.972 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/kcopyd.txt 2024-03-27T06:45:09.974 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/linear.txt 2024-03-27T06:45:09.975 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/persistent-data.txt 2024-03-27T06:45:09.977 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/snapshot.txt 2024-03-27T06:45:09.978 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/statistics.txt 2024-03-27T06:45:09.980 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/striped.txt 2024-03-27T06:45:09.982 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/switch.txt 2024-03-27T06:45:09.983 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/thin-provisioning.txt 2024-03-27T06:45:09.985 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/verity.txt 2024-03-27T06:45:09.987 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/device-mapper/zero.txt 2024-03-27T06:45:09.988 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devices.txt 2024-03-27T06:45:09.991 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/ 2024-03-27T06:45:09.992 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/00-INDEX 2024-03-27T06:45:09.994 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ 2024-03-27T06:45:09.995 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ABI.txt 2024-03-27T06:45:09.996 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/ 2024-03-27T06:45:09.998 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/interrupts.txt 2024-03-27T06:45:10.000 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/pmu.txt 2024-03-27T06:45:10.001 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ 2024-03-27T06:45:10.003 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/adapteva.txt 2024-03-27T06:45:10.004 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/ 2024-03-27T06:45:10.006 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.txt 2024-03-27T06:45:10.007 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-sdram-edac.txt 2024-03-27T06:45:10.008 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-system.txt 2024-03-27T06:45:10.010 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/amlogic.txt 2024-03-27T06:45:10.012 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arch_timer.txt 2024-03-27T06:45:10.013 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arm-boards 2024-03-27T06:45:10.015 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt 2024-03-27T06:45:10.017 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp.txt 2024-03-27T06:45:10.018 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-375.txt 2024-03-27T06:45:10.020 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-380-mpcore-soc-ctrl.txt 2024-03-27T06:45:10.021 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-38x.txt 2024-03-27T06:45:10.023 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-cpu-reset.txt 2024-03-27T06:45:10.025 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armadeus.txt 2024-03-27T06:45:10.027 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-at91.txt 2024-03-27T06:45:10.028 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-pmc.txt 2024-03-27T06:45:10.030 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/axxia.txt 2024-03-27T06:45:10.031 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/ 2024-03-27T06:45:10.033 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm11351.txt 2024-03-27T06:45:10.034 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm21664.txt 2024-03-27T06:45:10.036 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm63138.txt 2024-03-27T06:45:10.038 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/brcm,bcm11351-cpu-method 2024-03-27T06:45:10.042 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/cygnus.txt 2024-03-27T06:45:10.043 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-resetmgr.txt 2024-03-27T06:45:10.043 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-timer.txt 2024-03-27T06:45:10.045 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-wdt.txt 2024-03-27T06:45:10.046 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm2835.txt 2024-03-27T06:45:10.047 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm4708.txt 2024-03-27T06:45:10.049 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/brcm-brcmstb.txt 2024-03-27T06:45:10.050 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda.txt 2024-03-27T06:45:10.052 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/ 2024-03-27T06:45:10.054 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/combophy.txt 2024-03-27T06:45:10.055 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/l2ecc.txt 2024-03-27T06:45:10.057 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/mem-ctrlr.txt 2024-03-27T06:45:10.058 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cavium-thunder.txt 2024-03-27T06:45:10.060 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cci.txt 2024-03-27T06:45:10.061 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ccn.txt 2024-03-27T06:45:10.063 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coherency-fabric.txt 2024-03-27T06:45:10.065 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coresight.txt 2024-03-27T06:45:10.066 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/ 2024-03-27T06:45:10.068 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/marvell,berlin-smp 2024-03-27T06:45:10.069 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpus.txt 2024-03-27T06:45:10.071 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci.txt 2024-03-27T06:45:10.072 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/ 2024-03-27T06:45:10.073 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/cp-intc.txt 2024-03-27T06:45:10.074 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/digicolor.txt 2024-03-27T06:45:10.075 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/ 2024-03-27T06:45:10.076 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/power_domain.txt 2024-03-27T06:45:10.077 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/smp-sysram.txt 2024-03-27T06:45:10.078 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/ 2024-03-27T06:45:10.079 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/tlm,trusted-foundations.txt 2024-03-27T06:45:10.080 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fsl.txt 2024-03-27T06:45:10.081 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fw-cfg.txt 2024-03-27T06:45:10.082 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/geniatech.txt 2024-03-27T06:45:10.083 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic-v3.txt 2024-03-27T06:45:10.084 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic.txt 2024-03-27T06:45:10.085 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/global_timer.txt 2024-03-27T06:45:10.086 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/ 2024-03-27T06:45:10.087 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt 2024-03-27T06:45:10.088 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/idle-states.txt 2024-03-27T06:45:10.089 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/insignal-boards.txt 2024-03-27T06:45:10.090 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/ 2024-03-27T06:45:10.091 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/keystone.txt 2024-03-27T06:45:10.092 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/kirkwood.txt 2024-03-27T06:45:10.093 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/l2cc.txt 2024-03-27T06:45:10.094 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx-mic.txt 2024-03-27T06:45:10.095 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx.txt 2024-03-27T06:45:10.096 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,berlin.txt 2024-03-27T06:45:10.097 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,dove.txt 2024-03-27T06:45:10.098 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,kirkwood.txt 2024-03-27T06:45:10.099 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek.txt 2024-03-27T06:45:10.100 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/ 2024-03-27T06:45:10.101 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/mediatek,sysirq.txt 2024-03-27T06:45:10.102 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/moxart.txt 2024-03-27T06:45:10.103 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/ 2024-03-27T06:45:10.104 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/feroceon.txt 2024-03-27T06:45:10.105 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/intc.txt 2024-03-27T06:45:10.106 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/mrvl.txt 2024-03-27T06:45:10.107 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/tauros2.txt 2024-03-27T06:45:10.108 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/timer.txt 2024-03-27T06:45:10.110 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ 2024-03-27T06:45:10.111 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,kpss-acc.txt 2024-03-27T06:45:10.113 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,saw2.txt 2024-03-27T06:45:10.114 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ssbi.txt 2024-03-27T06:45:10.116 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/timer.txt 2024-03-27T06:45:10.117 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mvebu-system-controller.txt 2024-03-27T06:45:10.119 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/nspire.txt 2024-03-27T06:45:10.121 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/olimex.txt 2024-03-27T06:45:10.122 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/ 2024-03-27T06:45:10.124 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/counter.txt 2024-03-27T06:45:10.125 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/crossbar.txt 2024-03-27T06:45:10.127 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dmm.txt 2024-03-27T06:45:10.129 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dsp.txt 2024-03-27T06:45:10.130 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/intc.txt 2024-03-27T06:45:10.132 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/iva.txt 2024-03-27T06:45:10.134 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/l3-noc.txt 2024-03-27T06:45:10.135 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/mpu.txt 2024-03-27T06:45:10.137 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/omap.txt 2024-03-27T06:45:10.139 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/prcm.txt 2024-03-27T06:45:10.141 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/timer.txt 2024-03-27T06:45:10.142 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/picoxcell.txt 2024-03-27T06:45:10.144 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/pmu.txt 2024-03-27T06:45:10.145 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/primecell.txt 2024-03-27T06:45:10.147 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/psci.txt 2024-03-27T06:45:10.149 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip.txt 2024-03-27T06:45:10.151 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/ 2024-03-27T06:45:10.152 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu-sram.txt 2024-03-27T06:45:10.153 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu.txt 2024-03-27T06:45:10.154 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/smp-sram.txt 2024-03-27T06:45:10.155 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rtsm-dcscb.txt 2024-03-27T06:45:10.156 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung-boards.txt 2024-03-27T06:45:10.158 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/ 2024-03-27T06:45:10.159 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-adc.txt 2024-03-27T06:45:10.161 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.txt 2024-03-27T06:45:10.162 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/interrupt-combiner.txt 2024-03-27T06:45:10.164 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/pmu.txt 2024-03-27T06:45:10.165 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/sysreg.txt 2024-03-27T06:45:10.167 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/shmobile.txt 2024-03-27T06:45:10.169 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sirf.txt 2024-03-27T06:45:10.170 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-misc.txt 2024-03-27T06:45:10.172 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-timer.txt 2024-03-27T06:45:10.173 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear.txt 2024-03-27T06:45:10.175 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/ 2024-03-27T06:45:10.176 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/shirq.txt 2024-03-27T06:45:10.178 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sprd.txt 2024-03-27T06:45:10.180 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-nomadik.txt 2024-03-27T06:45:10.181 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-u300.txt 2024-03-27T06:45:10.182 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sti.txt 2024-03-27T06:45:10.183 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sunxi.txt 2024-03-27T06:45:10.184 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra.txt 2024-03-27T06:45:10.185 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/ 2024-03-27T06:45:10.186 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-ahb.txt 2024-03-27T06:45:10.187 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-emc.txt 2024-03-27T06:45:10.188 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-flowctrl.txt 2024-03-27T06:45:10.189 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-mc.txt 2024-03-27T06:45:10.190 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-pmc.txt 2024-03-27T06:45:10.191 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra30-mc.txt 2024-03-27T06:45:10.192 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/topology.txt 2024-03-27T06:45:10.193 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/twd.txt 2024-03-27T06:45:10.195 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/ 2024-03-27T06:45:10.197 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/power_domain.txt 2024-03-27T06:45:10.198 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-fpga-irq.txt 2024-03-27T06:45:10.200 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-sysreg.txt 2024-03-27T06:45:10.201 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-scc.txt 2024-03-27T06:45:10.203 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-sysreg.txt 2024-03-27T06:45:10.204 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress.txt 2024-03-27T06:45:10.205 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vic.txt 2024-03-27T06:45:10.206 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500.txt 2024-03-27T06:45:10.207 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/ 2024-03-27T06:45:10.208 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-intc.txt 2024-03-27T06:45:10.516 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-pmc.txt 2024-03-27T06:45:10.518 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-timer.txt 2024-03-27T06:45:10.519 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xen.txt 2024-03-27T06:45:10.520 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xilinx.txt 2024-03-27T06:45:10.521 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ 2024-03-27T06:45:10.522 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-platform.txt 2024-03-27T06:45:10.523 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-st.txt 2024-03-27T06:45:10.525 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/apm-xgene.txt 2024-03-27T06:45:10.527 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/atmel-at91_cf.txt 2024-03-27T06:45:10.528 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/cavium-compact-flash.txt 2024-03-27T06:45:10.530 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/exynos-sata.txt 2024-03-27T06:45:10.532 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/fsl-sata.txt 2024-03-27T06:45:10.533 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-pata.txt 2024-03-27T06:45:10.535 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-sata.txt 2024-03-27T06:45:10.537 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/marvell.txt 2024-03-27T06:45:10.538 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/pata-arasan.txt 2024-03-27T06:45:10.540 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/qcom-sata.txt 2024-03-27T06:45:10.541 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_highbank.txt 2024-03-27T06:45:10.543 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_rcar.txt 2024-03-27T06:45:10.545 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/tegra-sata.txt 2024-03-27T06:45:10.547 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/btmrvl.txt 2024-03-27T06:45:10.548 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ 2024-03-27T06:45:10.550 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/bcma.txt 2024-03-27T06:45:10.551 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/brcm,gisb-arb.txt 2024-03-27T06:45:10.553 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/imx-weim.txt 2024-03-27T06:45:10.554 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/mvebu-mbus.txt 2024-03-27T06:45:10.556 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/omap-ocp2scp.txt 2024-03-27T06:45:10.558 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ti-gpmc.txt 2024-03-27T06:45:10.560 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/ 2024-03-27T06:45:10.561 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/clocks.txt 2024-03-27T06:45:10.563 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/dscr.txt 2024-03-27T06:45:10.564 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/emifa.txt 2024-03-27T06:45:10.566 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/interrupt.txt 2024-03-27T06:45:10.568 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/soc.txt 2024-03-27T06:45:10.569 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/timer64.txt 2024-03-27T06:45:10.571 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/chosen.txt 2024-03-27T06:45:10.573 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ 2024-03-27T06:45:10.574 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/alphascale,acc.txt 2024-03-27T06:45:10.576 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/altr_socfpga.txt 2024-03-27T06:45:10.577 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/arm-integrator.txt 2024-03-27T06:45:10.579 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/at91-clock.txt 2024-03-27T06:45:10.580 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/axi-clkgen.txt 2024-03-27T06:45:10.582 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-cygnus-clock.txt 2024-03-27T06:45:10.583 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt 2024-03-27T06:45:10.585 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/calxeda.txt 2024-03-27T06:45:10.587 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-exynos-audss.txt 2024-03-27T06:45:10.588 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-palmas-clk32kg-clocks.txt 2024-03-27T06:45:10.590 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-s5pv210-audss.txt 2024-03-27T06:45:10.592 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clock-bindings.txt 2024-03-27T06:45:10.593 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clps711x-clock.txt 2024-03-27T06:45:10.595 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/efm32-clock.txt 2024-03-27T06:45:10.596 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/emev2-clock.txt 2024-03-27T06:45:10.598 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos3250-clock.txt 2024-03-27T06:45:10.600 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4-clock.txt 2024-03-27T06:45:10.601 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4415-clock.txt 2024-03-27T06:45:10.603 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5250-clock.txt 2024-03-27T06:45:10.604 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5260-clock.txt 2024-03-27T06:45:10.606 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5410-clock.txt 2024-03-27T06:45:10.607 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5420-clock.txt 2024-03-27T06:45:10.608 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5440-clock.txt 2024-03-27T06:45:10.609 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos7-clock.txt 2024-03-27T06:45:10.610 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-clock.txt 2024-03-27T06:45:10.611 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-factor-clock.txt 2024-03-27T06:45:10.612 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/gpio-gate-clock.txt 2024-03-27T06:45:10.613 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hi3620-clock.txt 2024-03-27T06:45:10.614 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt 2024-03-27T06:45:10.615 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx1-clock.txt 2024-03-27T06:45:10.617 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx21-clock.txt 2024-03-27T06:45:10.618 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx23-clock.txt 2024-03-27T06:45:10.620 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx25-clock.txt 2024-03-27T06:45:10.622 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx27-clock.txt 2024-03-27T06:45:10.623 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx28-clock.txt 2024-03-27T06:45:10.625 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx31-clock.txt 2024-03-27T06:45:10.627 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx35-clock.txt 2024-03-27T06:45:10.628 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx5-clock.txt 2024-03-27T06:45:10.630 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6q-clock.txt 2024-03-27T06:45:10.631 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sl-clock.txt 2024-03-27T06:45:10.633 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sx-clock.txt 2024-03-27T06:45:10.635 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-gate.txt 2024-03-27T06:45:10.636 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-pll.txt 2024-03-27T06:45:10.638 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt 2024-03-27T06:45:10.639 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,mmp2.txt 2024-03-27T06:45:10.640 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa168.txt 2024-03-27T06:45:10.641 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa910.txt 2024-03-27T06:45:10.643 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77686.txt 2024-03-27T06:45:10.644 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77802.txt 2024-03-27T06:45:10.646 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/moxa,moxart-clock.txt 2024-03-27T06:45:10.650 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt 2024-03-27T06:45:10.651 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-corediv-clock.txt 2024-03-27T06:45:10.651 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt 2024-03-27T06:45:10.652 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-gated-clock.txt 2024-03-27T06:45:10.654 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nspire-clock.txt 2024-03-27T06:45:10.656 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra114-car.txt 2024-03-27T06:45:10.657 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra124-car.txt 2024-03-27T06:45:10.659 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.txt 2024-03-27T06:45:10.661 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra30-car.txt 2024-03-27T06:45:10.662 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/prima2-clock.txt 2024-03-27T06:45:10.664 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/pxa-clock.txt 2024-03-27T06:45:10.665 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,gcc.txt 2024-03-27T06:45:10.667 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,lcc.txt 2024-03-27T06:45:10.669 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,mmcc.txt 2024-03-27T06:45:10.670 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qoriq-clock.txt 2024-03-27T06:45:10.672 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-div6-clocks.txt 2024-03-27T06:45:10.674 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt 2024-03-27T06:45:10.675 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a73a4-cpg-clocks.txt 2024-03-27T06:45:10.677 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt 2024-03-27T06:45:10.678 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt 2024-03-27T06:45:10.680 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rcar-gen2-cpg-clocks.txt 2024-03-27T06:45:10.682 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rz-cpg-clocks.txt 2024-03-27T06:45:10.683 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,sh73a0-cpg-clocks.txt 2024-03-27T06:45:10.684 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt 2024-03-27T06:45:10.686 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt 2024-03-27T06:45:10.688 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip.txt 2024-03-27T06:45:10.689 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2410-clock.txt 2024-03-27T06:45:10.691 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2412-clock.txt 2024-03-27T06:45:10.692 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2443-clock.txt 2024-03-27T06:45:10.694 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c64xx-clock.txt 2024-03-27T06:45:10.695 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt 2024-03-27T06:45:10.696 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si5351.txt 2024-03-27T06:45:10.697 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si570.txt 2024-03-27T06:45:10.698 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st,nomadik.txt 2024-03-27T06:45:10.699 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/ 2024-03-27T06:45:10.700 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-divmux.txt 2024-03-27T06:45:10.701 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-mux.txt 2024-03-27T06:45:10.702 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-pll.txt 2024-03-27T06:45:10.703 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-prediv.txt 2024-03-27T06:45:10.704 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-vcc.txt 2024-03-27T06:45:10.705 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen.txt 2024-03-27T06:45:10.706 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,flexgen.txt 2024-03-27T06:45:10.707 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,quadfs.txt 2024-03-27T06:45:10.709 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ste-u300-syscon-clock.txt 2024-03-27T06:45:10.710 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/sunxi.txt 2024-03-27T06:45:10.712 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti,cdce706.txt 2024-03-27T06:45:10.714 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti-keystone-pllctrl.txt 2024-03-27T06:45:10.714 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/ 2024-03-27T06:45:10.715 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/apll.txt 2024-03-27T06:45:10.716 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/autoidle.txt 2024-03-27T06:45:10.717 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/clockdomain.txt 2024-03-27T06:45:10.718 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/composite.txt 2024-03-27T06:45:10.719 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/divider.txt 2024-03-27T06:45:10.720 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dpll.txt 2024-03-27T06:45:10.721 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dra7-atl.txt 2024-03-27T06:45:10.722 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fapll.txt 2024-03-27T06:45:10.723 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fixed-factor-clock.txt 2024-03-27T06:45:10.724 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/gate.txt 2024-03-27T06:45:10.725 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/interface.txt 2024-03-27T06:45:10.726 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/mux.txt 2024-03-27T06:45:10.727 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vf610-clock.txt 2024-03-27T06:45:10.728 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vt8500.txt 2024-03-27T06:45:10.729 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/xgene.txt 2024-03-27T06:45:10.730 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/zynq-7000.txt 2024-03-27T06:45:10.731 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/ 2024-03-27T06:45:10.732 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/arm_big_little_dt.txt 2024-03-27T06:45:10.733 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-dt.txt 2024-03-27T06:45:10.734 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-exynos5440.txt 2024-03-27T06:45:10.735 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-spear.txt 2024-03-27T06:45:10.736 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/ 2024-03-27T06:45:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/amd-ccp.txt 2024-03-27T06:45:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/atmel-crypto.txt 2024-03-27T06:45:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-dcp.txt 2024-03-27T06:45:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-imx-sahara.txt 2024-03-27T06:45:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec2.txt 2024-03-27T06:45:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec4.txt 2024-03-27T06:45:10.744 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec6.txt 2024-03-27T06:45:10.746 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/mv_cesa.txt 2024-03-27T06:45:10.747 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-aes.txt 2024-03-27T06:45:10.748 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-des.txt 2024-03-27T06:45:10.749 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-sham.txt 2024-03-27T06:45:10.750 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/picochip-spacc.txt 2024-03-27T06:45:10.751 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/qcom-qce.txt 2024-03-27T06:45:10.752 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/samsung-sss.txt 2024-03-27T06:45:10.753 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/ 2024-03-27T06:45:10.754 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/ 2024-03-27T06:45:10.755 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/exynos-ppmu.txt 2024-03-27T06:45:10.756 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ 2024-03-27T06:45:10.758 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/arm-pl330.txt 2024-03-27T06:45:10.759 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-dma.txt 2024-03-27T06:45:10.761 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-xdma.txt 2024-03-27T06:45:10.763 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/bcm2835-dma.txt 2024-03-27T06:45:10.764 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/dma.txt 2024-03-27T06:45:10.766 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-edma.txt 2024-03-27T06:45:10.767 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-dma.txt 2024-03-27T06:45:10.768 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-sdma.txt 2024-03-27T06:45:10.769 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-mxs-dma.txt 2024-03-27T06:45:10.770 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/img-mdc-dma.txt 2024-03-27T06:45:10.771 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/k3dma.txt 2024-03-27T06:45:10.772 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mmp-dma.txt 2024-03-27T06:45:10.773 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/moxa,moxart-dma.txt 2024-03-27T06:45:10.774 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mpc512x-dma.txt 2024-03-27T06:45:10.775 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mv-xor.txt 2024-03-27T06:45:10.776 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/nbpfaxi.txt 2024-03-27T06:45:10.778 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_adm.txt 2024-03-27T06:45:10.780 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt 2024-03-27T06:45:10.781 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/rcar-audmapp.txt 2024-03-27T06:45:10.783 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt 2024-03-27T06:45:10.785 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/shdma.txt 2024-03-27T06:45:10.786 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sirfsoc-dma.txt 2024-03-27T06:45:10.787 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/snps-dma.txt 2024-03-27T06:45:10.788 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-coh901318.txt 2024-03-27T06:45:10.790 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-dma40.txt 2024-03-27T06:45:10.790 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sun6i-dma.txt 2024-03-27T06:45:10.791 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/tegra20-apbdma.txt 2024-03-27T06:45:10.792 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ti-edma.txt 2024-03-27T06:45:10.793 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/ 2024-03-27T06:45:10.794 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt 2024-03-27T06:45:10.795 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt 2024-03-27T06:45:10.796 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/ 2024-03-27T06:45:10.797 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/ 2024-03-27T06:45:10.798 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/marvell,dove-lcd.txt 2024-03-27T06:45:10.799 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/ 2024-03-27T06:45:10.800 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/hlcdc-dc.txt 2024-03-27T06:45:10.801 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/ 2024-03-27T06:45:10.802 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/dw_hdmi.txt 2024-03-27T06:45:10.803 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/ 2024-03-27T06:45:10.804 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/tda998x.txt 2024-03-27T06:45:10.805 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ 2024-03-27T06:45:10.805 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/fsl-imx-drm.txt 2024-03-27T06:45:10.806 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/hdmi.txt 2024-03-27T06:45:10.808 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ldb.txt 2024-03-27T06:45:10.808 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/ 2024-03-27T06:45:10.809 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/gpu.txt 2024-03-27T06:45:10.810 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/hdmi.txt 2024-03-27T06:45:10.811 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/mdp.txt 2024-03-27T06:45:10.812 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/ 2024-03-27T06:45:10.813 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/panel.txt 2024-03-27T06:45:10.814 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/slave.txt 2024-03-27T06:45:10.815 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tfp410.txt 2024-03-27T06:45:10.816 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tilcdc.txt 2024-03-27T06:45:10.817 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/eeprom.txt 2024-03-27T06:45:10.818 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/ 2024-03-27T06:45:10.819 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-palmas.txt 2024-03-27T06:45:10.820 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-rt8973a.txt 2024-03-27T06:45:10.821 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-sm5502.txt 2024-03-27T06:45:10.822 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/ 2024-03-27T06:45:10.823 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/mxsfb.txt 2024-03-27T06:45:10.824 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/sm501fb.txt 2024-03-27T06:45:10.825 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/ 2024-03-27T06:45:10.826 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/altera-socfpga-fpga-mgr.txt 2024-03-27T06:45:10.827 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/ 2024-03-27T06:45:10.828 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt 2024-03-27T06:45:10.829 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/ 2024-03-27T06:45:10.830 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/8xxx_gpio.txt 2024-03-27T06:45:10.831 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/abilis,tb10x-gpio.txt 2024-03-27T06:45:10.832 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cavium-octeon-gpio.txt 2024-03-27T06:45:10.833 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cirrus,clps711x-mctrl-gpio.txt 2024-03-27T06:45:10.834 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fsl-imx-gpio.txt 2024-03-27T06:45:10.835 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fujitsu,mb86s70-gpio.txt 2024-03-27T06:45:10.836 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74x164.txt 2024-03-27T06:45:10.837 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74xx-mmio.txt 2024-03-27T06:45:10.838 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-adnp.txt 2024-03-27T06:45:10.840 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-bcm-kona.txt 2024-03-27T06:45:10.841 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-clps711x.txt 2024-03-27T06:45:10.843 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-davinci.txt 2024-03-27T06:45:10.845 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-dsp-keystone.txt 2024-03-27T06:45:10.846 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-fan.txt 2024-03-27T06:45:10.847 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-grgpio.txt 2024-03-27T06:45:10.849 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-lp3943.txt 2024-03-27T06:45:10.850 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-max732x.txt 2024-03-27T06:45:10.854 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mcp23s08.txt 2024-03-27T06:45:10.854 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mm-lantiq.txt 2024-03-27T06:45:10.855 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-msm.txt 2024-03-27T06:45:10.857 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mvebu.txt 2024-03-27T06:45:10.858 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mxs.txt 2024-03-27T06:45:10.860 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-nmk.txt 2024-03-27T06:45:10.862 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-omap.txt 2024-03-27T06:45:10.863 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-palmas.txt 2024-03-27T06:45:10.865 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pca953x.txt 2024-03-27T06:45:10.866 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pcf857x.txt 2024-03-27T06:45:10.867 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-poweroff.txt 2024-03-27T06:45:10.868 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-restart.txt 2024-03-27T06:45:10.870 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-samsung.txt 2024-03-27T06:45:10.871 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stericsson-coh901.txt 2024-03-27T06:45:10.872 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stmpe.txt 2024-03-27T06:45:10.873 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stp-xway.txt 2024-03-27T06:45:10.874 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-sx150x.txt 2024-03-27T06:45:10.875 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-twl4030.txt 2024-03-27T06:45:10.877 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090-pdc.txt 2024-03-27T06:45:10.878 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090.txt 2024-03-27T06:45:10.879 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-vf610.txt 2024-03-27T06:45:10.880 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene-sb.txt 2024-03-27T06:45:10.881 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene.txt 2024-03-27T06:45:10.882 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xilinx.txt 2024-03-27T06:45:10.883 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zevio.txt 2024-03-27T06:45:10.884 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zynq.txt 2024-03-27T06:45:10.885 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio.txt 2024-03-27T06:45:10.886 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_atmel.txt 2024-03-27T06:45:10.887 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_lpc32xx.txt 2024-03-27T06:45:10.888 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/moxa,moxart-gpio.txt 2024-03-27T06:45:10.889 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/mrvl-gpio.txt 2024-03-27T06:45:10.890 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/nvidia,tegra20-gpio.txt 2024-03-27T06:45:10.891 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/pl061-gpio.txt 2024-03-27T06:45:10.892 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt 2024-03-27T06:45:10.893 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt 2024-03-27T06:45:10.894 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/sodaville.txt 2024-03-27T06:45:10.896 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/spear_spics.txt 2024-03-27T06:45:10.897 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/ 2024-03-27T06:45:10.898 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,gk20a.txt 2024-03-27T06:45:10.899 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,tegra20-host1x.txt 2024-03-27T06:45:10.900 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-g2d.txt 2024-03-27T06:45:10.901 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-rotator.txt 2024-03-27T06:45:10.902 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/st,stih4xx.txt 2024-03-27T06:45:10.903 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/graph.txt 2024-03-27T06:45:10.904 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/ 2024-03-27T06:45:10.905 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/hid-over-i2c.txt 2024-03-27T06:45:10.906 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/ 2024-03-27T06:45:10.907 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/client-devices.txt 2024-03-27T06:45:10.908 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/nokia-modem.txt 2024-03-27T06:45:10.909 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/omap-ssi.txt 2024-03-27T06:45:10.910 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ 2024-03-27T06:45:10.911 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ads1015.txt 2024-03-27T06:45:10.912 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/g762.txt 2024-03-27T06:45:10.913 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ibmpowernv.txt 2024-03-27T06:45:10.915 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/lm90.txt 2024-03-27T06:45:10.915 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ltc2978.txt 2024-03-27T06:45:10.916 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ntc_thermistor.txt 2024-03-27T06:45:10.917 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/pwm-fan.txt 2024-03-27T06:45:10.918 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/vexpress.txt 2024-03-27T06:45:10.919 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/ 2024-03-27T06:45:10.921 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/atmel-trng.txt 2024-03-27T06:45:10.922 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/omap_rng.txt 2024-03-27T06:45:10.924 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/timeriomem_rng.txt 2024-03-27T06:45:10.926 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ 2024-03-27T06:45:10.927 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,bcm2835-i2c.txt 2024-03-27T06:45:10.928 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,iproc-i2c.txt 2024-03-27T06:45:10.930 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt 2024-03-27T06:45:10.932 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-at91.txt 2024-03-27T06:45:10.933 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-axxia.txt 2024-03-27T06:45:10.935 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-bcm-kona.txt 2024-03-27T06:45:10.936 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cadence.txt 2024-03-27T06:45:10.937 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cbus-gpio.txt 2024-03-27T06:45:10.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt 2024-03-27T06:45:10.940 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-davinci.txt 2024-03-27T06:45:10.941 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-designware.txt 2024-03-27T06:45:10.942 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-efm32.txt 2024-03-27T06:45:10.943 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt 2024-03-27T06:45:10.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-gpio.txt 2024-03-27T06:45:10.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-hix5hd2.txt 2024-03-27T06:45:10.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-img-scb.txt 2024-03-27T06:45:10.947 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-imx.txt 2024-03-27T06:45:10.948 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-meson.txt 2024-03-27T06:45:10.949 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mpc.txt 2024-03-27T06:45:10.950 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-gpio.txt 2024-03-27T06:45:10.951 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pca954x.txt 2024-03-27T06:45:10.952 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pinctrl.txt 2024-03-27T06:45:10.954 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux.txt 2024-03-27T06:45:10.955 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt 2024-03-27T06:45:10.957 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mxs.txt 2024-03-27T06:45:10.958 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-nomadik.txt 2024-03-27T06:45:10.959 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-ocores.txt 2024-03-27T06:45:10.961 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-octeon.txt 2024-03-27T06:45:10.963 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-omap.txt 2024-03-27T06:45:10.964 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-opal.txt 2024-03-27T06:45:10.965 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pnx.txt 2024-03-27T06:45:10.967 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa-pci-ce4100.txt 2024-03-27T06:45:10.968 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa.txt 2024-03-27T06:45:10.969 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rcar.txt 2024-03-27T06:45:10.970 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-riic.txt 2024-03-27T06:45:10.971 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rk3x.txt 2024-03-27T06:45:10.972 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-s3c2410.txt 2024-03-27T06:45:10.973 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt 2024-03-27T06:45:10.974 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sirf.txt 2024-03-27T06:45:10.975 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st-ddci2c.txt 2024-03-27T06:45:10.976 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st.txt 2024-03-27T06:45:10.977 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sunxi-p2wi.txt 2024-03-27T06:45:10.978 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-versatile.txt 2024-03-27T06:45:10.979 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-vt8500.txt 2024-03-27T06:45:10.980 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-xiic.txt 2024-03-27T06:45:10.981 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina209.txt 2024-03-27T06:45:10.982 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina2xx.txt 2024-03-27T06:45:10.983 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/max6697.txt 2024-03-27T06:45:10.984 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/nvidia,tegra20-i2c.txt 2024-03-27T06:45:10.985 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/qcom,i2c-qup.txt 2024-03-27T06:45:10.986 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ti,bq32k.txt 2024-03-27T06:45:10.987 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/trivial-devices.txt 2024-03-27T06:45:10.988 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/ 2024-03-27T06:45:10.989 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/ 2024-03-27T06:45:10.990 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/bma180.txt 2024-03-27T06:45:10.991 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/ 2024-03-27T06:45:10.992 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/at91_adc.txt 2024-03-27T06:45:10.993 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/cc10001_adc.txt 2024-03-27T06:45:10.994 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/max1027-adc.txt 2024-03-27T06:45:10.995 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/nuvoton-nau7802.txt 2024-03-27T06:45:10.995 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-iadc.txt 2024-03-27T06:45:10.997 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-vadc.txt 2024-03-27T06:45:10.997 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt 2024-03-27T06:45:10.999 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/twl4030-madc.txt 2024-03-27T06:45:10.999 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/vf610-adc.txt 2024-03-27T06:45:11.000 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/xilinx-xadc.txt 2024-03-27T06:45:11.001 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ 2024-03-27T06:45:11.002 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ad7303.txt 2024-03-27T06:45:11.003 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/max5821.txt 2024-03-27T06:45:11.004 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/ 2024-03-27T06:45:11.005 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/adf4350.txt 2024-03-27T06:45:11.006 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/ 2024-03-27T06:45:11.007 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/dht11.txt 2024-03-27T06:45:11.008 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/iio-bindings.txt 2024-03-27T06:45:11.009 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/ 2024-03-27T06:45:11.010 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/apds9300.txt 2024-03-27T06:45:11.012 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/cm36651.txt 2024-03-27T06:45:11.014 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/gp2ap020a00f.txt 2024-03-27T06:45:11.015 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/tsl2563.txt 2024-03-27T06:45:11.017 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ 2024-03-27T06:45:11.018 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt 2024-03-27T06:45:11.020 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/hmc5843.txt 2024-03-27T06:45:11.021 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/ 2024-03-27T06:45:11.023 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/as3935.txt 2024-03-27T06:45:11.024 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/sensorhub.txt 2024-03-27T06:45:11.026 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/st-sensors.txt 2024-03-27T06:45:11.028 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ 2024-03-27T06:45:11.032 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ads7846.txt 2024-03-27T06:45:11.034 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/atmel,maxtouch.txt 2024-03-27T06:45:11.036 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cap11xx.txt 2024-03-27T06:45:11.037 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/clps711x-keypad.txt 2024-03-27T06:45:11.039 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cros-ec-keyb.txt 2024-03-27T06:45:11.040 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/e3x0-button.txt 2024-03-27T06:45:11.042 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elan_i2c.txt 2024-03-27T06:45:11.043 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elants_i2c.txt 2024-03-27T06:45:11.045 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/fsl-mma8450.txt 2024-03-27T06:45:11.046 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-beeper.txt 2024-03-27T06:45:11.047 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys-polled.txt 2024-03-27T06:45:11.049 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys.txt 2024-03-27T06:45:11.050 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-matrix-keypad.txt 2024-03-27T06:45:11.052 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/imx-keypad.txt 2024-03-27T06:45:11.054 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/input-reset.txt 2024-03-27T06:45:11.059 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/lpc32xx-key.txt 2024-03-27T06:45:11.059 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/matrix-keymap.txt 2024-03-27T06:45:11.059 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/nvidia,tegra20-kbc.txt 2024-03-27T06:45:11.060 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/omap-keypad.txt 2024-03-27T06:45:11.061 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ps2keyb-mouse-apbps2.txt 2024-03-27T06:45:11.062 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pwm-beeper.txt 2024-03-27T06:45:11.063 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pxa27x-keypad.txt 2024-03-27T06:45:11.064 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-keypad.txt 2024-03-27T06:45:11.065 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-pwrkey.txt 2024-03-27T06:45:11.066 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-vib.txt 2024-03-27T06:45:11.067 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/regulator-haptic.txt 2024-03-27T06:45:11.068 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/rotary-encoder.txt 2024-03-27T06:45:11.069 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/samsung-keypad.txt 2024-03-27T06:45:11.070 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/spear-keyboard.txt 2024-03-27T06:45:11.071 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/st-keyscan.txt 2024-03-27T06:45:11.072 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/stmpe-keypad.txt 2024-03-27T06:45:11.073 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/sun4i-lradc-keys.txt 2024-03-27T06:45:11.074 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tca8418_keypad.txt 2024-03-27T06:45:11.075 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv260x.txt 2024-03-27T06:45:11.076 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv2667.txt 2024-03-27T06:45:11.077 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,nspire-keypad.txt 2024-03-27T06:45:11.078 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,palmas-pwrbutton.txt 2024-03-27T06:45:11.079 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ 2024-03-27T06:45:11.080 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/auo_pixcir_ts.txt 2024-03-27T06:45:11.081 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/bu21013.txt 2024-03-27T06:45:11.082 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/edt-ft5x06.txt 2024-03-27T06:45:11.083 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/egalax-ts.txt 2024-03-27T06:45:11.084 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/lpc32xx-tsc.txt 2024-03-27T06:45:11.085 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/mms114.txt 2024-03-27T06:45:11.086 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/pixcir_i2c_ts.txt 2024-03-27T06:45:11.087 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sitronix-st1232.txt 2024-03-27T06:45:11.088 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/stmpe.txt 2024-03-27T06:45:11.090 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt 2024-03-27T06:45:11.091 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ti-tsc-adc.txt 2024-03-27T06:45:11.092 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt 2024-03-27T06:45:11.093 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt 2024-03-27T06:45:11.094 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2007.txt 2024-03-27T06:45:11.095 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/zforce_ts.txt 2024-03-27T06:45:11.096 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tps65218-pwrbutton.txt 2024-03-27T06:45:11.097 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-keypad.txt 2024-03-27T06:45:11.098 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-pwrbutton.txt 2024-03-27T06:45:11.099 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ 2024-03-27T06:45:11.100 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/abilis,tb10x-ictl.txt 2024-03-27T06:45:11.101 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun4i-ic.txt 2024-03-27T06:45:11.102 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun67i-sc-nmi.txt 2024-03-27T06:45:11.103 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/atmel,aic.txt 2024-03-27T06:45:11.104 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt 2024-03-27T06:45:11.105 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm7120-l2-intc.txt 2024-03-27T06:45:11.106 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,l2-intc.txt 2024-03-27T06:45:11.107 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/cirrus,clps711x-intc.txt 2024-03-27T06:45:11.108 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/digicolor-ic.txt 2024-03-27T06:45:11.109 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/interrupts.txt 2024-03-27T06:45:11.110 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/lsi,zevio-intc.txt 2024-03-27T06:45:11.111 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt 2024-03-27T06:45:11.112 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,orion-intc.txt 2024-03-27T06:45:11.113 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/mips-gic.txt 2024-03-27T06:45:11.114 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/opencores,or1k-pic.txt 2024-03-27T06:45:11.116 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,intc-irqpin.txt 2024-03-27T06:45:11.117 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.txt 2024-03-27T06:45:11.118 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/samsung,s3c24xx-irq.txt 2024-03-27T06:45:11.119 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt 2024-03-27T06:45:11.120 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,keystone-irq.txt 2024-03-27T06:45:11.121 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,omap-intc-irq.txt 2024-03-27T06:45:11.122 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ 2024-03-27T06:45:11.123 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/arm,smmu.txt 2024-03-27T06:45:11.124 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/iommu.txt 2024-03-27T06:45:11.125 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra20-gart.txt 2024-03-27T06:45:11.126 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra30-smmu.txt 2024-03-27T06:45:11.127 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/renesas,ipmmu-vmsa.txt 2024-03-27T06:45:11.128 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/rockchip,iommu.txt 2024-03-27T06:45:11.129 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt 2024-03-27T06:45:11.130 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ti,omap-iommu.txt 2024-03-27T06:45:11.131 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/ 2024-03-27T06:45:11.132 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/common.txt 2024-03-27T06:45:11.133 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-gpio.txt 2024-03-27T06:45:11.134 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp55xx.txt 2024-03-27T06:45:11.135 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp8860.txt 2024-03-27T06:45:11.136 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-ns2.txt 2024-03-27T06:45:11.515 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-pwm.txt 2024-03-27T06:45:11.517 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/pca963x.txt 2024-03-27T06:45:11.519 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/register-bit-led.txt 2024-03-27T06:45:11.520 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/tca6507.txt 2024-03-27T06:45:11.522 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/ 2024-03-27T06:45:11.523 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2-timings.txt 2024-03-27T06:45:11.525 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2.txt 2024-03-27T06:45:11.527 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/ 2024-03-27T06:45:11.528 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/altera-mailbox.txt 2024-03-27T06:45:11.530 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/mailbox.txt 2024-03-27T06:45:11.532 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt 2024-03-27T06:45:11.533 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/marvell.txt 2024-03-27T06:45:11.535 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ 2024-03-27T06:45:11.537 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/atmel-isi.txt 2024-03-27T06:45:11.538 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/coda.txt 2024-03-27T06:45:11.540 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-fimc-lite.txt 2024-03-27T06:45:11.541 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-jpeg-codec.txt 2024-03-27T06:45:11.542 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos4-fimc-is.txt 2024-03-27T06:45:11.543 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos5-gsc.txt 2024-03-27T06:45:11.544 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/gpio-ir-receiver.txt 2024-03-27T06:45:11.545 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/hix5hd2-ir.txt 2024-03-27T06:45:11.546 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ 2024-03-27T06:45:11.547 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7343.txt 2024-03-27T06:45:11.548 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7604.txt 2024-03-27T06:45:11.549 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9m111.txt 2024-03-27T06:45:11.550 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9p031.txt 2024-03-27T06:45:11.551 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/nokia,smia.txt 2024-03-27T06:45:11.552 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ths8200.txt 2024-03-27T06:45:11.553 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp514x.txt 2024-03-27T06:45:11.554 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp7002.txt 2024-03-27T06:45:11.555 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/img-ir-rev1.txt 2024-03-27T06:45:11.557 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/meson-ir.txt 2024-03-27T06:45:11.558 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/pxa-camera.txt 2024-03-27T06:45:11.559 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/rcar_vin.txt 2024-03-27T06:45:11.560 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/renesas,vsp1.txt 2024-03-27T06:45:11.561 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/s5p-mfc.txt 2024-03-27T06:45:11.562 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-fimc.txt 2024-03-27T06:45:11.563 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-mipi-csis.txt 2024-03-27T06:45:11.564 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5c73m3.txt 2024-03-27T06:45:11.565 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k5baf.txt 2024-03-27T06:45:11.566 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k6a3.txt 2024-03-27T06:45:11.567 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sh_mobile_ceu.txt 2024-03-27T06:45:11.568 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/si4713.txt 2024-03-27T06:45:11.569 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/st-rc.txt 2024-03-27T06:45:11.570 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sunxi-ir.txt 2024-03-27T06:45:11.571 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ti-am437x-vpfe.txt 2024-03-27T06:45:11.572 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/video-interfaces.txt 2024-03-27T06:45:11.573 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ 2024-03-27T06:45:11.574 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ 2024-03-27T06:45:11.575 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt 2024-03-27T06:45:11.576 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-devbus.txt 2024-03-27T06:45:11.577 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-sdram-controller.txt 2024-03-27T06:45:11.578 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt 2024-03-27T06:45:11.579 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/renesas-memory-controllers.txt 2024-03-27T06:45:11.580 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/synopsys.txt 2024-03-27T06:45:11.581 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti-aemif.txt 2024-03-27T06:45:11.582 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/ 2024-03-27T06:45:11.583 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/emif.txt 2024-03-27T06:45:11.584 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/ 2024-03-27T06:45:11.585 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta-intc.txt 2024-03-27T06:45:11.586 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta.txt 2024-03-27T06:45:11.587 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/pdc-intc.txt 2024-03-27T06:45:11.588 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ 2024-03-27T06:45:11.589 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/88pm860x.txt 2024-03-27T06:45:11.590 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ab8500.txt 2024-03-27T06:45:11.591 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/arizona.txt 2024-03-27T06:45:11.592 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3711.txt 2024-03-27T06:45:11.593 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3722.txt 2024-03-27T06:45:11.594 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-gpbr.txt 2024-03-27T06:45:11.595 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-hlcdc.txt 2024-03-27T06:45:11.596 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-matrix.txt 2024-03-27T06:45:11.597 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-smc.txt 2024-03-27T06:45:11.598 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bcm590xx.txt 2024-03-27T06:45:11.599 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bfticu.txt 2024-03-27T06:45:11.600 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/cros-ec.txt 2024-03-27T06:45:11.601 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9052-i2c.txt 2024-03-27T06:45:11.602 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9055.txt 2024-03-27T06:45:11.603 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9063.txt 2024-03-27T06:45:11.604 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/hi6421.txt 2024-03-27T06:45:11.605 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/lp3943.txt 2024-03-27T06:45:11.606 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max14577.txt 2024-03-27T06:45:11.607 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77686.txt 2024-03-27T06:45:11.608 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77693.txt 2024-03-27T06:45:11.609 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8925.txt 2024-03-27T06:45:11.610 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8998.txt 2024-03-27T06:45:11.611 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/mc13xxx.txt 2024-03-27T06:45:11.612 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-host.txt 2024-03-27T06:45:11.613 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-tll.txt 2024-03-27T06:45:11.614 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/palmas.txt 2024-03-27T06:45:11.615 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom,spmi-pmic.txt 2024-03-27T06:45:11.616 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-pm8xxx.txt 2024-03-27T06:45:11.617 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-rpm.txt 2024-03-27T06:45:11.618 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qriox.txt 2024-03-27T06:45:11.619 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rk808.txt 2024-03-27T06:45:11.620 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rn5t618.txt 2024-03-27T06:45:11.621 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mpa01.txt 2024-03-27T06:45:11.622 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mps11.txt 2024-03-27T06:45:11.623 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/stmpe.txt 2024-03-27T06:45:11.625 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt 2024-03-27T06:45:11.625 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/syscon.txt 2024-03-27T06:45:11.626 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tc3589x.txt 2024-03-27T06:45:11.627 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt 2024-03-27T06:45:11.628 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps6507x.txt 2024-03-27T06:45:11.629 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps65910.txt 2024-03-27T06:45:11.630 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl-familly.txt 2024-03-27T06:45:11.631 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-audio.txt 2024-03-27T06:45:11.632 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-power.txt 2024-03-27T06:45:11.633 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl6040.txt 2024-03-27T06:45:11.634 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/ 2024-03-27T06:45:11.635 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/ 2024-03-27T06:45:11.636 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt 2024-03-27T06:45:11.637 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/nvidia,tegra114-mipi.txt 2024-03-27T06:45:11.638 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ 2024-03-27T06:45:11.639 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/ 2024-03-27T06:45:11.640 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bcm3384-intc.txt 2024-03-27T06:45:11.641 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bmips.txt 2024-03-27T06:45:11.642 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/cm-dsl.txt 2024-03-27T06:45:11.643 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/usb.txt 2024-03-27T06:45:11.644 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ 2024-03-27T06:45:11.645 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/bootbus.txt 2024-03-27T06:45:11.646 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/cib.txt 2024-03-27T06:45:11.647 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu.txt 2024-03-27T06:45:11.648 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu2.txt 2024-03-27T06:45:11.649 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/dma-engine.txt 2024-03-27T06:45:11.650 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/uctl.txt 2024-03-27T06:45:11.651 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cpu_irq.txt 2024-03-27T06:45:11.652 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ralink.txt 2024-03-27T06:45:11.653 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ 2024-03-27T06:45:11.654 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/allwinner,sunxi-sid.txt 2024-03-27T06:45:11.655 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/arm-charlcd.txt 2024-03-27T06:45:11.656 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/at25.txt 2024-03-27T06:45:11.657 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/atmel-ssc.txt 2024-03-27T06:45:11.658 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/bmp085.txt 2024-03-27T06:45:11.660 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt 2024-03-27T06:45:11.662 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ifm-csi.txt 2024-03-27T06:45:11.663 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/lis302.txt 2024-03-27T06:45:11.665 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt 2024-03-27T06:45:11.666 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/smc.txt 2024-03-27T06:45:11.668 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/sram.txt 2024-03-27T06:45:11.670 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ti,dac7512.txt 2024-03-27T06:45:11.675 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ 2024-03-27T06:45:11.675 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/arasan,sdhci.txt 2024-03-27T06:45:11.675 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/atmel-hsmci.txt 2024-03-27T06:45:11.675 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/brcm,bcm2835-sdhci.txt 2024-03-27T06:45:11.677 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/davinci_mmc.txt 2024-03-27T06:45:11.678 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/exynos-dw-mshc.txt 2024-03-27T06:45:11.680 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-esdhc.txt 2024-03-27T06:45:11.682 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt 2024-03-27T06:45:11.683 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-mmc.txt 2024-03-27T06:45:11.685 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/img-dw-mshc.txt 2024-03-27T06:45:11.686 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/k3-dw-mshc.txt 2024-03-27T06:45:11.688 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/kona-sdhci.txt 2024-03-27T06:45:11.689 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt 2024-03-27T06:45:11.691 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-simple.txt 2024-03-27T06:45:11.693 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-spi-slot.txt 2024-03-27T06:45:11.694 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc.txt 2024-03-27T06:45:11.696 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmci.txt 2024-03-27T06:45:11.697 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt 2024-03-27T06:45:11.699 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mxs-mmc.txt 2024-03-27T06:45:11.701 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt 2024-03-27T06:45:11.702 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/orion-sdio.txt 2024-03-27T06:45:11.704 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/pxa-mmc.txt 2024-03-27T06:45:11.705 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/renesas,mmcif.txt 2024-03-27T06:45:11.707 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.txt 2024-03-27T06:45:11.708 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/samsung-sdhci.txt 2024-03-27T06:45:11.709 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-dove.txt 2024-03-27T06:45:11.711 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-fujitsu.txt 2024-03-27T06:45:11.713 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-msm.txt 2024-03-27T06:45:11.714 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-pxa.txt 2024-03-27T06:45:11.716 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-sirf.txt 2024-03-27T06:45:11.717 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-spear.txt 2024-03-27T06:45:11.719 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-st.txt 2024-03-27T06:45:11.720 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/socfpga-dw-mshc.txt 2024-03-27T06:45:11.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:11 smithi156 ceph-mon[36261]: pgmap v51: 97 pgs: 97 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-03-27T06:45:11.722 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt 2024-03-27T06:45:11.724 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt 2024-03-27T06:45:11.725 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap-hsmmc.txt 2024-03-27T06:45:11.727 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap.txt 2024-03-27T06:45:11.729 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/tmio_mmc.txt 2024-03-27T06:45:11.730 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt 2024-03-27T06:45:11.732 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/vt8500-sdmmc.txt 2024-03-27T06:45:11.734 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/ 2024-03-27T06:45:11.735 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/arm-versatile.txt 2024-03-27T06:45:11.737 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-dataflash.txt 2024-03-27T06:45:11.738 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-nand.txt 2024-03-27T06:45:11.740 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/davinci-nand.txt 2024-03-27T06:45:11.741 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/denali-nand.txt 2024-03-27T06:45:11.743 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/diskonchip.txt 2024-03-27T06:45:11.745 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/elm.txt 2024-03-27T06:45:11.746 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/flctl-nand.txt 2024-03-27T06:45:11.747 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt 2024-03-27T06:45:11.749 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-upm-nand.txt 2024-03-27T06:45:11.750 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsmc-nand.txt 2024-03-27T06:45:11.752 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpio-control-nand.txt 2024-03-27T06:45:11.754 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nand.txt 2024-03-27T06:45:11.755 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nor.txt 2024-03-27T06:45:11.756 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-onenand.txt 2024-03-27T06:45:11.758 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmi-nand.txt 2024-03-27T06:45:11.760 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/hisi504-nand.txt 2024-03-27T06:45:11.761 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-mlc.txt 2024-03-27T06:45:11.763 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-slc.txt 2024-03-27T06:45:11.764 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/m25p80.txt 2024-03-27T06:45:11.766 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mtd-physmap.txt 2024-03-27T06:45:11.767 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mxc-nand.txt 2024-03-27T06:45:11.769 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/nand.txt 2024-03-27T06:45:11.770 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/orion-nand.txt 2024-03-27T06:45:11.772 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/partition.txt 2024-03-27T06:45:11.773 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt 2024-03-27T06:45:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/spear_smi.txt 2024-03-27T06:45:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/st-fsm.txt 2024-03-27T06:45:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/sunxi-nand.txt 2024-03-27T06:45:11.780 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ 2024-03-27T06:45:11.781 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-emac.txt 2024-03-27T06:45:11.783 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-mdio.txt 2024-03-27T06:45:11.785 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.txt 2024-03-27T06:45:11.787 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/altera_tse.txt 2024-03-27T06:45:11.789 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe-phy.txt 2024-03-27T06:45:11.791 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe.txt 2024-03-27T06:45:11.793 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/apm-xgene-enet.txt 2024-03-27T06:45:11.795 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/arc_emac.txt 2024-03-27T06:45:11.797 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcm87xx.txt 2024-03-27T06:45:11.799 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcmgenet.txt 2024-03-27T06:45:11.801 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-mdio-unimac.txt 2024-03-27T06:45:11.803 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-sf2.txt 2024-03-27T06:45:11.805 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-systemport.txt 2024-03-27T06:45:11.807 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/calxeda-xgmac.txt 2024-03-27T06:45:11.808 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/ 2024-03-27T06:45:11.810 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/atmel-can.txt 2024-03-27T06:45:11.812 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/c_can.txt 2024-03-27T06:45:11.814 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/cc770.txt 2024-03-27T06:45:11.816 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/fsl-flexcan.txt 2024-03-27T06:45:11.818 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/grcan.txt 2024-03-27T06:45:11.820 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/m_can.txt 2024-03-27T06:45:11.822 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt 2024-03-27T06:45:11.824 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/mpc5xxx-mscan.txt 2024-03-27T06:45:11.826 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/rcar_can.txt 2024-03-27T06:45:11.828 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/sja1000.txt 2024-03-27T06:45:11.830 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/xilinx_can.txt 2024-03-27T06:45:11.831 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mdio.txt 2024-03-27T06:45:11.833 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mix.txt 2024-03-27T06:45:11.835 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-pip.txt 2024-03-27T06:45:11.836 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cdns-emac.txt 2024-03-27T06:45:11.838 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw-phy-sel.txt 2024-03-27T06:45:11.840 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw.txt 2024-03-27T06:45:11.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:11 smithi170 ceph-mon[36438]: pgmap v51: 97 pgs: 97 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-03-27T06:45:11.937 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davicom-dm9000.txt 2024-03-27T06:45:11.937 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci-mdio.txt 2024-03-27T06:45:11.937 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci_emac.txt 2024-03-27T06:45:11.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/ 2024-03-27T06:45:11.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/dsa.txt 2024-03-27T06:45:11.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/emac_rockchip.txt 2024-03-27T06:45:11.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ethernet.txt 2024-03-27T06:45:11.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fixed-link.txt 2024-03-27T06:45:11.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-fec.txt 2024-03-27T06:45:11.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-tsec-phy.txt 2024-03-27T06:45:11.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/gpmc-eth.txt 2024-03-27T06:45:11.938 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:11 smithi016 ceph-mon[31908]: pgmap v51: 97 pgs: 97 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-03-27T06:45:11.943 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hip04-net.txt 2024-03-27T06:45:11.943 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hix5hd2-gmac.txt 2024-03-27T06:45:11.943 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/ 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/at86rf230.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/cc2520.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/keystone-netcp.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/lpc-eth.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/macb.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-mdio.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-net.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pp2.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pxa168.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-gpio.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-gpio.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-mmioreg.txt 2024-03-27T06:45:11.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/meson-dwmac.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ks8851.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ksz90x1.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/moxa,moxart-mac.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/ 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/pn544.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfca.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfcb.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/trf7970a.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/opencores-ethoc.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/phy.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/qca-qca7000-spi.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/rockchip-dwmac.txt 2024-03-27T06:45:11.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/samsung-sxgbe.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sh_eth.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc-lan91c111.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc911x.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/socfpga-dwmac.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sti-dwmac.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/stmmac.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-rhine.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-velocity.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/brcm,bcm43xx-fmac.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ti,wl1251.txt 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/ 2024-03-27T06:45:11.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/nios2.txt 2024-03-27T06:45:11.947 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/timer.txt 2024-03-27T06:45:11.947 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/ 2024-03-27T06:45:11.947 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/nvidia,nvec.txt 2024-03-27T06:45:11.947 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/open-pic.txt 2024-03-27T06:45:11.947 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/ 2024-03-27T06:45:11.947 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101aw03.txt 2024-03-27T06:45:11.947 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101xtn01.txt 2024-03-27T06:45:11.948 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b116xw03.txt 2024-03-27T06:45:11.950 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133htn01.txt 2024-03-27T06:45:11.952 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133xtn01.txt 2024-03-27T06:45:11.954 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/avic,tm070ddh03.txt 2024-03-27T06:45:11.956 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wa01a.txt 2024-03-27T06:45:11.958 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wb03.txt 2024-03-27T06:45:11.960 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et057090dhu.txt 2024-03-27T06:45:11.962 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et070080dh6.txt 2024-03-27T06:45:11.964 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,etm0700g0dh6.txt 2024-03-27T06:45:11.965 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/foxlink,fl500wvr00-a0t.txt 2024-03-27T06:45:11.967 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/giantplus,gpg482739qs5.txt 2024-03-27T06:45:11.968 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hannstar,hsd070pww1.txt 2024-03-27T06:45:11.970 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hit,tx23d38vm0caa.txt 2024-03-27T06:45:11.972 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,g121i1-l01.txt 2024-03-27T06:45:11.973 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n116bge.txt 2024-03-27T06:45:11.975 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n156bge-l21.txt 2024-03-27T06:45:11.977 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,ld070wx3-sl01.txt 2024-03-27T06:45:11.979 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lh500wx1-sd03.txt 2024-03-27T06:45:11.981 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lp129qe.txt 2024-03-27T06:45:11.983 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/panasonic,vvx10f004b00.txt 2024-03-27T06:45:11.984 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ld9040.txt 2024-03-27T06:45:11.985 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ltn101nt05.txt 2024-03-27T06:45:11.986 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,s6e8aa0.txt 2024-03-27T06:45:11.988 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/sharp,lq101r1sx01.txt 2024-03-27T06:45:11.989 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/simple-panel.txt 2024-03-27T06:45:11.990 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ 2024-03-27T06:45:11.991 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/83xx-512x-pci.txt 2024-03-27T06:45:11.993 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/designware-pcie.txt 2024-03-27T06:45:11.994 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt 2024-03-27T06:45:11.995 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,pci.txt 2024-03-27T06:45:11.996 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/host-generic-pci.txt 2024-03-27T06:45:11.998 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/layerscape-pci.txt 2024-03-27T06:45:11.999 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/mvebu-pci.txt 2024-03-27T06:45:12.000 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt 2024-03-27T06:45:12.002 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-keystone.txt 2024-03-27T06:45:12.003 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-rcar-gen2.txt 2024-03-27T06:45:12.004 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci.txt 2024-03-27T06:45:12.005 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ralink,rt3883-pci.txt 2024-03-27T06:45:12.007 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/rcar-pci.txt 2024-03-27T06:45:12.008 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/samsung,exynos5440-pcie.txt 2024-03-27T06:45:12.009 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/spear13xx-pcie.txt 2024-03-27T06:45:12.011 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ti-pci.txt 2024-03-27T06:45:12.012 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/v3-v360epc-pci.txt 2024-03-27T06:45:12.013 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/versatile.txt 2024-03-27T06:45:12.015 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xgene-pci.txt 2024-03-27T06:45:12.016 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xilinx-pcie.txt 2024-03-27T06:45:12.017 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ 2024-03-27T06:45:12.018 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/apm-xgene-phy.txt 2024-03-27T06:45:12.019 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/bcm-phy.txt 2024-03-27T06:45:12.021 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-sata-phy.txt 2024-03-27T06:45:12.022 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-usb-phy.txt 2024-03-27T06:45:12.023 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/hix5hd2-phy.txt 2024-03-27T06:45:12.024 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-bindings.txt 2024-03-27T06:45:12.025 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy28lp.txt 2024-03-27T06:45:12.026 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy365x.txt 2024-03-27T06:45:12.028 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-mvebu.txt 2024-03-27T06:45:12.029 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih407-usb.txt 2024-03-27T06:45:12.030 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt 2024-03-27T06:45:12.031 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-apq8064-sata-phy.txt 2024-03-27T06:45:12.032 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-dwc3-usb-phy.txt 2024-03-27T06:45:12.033 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-ipq806x-sata-phy.txt 2024-03-27T06:45:12.034 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt 2024-03-27T06:45:12.035 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rockchip-usb-phy.txt 2024-03-27T06:45:12.036 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/samsung-phy.txt 2024-03-27T06:45:12.037 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/st-spear-miphy.txt 2024-03-27T06:45:12.038 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/sun4i-usb-phy.txt 2024-03-27T06:45:12.039 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ti-phy.txt 2024-03-27T06:45:12.040 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ 2024-03-27T06:45:12.043 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/abilis,tb10x-iomux.txt 2024-03-27T06:45:12.044 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt 2024-03-27T06:45:12.045 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/atmel,at91-pinctrl.txt 2024-03-27T06:45:12.046 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm11351-pinctrl.txt 2024-03-27T06:45:12.047 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm2835-gpio.txt 2024-03-27T06:45:12.048 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx-pinctrl.txt 2024-03-27T06:45:12.049 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx25-pinctrl.txt 2024-03-27T06:45:12.050 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx27-pinctrl.txt 2024-03-27T06:45:12.051 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx35-pinctrl.txt 2024-03-27T06:45:12.052 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx51-pinctrl.txt 2024-03-27T06:45:12.054 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx53-pinctrl.txt 2024-03-27T06:45:12.055 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6dl-pinctrl.txt 2024-03-27T06:45:12.056 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6q-pinctrl.txt 2024-03-27T06:45:12.057 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sl-pinctrl.txt 2024-03-27T06:45:12.058 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sx-pinctrl.txt 2024-03-27T06:45:12.059 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,mxs-pinctrl.txt 2024-03-27T06:45:12.060 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,vf610-pinctrl.txt 2024-03-27T06:45:12.061 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pdc-pinctrl.txt 2024-03-27T06:45:12.062 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pinctrl.txt 2024-03-27T06:45:12.063 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,falcon-pinumx.txt 2024-03-27T06:45:12.064 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,xway-pinumx.txt 2024-03-27T06:45:12.065 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-370-pinctrl.txt 2024-03-27T06:45:12.066 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-375-pinctrl.txt 2024-03-27T06:45:12.067 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-38x-pinctrl.txt 2024-03-27T06:45:12.068 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-xp-pinctrl.txt 2024-03-27T06:45:12.069 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,dove-pinctrl.txt 2024-03-27T06:45:12.070 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,kirkwood-pinctrl.txt 2024-03-27T06:45:12.071 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,mvebu-pinctrl.txt 2024-03-27T06:45:12.072 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,orion-pinctrl.txt 2024-03-27T06:45:12.073 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/meson,pinctrl.txt 2024-03-27T06:45:12.074 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra114-pinmux.txt 2024-03-27T06:45:12.075 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-pinmux.txt 2024-03-27T06:45:12.076 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt 2024-03-27T06:45:12.077 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra20-pinmux.txt 2024-03-27T06:45:12.078 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra30-pinmux.txt 2024-03-27T06:45:12.080 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt 2024-03-27T06:45:12.081 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 2024-03-27T06:45:12.082 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-single.txt 2024-03-27T06:45:12.087 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-sirf.txt 2024-03-27T06:45:12.087 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-st.txt 2024-03-27T06:45:12.087 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-vt8500.txt 2024-03-27T06:45:12.087 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl_spear.txt 2024-03-27T06:45:12.087 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8064-pinctrl.txt 2024-03-27T06:45:12.088 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8084-pinctrl.txt 2024-03-27T06:45:12.089 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,ipq8064-pinctrl.txt 2024-03-27T06:45:12.090 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8916-pinctrl.txt 2024-03-27T06:45:12.091 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8960-pinctrl.txt 2024-03-27T06:45:12.092 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8974-pinctrl.txt 2024-03-27T06:45:12.093 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt 2024-03-27T06:45:12.094 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt 2024-03-27T06:45:12.095 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt 2024-03-27T06:45:12.096 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt 2024-03-27T06:45:12.097 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt 2024-03-27T06:45:12.098 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,abx500.txt 2024-03-27T06:45:12.099 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt 2024-03-27T06:45:12.100 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ti,omap-pinctrl.txt 2024-03-27T06:45:12.101 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/xlnx,zynq-pinctrl.txt 2024-03-27T06:45:12.102 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ 2024-03-27T06:45:12.103 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/bq2415x.txt 2024-03-27T06:45:12.104 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/isp1704.txt 2024-03-27T06:45:12.105 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ltc2941.txt 2024-03-27T06:45:12.106 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/opp.txt 2024-03-27T06:45:12.107 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power-controller.txt 2024-03-27T06:45:12.108 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power_domain.txt 2024-03-27T06:45:12.109 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/renesas,sysc-rmobile.txt 2024-03-27T06:45:12.111 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ 2024-03-27T06:45:12.112 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/keystone-reset.txt 2024-03-27T06:45:12.113 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ltc2952-poweroff.txt 2024-03-27T06:45:12.114 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/st-reset.txt 2024-03-27T06:45:12.115 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/syscon-reboot.txt 2024-03-27T06:45:12.116 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rockchip-io-domain.txt 2024-03-27T06:45:12.117 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rx51-battery.txt 2024-03-27T06:45:12.119 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/twl-charger.txt 2024-03-27T06:45:12.120 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ 2024-03-27T06:45:12.121 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/ 2024-03-27T06:45:12.122 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/btemp.txt 2024-03-27T06:45:12.123 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/chargalg.txt 2024-03-27T06:45:12.124 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/charger.txt 2024-03-27T06:45:12.125 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/fg.txt 2024-03-27T06:45:12.126 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/axxia-reset.txt 2024-03-27T06:45:12.127 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/charger-manager.txt 2024-03-27T06:45:12.128 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/gpio-charger.txt 2024-03-27T06:45:12.129 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/imx-snvs-poweroff.txt 2024-03-27T06:45:12.130 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/lp8727_charger.txt 2024-03-27T06:45:12.131 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max17042_battery.txt 2024-03-27T06:45:12.132 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max8925_batter.txt 2024-03-27T06:45:12.133 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/msm-poweroff.txt 2024-03-27T06:45:12.134 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/olpc_battery.txt 2024-03-27T06:45:12.135 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/power_supply.txt 2024-03-27T06:45:12.136 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/qnap-poweroff.txt 2024-03-27T06:45:12.137 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/restart-poweroff.txt 2024-03-27T06:45:12.138 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/sbs_sbs-battery.txt 2024-03-27T06:45:12.139 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ti,bq24735.txt 2024-03-27T06:45:12.141 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/tps65090.txt 2024-03-27T06:45:12.142 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/ 2024-03-27T06:45:12.143 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ 2024-03-27T06:45:12.144 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt 2024-03-27T06:45:12.146 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/cpm.txt 2024-03-27T06:45:12.146 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/emac.txt 2024-03-27T06:45:12.148 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt 2024-03-27T06:45:12.149 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ndfc.txt 2024-03-27T06:45:12.150 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ppc440spe-adma.txt 2024-03-27T06:45:12.151 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/reboot.txt 2024-03-27T06:45:12.153 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ 2024-03-27T06:45:12.153 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/board.txt 2024-03-27T06:45:12.155 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cache_sram.txt 2024-03-27T06:45:12.156 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt 2024-03-27T06:45:12.158 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/ 2024-03-27T06:45:12.158 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm.txt 2024-03-27T06:45:12.159 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/ 2024-03-27T06:45:12.160 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/brg.txt 2024-03-27T06:45:12.161 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/i2c.txt 2024-03-27T06:45:12.162 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/pic.txt 2024-03-27T06:45:12.163 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/usb.txt 2024-03-27T06:45:12.164 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/gpio.txt 2024-03-27T06:45:12.165 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/network.txt 2024-03-27T06:45:12.166 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe.txt 2024-03-27T06:45:12.167 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ 2024-03-27T06:45:12.168 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/firmware.txt 2024-03-27T06:45:12.169 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/par_io.txt 2024-03-27T06:45:12.170 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/pincfg.txt 2024-03-27T06:45:12.171 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ucc.txt 2024-03-27T06:45:12.172 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/usb.txt 2024-03-27T06:45:12.173 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/serial.txt 2024-03-27T06:45:12.175 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt 2024-03-27T06:45:12.176 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dcsr.txt 2024-03-27T06:45:12.177 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/diu.txt 2024-03-27T06:45:12.178 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dma.txt 2024-03-27T06:45:12.179 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ecm.txt 2024-03-27T06:45:12.180 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/fman.txt 2024-03-27T06:45:12.181 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/gtm.txt 2024-03-27T06:45:12.183 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/guts.txt 2024-03-27T06:45:12.184 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/interlaken-lac.txt 2024-03-27T06:45:12.185 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/l2cache.txt 2024-03-27T06:45:12.186 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/lbc.txt 2024-03-27T06:45:12.187 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcm.txt 2024-03-27T06:45:12.188 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcu-mpc8349emitx.txt 2024-03-27T06:45:12.189 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mem-ctrlr.txt 2024-03-27T06:45:12.190 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5121-psc.txt 2024-03-27T06:45:12.191 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5200.txt 2024-03-27T06:45:12.193 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt 2024-03-27T06:45:12.194 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-timer.txt 2024-03-27T06:45:12.195 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic.txt 2024-03-27T06:45:12.196 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/msi-pic.txt 2024-03-27T06:45:12.196 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt 2024-03-27T06:45:12.198 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pmc.txt 2024-03-27T06:45:12.199 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/raideng.txt 2024-03-27T06:45:12.200 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio-rmu.txt 2024-03-27T06:45:12.201 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio.txt 2024-03-27T06:45:12.202 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/ 2024-03-27T06:45:12.203 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/gamecube.txt 2024-03-27T06:45:12.204 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/wii.txt 2024-03-27T06:45:12.206 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/ 2024-03-27T06:45:12.207 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/pps-gpio.txt 2024-03-27T06:45:12.208 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ 2024-03-27T06:45:12.209 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-hlcdc-pwm.txt 2024-03-27T06:45:12.210 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-pwm.txt 2024-03-27T06:45:12.211 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-tcb-pwm.txt 2024-03-27T06:45:12.213 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/bcm-kona-pwm.txt 2024-03-27T06:45:12.214 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/cirrus,clps711x-pwm.txt 2024-03-27T06:45:12.215 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/img-pwm.txt 2024-03-27T06:45:12.216 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/imx-pwm.txt 2024-03-27T06:45:12.217 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/lpc32xx-pwm.txt 2024-03-27T06:45:12.218 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/mxs-pwm.txt 2024-03-27T06:45:12.219 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nvidia,tegra20-pwm.txt 2024-03-27T06:45:12.221 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nxp,pca9685-pwm.txt 2024-03-27T06:45:12.222 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-bcm2835.txt 2024-03-27T06:45:12.223 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-fsl-ftm.txt 2024-03-27T06:45:12.224 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-lp3943.txt 2024-03-27T06:45:12.226 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-rockchip.txt 2024-03-27T06:45:12.228 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-samsung.txt 2024-03-27T06:45:12.230 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-st.txt 2024-03-27T06:45:12.232 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-sun4i.txt 2024-03-27T06:45:12.233 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiecap.txt 2024-03-27T06:45:12.235 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiehrpwm.txt 2024-03-27T06:45:12.237 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tipwmss.txt 2024-03-27T06:45:12.239 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm.txt 2024-03-27T06:45:12.241 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pxa-pwm.txt 2024-03-27T06:45:12.242 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/renesas,tpu-pwm.txt 2024-03-27T06:45:12.244 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/spear-pwm.txt 2024-03-27T06:45:12.245 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwm.txt 2024-03-27T06:45:12.246 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwmled.txt 2024-03-27T06:45:12.247 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/vt8500-pwm.txt 2024-03-27T06:45:12.248 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/ 2024-03-27T06:45:12.249 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/regmap.txt 2024-03-27T06:45:12.250 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ 2024-03-27T06:45:12.251 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm800.txt 2024-03-27T06:45:12.252 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm860x.txt 2024-03-27T06:45:12.253 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/act8865-regulator.txt 2024-03-27T06:45:12.254 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/anatop-regulator.txt 2024-03-27T06:45:12.255 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/as3722-regulator.txt 2024-03-27T06:45:12.257 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9210.txt 2024-03-27T06:45:12.258 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9211.txt 2024-03-27T06:45:12.259 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fan53555.txt 2024-03-27T06:45:12.260 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fixed-regulator.txt 2024-03-27T06:45:12.262 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/gpio-regulator.txt 2024-03-27T06:45:12.264 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/isl9305.txt 2024-03-27T06:45:12.265 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/lp872x.txt 2024-03-27T06:45:12.267 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ltc3589.txt 2024-03-27T06:45:12.268 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max1586-regulator.txt 2024-03-27T06:45:12.270 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max77802.txt 2024-03-27T06:45:12.272 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8660.txt 2024-03-27T06:45:12.274 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8907.txt 2024-03-27T06:45:12.275 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8925-regulator.txt 2024-03-27T06:45:12.277 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8952.txt 2024-03-27T06:45:12.279 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8973-regulator.txt 2024-03-27T06:45:12.281 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8997-regulator.txt 2024-03-27T06:45:12.286 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/mt6397-regulator.txt 2024-03-27T06:45:12.286 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/palmas-pmic.txt 2024-03-27T06:45:12.286 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pbias-regulator.txt 2024-03-27T06:45:12.288 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pfuze100.txt 2024-03-27T06:45:12.289 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pwm-regulator.txt 2024-03-27T06:45:12.291 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/regulator.txt 2024-03-27T06:45:12.293 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/s5m8767-regulator.txt 2024-03-27T06:45:12.294 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/sky81452-regulator.txt 2024-03-27T06:45:12.296 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ti-abb-regulator.txt 2024-03-27T06:45:12.298 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps51632-regulator.txt 2024-03-27T06:45:12.300 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps62360-regulator.txt 2024-03-27T06:45:12.302 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65090.txt 2024-03-27T06:45:12.303 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65217.txt 2024-03-27T06:45:12.305 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65218.txt 2024-03-27T06:45:12.306 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps6586x.txt 2024-03-27T06:45:12.307 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/twl-regulator.txt 2024-03-27T06:45:12.308 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/vexpress.txt 2024-03-27T06:45:12.309 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/ 2024-03-27T06:45:12.310 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt 2024-03-27T06:45:12.312 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/ 2024-03-27T06:45:12.313 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/allwinner,sunxi-clock-reset.txt 2024-03-27T06:45:12.314 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/fsl,imx-src.txt 2024-03-27T06:45:12.315 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/reset.txt 2024-03-27T06:45:12.316 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/sirf,rstc.txt 2024-03-27T06:45:12.317 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/socfpga-reset.txt 2024-03-27T06:45:12.318 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-picophyreset.txt 2024-03-27T06:45:12.319 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-powerdown.txt 2024-03-27T06:45:12.320 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-softreset.txt 2024-03-27T06:45:12.321 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/resource-names.txt 2024-03-27T06:45:12.323 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/ 2024-03-27T06:45:12.324 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/apm,rng.txt 2024-03-27T06:45:12.325 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/brcm,bcm2835.txt 2024-03-27T06:45:12.327 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/qcom,prng.txt 2024-03-27T06:45:12.328 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/ 2024-03-27T06:45:12.330 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/armada-380-rtc.txt 2024-03-27T06:45:12.332 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91rm9200-rtc.txt 2024-03-27T06:45:12.334 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91sam9-rtc.txt 2024-03-27T06:45:12.335 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dallas,ds1339.txt 2024-03-27T06:45:12.337 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dw-apb.txt 2024-03-27T06:45:12.338 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt 2024-03-27T06:45:12.340 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/imxdi-rtc.txt 2024-03-27T06:45:12.342 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/isil,isl12057.txt 2024-03-27T06:45:12.343 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/lpc32xx-rtc.txt 2024-03-27T06:45:12.345 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/maxim,ds1742.txt 2024-03-27T06:45:12.347 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/moxa,moxart-rtc.txt 2024-03-27T06:45:12.348 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt 2024-03-27T06:45:12.351 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nxp,rtc-2123.txt 2024-03-27T06:45:12.352 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/olpc-xo1-rtc.txt 2024-03-27T06:45:12.354 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/orion-rtc.txt 2024-03-27T06:45:12.355 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/pxa-rtc.txt 2024-03-27T06:45:12.357 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-cmos.txt 2024-03-27T06:45:12.359 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-omap.txt 2024-03-27T06:45:12.360 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-opal.txt 2024-03-27T06:45:12.362 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-palmas.txt 2024-03-27T06:45:12.363 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/s3c-rtc.txt 2024-03-27T06:45:12.365 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt 2024-03-27T06:45:12.366 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/snvs-rtc.txt 2024-03-27T06:45:12.367 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/spear-rtc.txt 2024-03-27T06:45:12.369 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/stmp3xxx-rtc.txt 2024-03-27T06:45:12.371 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sun6i-rtc.txt 2024-03-27T06:45:12.372 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sunxi-rtc.txt 2024-03-27T06:45:12.374 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/twl-rtc.txt 2024-03-27T06:45:12.376 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/via,vt8500-rtc.txt 2024-03-27T06:45:12.377 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/xgene-rtc.txt 2024-03-27T06:45:12.379 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/ 2024-03-27T06:45:12.381 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/ 2024-03-27T06:45:12.382 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/st33zp24-i2c.txt 2024-03-27T06:45:12.384 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/ 2024-03-27T06:45:12.386 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/8250.txt 2024-03-27T06:45:12.387 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_jtaguart.txt 2024-03-27T06:45:12.389 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_uart.txt 2024-03-27T06:45:12.391 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/arc-uart.txt 2024-03-27T06:45:12.392 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/atmel-usart.txt 2024-03-27T06:45:12.394 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/axis,etraxfs-uart.txt 2024-03-27T06:45:12.395 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/bcm63xx-uart.txt 2024-03-27T06:45:12.397 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cavium-uart.txt 2024-03-27T06:45:12.398 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cdns,uart.txt 2024-03-27T06:45:12.399 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cirrus,clps711x-uart.txt 2024-03-27T06:45:12.400 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/digicolor-usart.txt 2024-03-27T06:45:12.401 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/efm32-uart.txt 2024-03-27T06:45:12.402 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt 2024-03-27T06:45:12.404 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-lpuart.txt 2024-03-27T06:45:12.407 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt 2024-03-27T06:45:12.409 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/lantiq_asc.txt 2024-03-27T06:45:12.411 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/maxim,max310x.txt 2024-03-27T06:45:12.412 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl,pxa-ssp.txt 2024-03-27T06:45:12.414 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl-serial.txt 2024-03-27T06:45:12.415 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mtk-uart.txt 2024-03-27T06:45:12.417 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nvidia,tegra20-hsuart.txt 2024-03-27T06:45:12.418 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp,sc16is7xx.txt 2024-03-27T06:45:12.420 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp-lpc32xx-hsuart.txt 2024-03-27T06:45:12.422 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/omap_serial.txt 2024-03-27T06:45:12.423 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/pl011.txt 2024-03-27T06:45:12.425 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qca,ar9330-uart.txt 2024-03-27T06:45:12.427 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uart.txt 2024-03-27T06:45:12.428 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uartdm.txt 2024-03-27T06:45:12.430 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/renesas,sci-serial.txt 2024-03-27T06:45:12.432 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/rs485.txt 2024-03-27T06:45:12.433 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/samsung_uart.txt 2024-03-27T06:45:12.435 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sirf-uart.txt 2024-03-27T06:45:12.437 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.txt 2024-03-27T06:45:12.439 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sprd-uart.txt 2024-03-27T06:45:12.441 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/st-asc.txt 2024-03-27T06:45:12.442 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/vt8500-uart.txt 2024-03-27T06:45:12.444 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/ 2024-03-27T06:45:12.445 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/allwinner,sun4i-ps2.txt 2024-03-27T06:45:12.446 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/altera_ps2.txt 2024-03-27T06:45:12.447 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/olpc,ap-sp.txt 2024-03-27T06:45:12.449 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/snps-arc_ps2.txt 2024-03-27T06:45:12.450 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ 2024-03-27T06:45:12.451 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/ 2024-03-27T06:45:12.453 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman-portals.txt 2024-03-27T06:45:12.454 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman.txt 2024-03-27T06:45:12.456 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman-portals.txt 2024-03-27T06:45:12.458 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman.txt 2024-03-27T06:45:12.459 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/ 2024-03-27T06:45:12.460 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/qcom,gsbi.txt 2024-03-27T06:45:12.462 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/ 2024-03-27T06:45:12.463 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-dma.txt 2024-03-27T06:45:12.464 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt 2024-03-27T06:45:12.465 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ 2024-03-27T06:45:12.467 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,adau1701.txt 2024-03-27T06:45:12.468 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-i2s.txt 2024-03-27T06:45:12.469 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-spdif-tx.txt 2024-03-27T06:45:12.470 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,ssm2602.txt 2024-03-27T06:45:12.471 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4104.txt 2024-03-27T06:45:12.473 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4554.c 2024-03-27T06:45:12.474 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4642.txt 2024-03-27T06:45:12.475 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak5386.txt 2024-03-27T06:45:12.476 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5623.txt 2024-03-27T06:45:12.477 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5632.txt 2024-03-27T06:45:12.478 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/armada-370db-audio.txt 2024-03-27T06:45:12.479 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/arndale.txt 2024-03-27T06:45:12.480 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-at91sam9g20ek-wm8731-audio.txt 2024-03-27T06:45:12.481 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-sam9x5-wm8731-audio.txt 2024-03-27T06:45:12.482 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-wm8904.txt 2024-03-27T06:45:12.483 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel_ac97c.txt 2024-03-27T06:45:12.485 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/bcm2835-i2s.txt 2024-03-27T06:45:12.486 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cdns,xtfpga-i2s.txt 2024-03-27T06:45:12.491 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs35l32.txt 2024-03-27T06:45:12.491 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4265.txt 2024-03-27T06:45:12.492 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4270.txt 2024-03-27T06:45:12.494 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4271.txt 2024-03-27T06:45:12.495 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l52.txt 2024-03-27T06:45:12.497 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l56.txt 2024-03-27T06:45:12.499 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l73.txt 2024-03-27T06:45:12.501 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42xx8.txt 2024-03-27T06:45:12.502 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/da9055.txt 2024-03-27T06:45:12.504 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-evm-audio.txt 2024-03-27T06:45:12.506 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-mcasp-audio.txt 2024-03-27T06:45:12.508 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/designware-i2s.txt 2024-03-27T06:45:12.509 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/es8328.txt 2024-03-27T06:45:12.511 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/eukrea-tlv320.txt 2024-03-27T06:45:12.513 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,asrc.txt 2024-03-27T06:45:12.515 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,esai.txt 2024-03-27T06:45:12.517 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,spdif.txt 2024-03-27T06:45:12.518 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,ssi.txt 2024-03-27T06:45:12.520 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-asoc-card.txt 2024-03-27T06:45:12.522 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-sai.txt 2024-03-27T06:45:12.524 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/hdmi.txt 2024-03-27T06:45:12.526 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-es8328.txt 2024-03-27T06:45:12.528 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-sgtl5000.txt 2024-03-27T06:45:12.529 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-spdif.txt 2024-03-27T06:45:12.531 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-wm8962.txt 2024-03-27T06:45:12.533 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audmux.txt 2024-03-27T06:45:12.534 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ingenic,jz4740-i2s.txt 2024-03-27T06:45:12.535 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98090.txt 2024-03-27T06:45:12.537 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98095.txt 2024-03-27T06:45:12.539 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98357a.txt 2024-03-27T06:45:12.541 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa-ssp.txt 2024-03-27T06:45:12.542 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa2xx-pcm.txt 2024-03-27T06:45:12.544 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mvebu-audio.txt 2024-03-27T06:45:12.546 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-audio-sgtl5000.txt 2024-03-27T06:45:12.548 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-saif.txt 2024-03-27T06:45:12.550 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nokia,rx51.txt 2024-03-27T06:45:12.551 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-alc5632.txt 2024-03-27T06:45:12.553 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-max98090.txt 2024-03-27T06:45:12.555 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5640.txt 2024-03-27T06:45:12.557 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5677.txt 2024-03-27T06:45:12.558 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-trimslice.txt 2024-03-27T06:45:12.560 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8753.txt 2024-03-27T06:45:12.562 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8903.txt 2024-03-27T06:45:12.563 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm9712.txt 2024-03-27T06:45:12.565 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-ac97.txt 2024-03-27T06:45:12.567 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-das.txt 2024-03-27T06:45:12.569 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-i2s.txt 2024-03-27T06:45:12.570 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-ahub.txt 2024-03-27T06:45:12.572 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt 2024-03-27T06:45:12.574 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-i2s.txt 2024-03-27T06:45:12.576 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-abe-twl6040.txt 2024-03-27T06:45:12.577 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-dmic.txt 2024-03-27T06:45:12.578 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcbsp.txt 2024-03-27T06:45:12.579 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcpdm.txt 2024-03-27T06:45:12.580 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-twl4030.txt 2024-03-27T06:45:12.581 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm1792a.txt 2024-03-27T06:45:12.582 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm512x.txt 2024-03-27T06:45:12.583 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,fsi.txt 2024-03-27T06:45:12.584 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,rsnd.txt 2024-03-27T06:45:12.585 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rockchip-i2s.txt 2024-03-27T06:45:12.587 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5631.txt 2024-03-27T06:45:12.589 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5640.txt 2024-03-27T06:45:12.590 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5677.txt 2024-03-27T06:45:12.592 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,odroidx2-max98090.txt 2024-03-27T06:45:12.594 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,smdk-wm8994.txt 2024-03-27T06:45:12.596 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung-i2s.txt 2024-03-27T06:45:12.598 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sgtl5000.txt 2024-03-27T06:45:12.599 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/simple-card.txt 2024-03-27T06:45:12.601 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-codec.txt 2024-03-27T06:45:12.603 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-port.txt 2024-03-27T06:45:12.604 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio.txt 2024-03-27T06:45:12.606 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-usp.txt 2024-03-27T06:45:12.608 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/snow.txt 2024-03-27T06:45:12.609 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/soc-ac97link.txt 2024-03-27T06:45:12.611 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-receiver.txt 2024-03-27T06:45:12.613 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-transmitter.txt 2024-03-27T06:45:12.615 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm2518.txt 2024-03-27T06:45:12.616 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm4567.txt 2024-03-27T06:45:12.618 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta32x.txt 2024-03-27T06:45:12.620 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta350.txt 2024-03-27T06:45:12.622 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tas2552.txt 2024-03-27T06:45:12.623 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tdm-slot.txt 2024-03-27T06:45:12.625 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,pcm1681.txt 2024-03-27T06:45:12.627 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,tas5086.txt 2024-03-27T06:45:12.629 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic31xx.txt 2024-03-27T06:45:12.631 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic32x4.txt 2024-03-27T06:45:12.633 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic3x.txt 2024-03-27T06:45:12.634 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tpa6130a2.txt 2024-03-27T06:45:12.636 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ts3a227e.txt 2024-03-27T06:45:12.638 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-mop500.txt 2024-03-27T06:45:12.640 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-msp.txt 2024-03-27T06:45:12.641 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/widgets.txt 2024-03-27T06:45:12.643 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8510.txt 2024-03-27T06:45:12.645 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8523.txt 2024-03-27T06:45:12.646 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8580.txt 2024-03-27T06:45:12.648 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8711.txt 2024-03-27T06:45:12.650 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8728.txt 2024-03-27T06:45:12.651 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8731.txt 2024-03-27T06:45:12.653 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8737.txt 2024-03-27T06:45:12.655 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8741.txt 2024-03-27T06:45:12.657 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8750.txt 2024-03-27T06:45:12.658 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8753.txt 2024-03-27T06:45:12.660 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8770.txt 2024-03-27T06:45:12.662 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8776.txt 2024-03-27T06:45:12.663 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8804.txt 2024-03-27T06:45:12.664 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8903.txt 2024-03-27T06:45:12.665 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8904.txt 2024-03-27T06:45:12.666 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8960.txt 2024-03-27T06:45:12.667 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8962.txt 2024-03-27T06:45:12.668 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8994.txt 2024-03-27T06:45:12.669 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ 2024-03-27T06:45:12.670 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/brcm,bcm2835-spi.txt 2024-03-27T06:45:12.671 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/efm32-spi.txt 2024-03-27T06:45:12.672 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-imx-cspi.txt 2024-03-27T06:45:12.674 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-spi.txt 2024-03-27T06:45:12.674 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/mxs-spi.txt 2024-03-27T06:45:12.676 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra114-spi.txt 2024-03-27T06:45:12.678 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-sflash.txt 2024-03-27T06:45:12.680 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-slink.txt 2024-03-27T06:45:12.681 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/omap-spi.txt 2024-03-27T06:45:12.683 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/qcom,spi-qup.txt 2024-03-27T06:45:12.685 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-hspi.txt 2024-03-27T06:45:12.686 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-msiof.txt 2024-03-27T06:45:12.687 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/snps,dw-apb-ssi.txt 2024-03-27T06:45:12.688 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-bus.txt 2024-03-27T06:45:12.689 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-cadence.txt 2024-03-27T06:45:12.691 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-davinci.txt 2024-03-27T06:45:12.692 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-dw.txt 2024-03-27T06:45:12.693 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt 2024-03-27T06:45:12.694 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-gpio.txt 2024-03-27T06:45:12.695 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-img-spfi.txt 2024-03-27T06:45:12.697 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-meson.txt 2024-03-27T06:45:12.698 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-octeon.txt 2024-03-27T06:45:12.699 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-orion.txt 2024-03-27T06:45:12.700 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rockchip.txt 2024-03-27T06:45:12.701 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rspi.txt 2024-03-27T06:45:12.702 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-samsung.txt 2024-03-27T06:45:12.703 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sc18is602.txt 2024-03-27T06:45:12.704 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sirf.txt 2024-03-27T06:45:12.706 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-st-ssc.txt 2024-03-27T06:45:12.707 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun4i.txt 2024-03-27T06:45:12.708 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun6i.txt 2024-03-27T06:45:12.710 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-xtensa-xtfpga.txt 2024-03-27T06:45:12.711 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_altera.txt 2024-03-27T06:45:12.713 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_atmel.txt 2024-03-27T06:45:12.715 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_oc_tiny.txt 2024-03-27T06:45:12.716 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_pl022.txt 2024-03-27T06:45:12.718 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ti_qspi.txt 2024-03-27T06:45:12.720 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/ 2024-03-27T06:45:12.722 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/qcom,spmi-pmic-arb.txt 2024-03-27T06:45:12.723 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/spmi.txt 2024-03-27T06:45:12.725 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/ 2024-03-27T06:45:12.727 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/ 2024-03-27T06:45:12.728 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/ 2024-03-27T06:45:12.729 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/lpc32xx-adc.txt 2024-03-27T06:45:12.730 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt 2024-03-27T06:45:12.731 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/spear-adc.txt 2024-03-27T06:45:12.732 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/submitting-patches.txt 2024-03-27T06:45:12.733 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ 2024-03-27T06:45:12.735 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/armada-thermal.txt 2024-03-27T06:45:12.736 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/db8500-thermal.txt 2024-03-27T06:45:12.737 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/dove-thermal.txt 2024-03-27T06:45:12.738 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/exynos-thermal.txt 2024-03-27T06:45:12.739 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/imx-thermal.txt 2024-03-27T06:45:12.740 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/kirkwood-thermal.txt 2024-03-27T06:45:12.741 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rcar-thermal.txt 2024-03-27T06:45:12.743 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rockchip-thermal.txt 2024-03-27T06:45:12.744 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/spear-thermal.txt 2024-03-27T06:45:12.745 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/st-thermal.txt 2024-03-27T06:45:12.746 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt 2024-03-27T06:45:12.747 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/thermal.txt 2024-03-27T06:45:12.748 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ti_soc_thermal.txt 2024-03-27T06:45:12.749 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ 2024-03-27T06:45:12.750 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun4i-timer.txt 2024-03-27T06:45:12.751 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun5i-a13-hstimer.txt 2024-03-27T06:45:12.752 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/amlogic,meson6-timer.txt 2024-03-27T06:45:12.753 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/arm,sp804.txt 2024-03-27T06:45:12.754 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/brcm,bcm2835-system-timer.txt 2024-03-27T06:45:12.755 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cadence,ttc-timer.txt 2024-03-27T06:45:12.757 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cirrus,clps711x-timer.txt 2024-03-27T06:45:12.758 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/digicolor-timer.txt 2024-03-27T06:45:12.759 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/energymicro,efm32-timer.txt 2024-03-27T06:45:12.760 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,ftm-timer.txt 2024-03-27T06:45:12.761 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt 2024-03-27T06:45:12.762 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/lsi,zevio-timer.txt 2024-03-27T06:45:12.764 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,armada-370-xp-timer.txt 2024-03-27T06:45:12.766 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,orion-timer.txt 2024-03-27T06:45:12.767 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/mediatek,mtk-timer.txt 2024-03-27T06:45:12.769 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/moxa,moxart-timer.txt 2024-03-27T06:45:12.771 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra20-timer.txt 2024-03-27T06:45:12.772 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra30-timer.txt 2024-03-27T06:45:12.774 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,cmt.txt 2024-03-27T06:45:12.776 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,mtu2.txt 2024-03-27T06:45:12.778 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,tmu.txt 2024-03-27T06:45:12.780 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/rockchip,rk3288-timer.txt 2024-03-27T06:45:12.781 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/samsung,exynos4210-mct.txt 2024-03-27T06:45:12.783 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/stericsson-u300-apptimer.txt 2024-03-27T06:45:12.785 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ti,keystone-timer.txt 2024-03-27T06:45:12.787 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ 2024-03-27T06:45:12.788 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ufshcd-pltfrm.txt 2024-03-27T06:45:12.790 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/unittest.txt 2024-03-27T06:45:12.792 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ 2024-03-27T06:45:12.794 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/am33xx-usb.txt 2024-03-27T06:45:12.795 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/atmel-usb.txt 2024-03-27T06:45:12.796 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-imx.txt 2024-03-27T06:45:12.798 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-qcom.txt 2024-03-27T06:45:12.799 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-usb2.txt 2024-03-27T06:45:12.800 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-zevio.txt 2024-03-27T06:45:12.802 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc2.txt 2024-03-27T06:45:12.803 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3-st.txt 2024-03-27T06:45:12.805 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3.txt 2024-03-27T06:45:12.807 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-omap.txt 2024-03-27T06:45:12.808 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-orion.txt 2024-03-27T06:45:12.810 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-st.txt 2024-03-27T06:45:12.812 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/exynos-usb.txt 2024-03-27T06:45:12.813 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/fsl-usb.txt 2024-03-27T06:45:12.814 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/generic.txt 2024-03-27T06:45:12.815 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/gr-udc.txt 2024-03-27T06:45:12.817 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/isp1301.txt 2024-03-27T06:45:12.818 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-phy.txt 2024-03-27T06:45:12.819 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-usb.txt 2024-03-27T06:45:12.820 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/lpc32xx-udc.txt 2024-03-27T06:45:12.821 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/msm-hsusb.txt 2024-03-27T06:45:12.822 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/mxs-phy.txt 2024-03-27T06:45:12.823 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-ehci.txt 2024-03-27T06:45:12.825 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-usb-phy.txt 2024-03-27T06:45:12.826 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-nxp.txt 2024-03-27T06:45:12.827 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-omap3.txt 2024-03-27T06:45:12.828 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-st.txt 2024-03-27T06:45:12.829 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/omap-usb.txt 2024-03-27T06:45:12.830 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/pxa-usb.txt 2024-03-27T06:45:12.831 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/qcom,dwc3.txt 2024-03-27T06:45:12.832 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/renesas_usbhs.txt 2024-03-27T06:45:12.833 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-hsotg.txt 2024-03-27T06:45:12.834 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-usbphy.txt 2024-03-27T06:45:12.836 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/spear-usb.txt 2024-03-27T06:45:12.837 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/twlxxxx-usb.txt 2024-03-27T06:45:12.838 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/udc-xilinx.txt 2024-03-27T06:45:12.839 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ehci.txt 2024-03-27T06:45:12.840 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-nop-xceiv.txt 2024-03-27T06:45:12.841 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ohci.txt 2024-03-27T06:45:12.842 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-uhci.txt 2024-03-27T06:45:12.843 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-xhci.txt 2024-03-27T06:45:12.845 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb3503.txt 2024-03-27T06:45:12.846 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usbmisc-imx.txt 2024-03-27T06:45:12.847 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ux500-usb.txt 2024-03-27T06:45:12.848 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/vendor-prefixes.txt 2024-03-27T06:45:12.849 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ 2024-03-27T06:45:12.851 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7123.txt 2024-03-27T06:45:12.852 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7511.txt 2024-03-27T06:45:12.853 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/analog-tv-connector.txt 2024-03-27T06:45:12.854 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/arm,pl11x.txt 2024-03-27T06:45:12.855 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/atmel,lcdc.txt 2024-03-27T06:45:12.856 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/ 2024-03-27T06:45:12.858 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/88pm860x.txt 2024-03-27T06:45:12.858 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/gpio-backlight.txt 2024-03-27T06:45:12.860 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/lp855x.txt 2024-03-27T06:45:12.861 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/max8925-backlight.txt 2024-03-27T06:45:12.862 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/pwm-backlight.txt 2024-03-27T06:45:12.863 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/tps65217-backlight.txt 2024-03-27T06:45:12.864 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ 2024-03-27T06:45:12.865 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ps8622.txt 2024-03-27T06:45:12.866 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ptn3460.txt 2024-03-27T06:45:12.867 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/cirrus,clps711x-fb.txt 2024-03-27T06:45:12.868 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/display-timing.txt 2024-03-27T06:45:12.869 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dvi-connector.txt 2024-03-27T06:45:12.870 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dw_hdmi-rockchip.txt 2024-03-27T06:45:12.871 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos7-decon.txt 2024-03-27T06:45:12.873 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dp.txt 2024-03-27T06:45:12.874 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dsim.txt 2024-03-27T06:45:12.875 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmi.txt 2024-03-27T06:45:12.876 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiddc.txt 2024-03-27T06:45:12.877 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiphy.txt 2024-03-27T06:45:12.878 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_mixer.txt 2024-03-27T06:45:12.880 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/fsl,imx-fb.txt 2024-03-27T06:45:12.881 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/hdmi-connector.txt 2024-03-27T06:45:12.882 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/lgphilips,lb035q02.txt 2024-03-27T06:45:12.883 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dpi.txt 2024-03-27T06:45:12.884 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dsi-cm.txt 2024-03-27T06:45:12.885 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/renesas,du.txt 2024-03-27T06:45:12.886 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-drm.txt 2024-03-27T06:45:12.887 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-vop.txt 2024-03-27T06:45:13.391 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/samsung-fimd.txt 2024-03-27T06:45:13.392 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sharp,ls037v7dw01.txt 2024-03-27T06:45:13.393 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer-sunxi.txt 2024-03-27T06:45:13.395 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer.txt 2024-03-27T06:45:13.396 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sony,acx565akm.txt 2024-03-27T06:45:13.397 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1289fb.txt 2024-03-27T06:45:13.399 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1307fb.txt 2024-03-27T06:45:13.400 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/thine,thc63lvdm83d 2024-03-27T06:45:13.402 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,dra7-dss.txt 2024-03-27T06:45:13.403 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap-dss.txt 2024-03-27T06:45:13.405 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap2-dss.txt 2024-03-27T06:45:13.407 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap3-dss.txt 2024-03-27T06:45:13.409 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap4-dss.txt 2024-03-27T06:45:13.411 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap5-dss.txt 2024-03-27T06:45:13.412 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,opa362.txt 2024-03-27T06:45:13.413 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tfp410.txt 2024-03-27T06:45:13.415 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tpd12s015.txt 2024-03-27T06:45:13.417 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/toppoly,td028ttec1.txt 2024-03-27T06:45:13.418 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/tpo,td043mtea1.txt 2024-03-27T06:45:13.420 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/vga-connector.txt 2024-03-27T06:45:13.422 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/via,vt8500-fb.txt 2024-03-27T06:45:13.423 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,prizm-ge-rops.txt 2024-03-27T06:45:13.424 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,wm8505-fb.txt 2024-03-27T06:45:13.425 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/ 2024-03-27T06:45:13.427 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/mmio.txt 2024-03-27T06:45:13.428 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/ 2024-03-27T06:45:13.429 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/fsl-imx-owire.txt 2024-03-27T06:45:13.430 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/omap-hdq.txt 2024-03-27T06:45:13.431 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/w1-gpio.txt 2024-03-27T06:45:13.432 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ 2024-03-27T06:45:13.434 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-at91rm9200-wdt.txt 2024-03-27T06:45:13.435 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-wdt.txt 2024-03-27T06:45:13.436 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/brcm,bcm2835-pm-wdog.txt 2024-03-27T06:45:13.437 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/cadence-wdt.txt 2024-03-27T06:45:13.438 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/davinci-wdt.txt 2024-03-27T06:45:13.439 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/dw_wdt.txt 2024-03-27T06:45:13.440 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/fsl-imx-wdt.txt 2024-03-27T06:45:13.441 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/gpio-wdt.txt 2024-03-27T06:45:13.443 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/imgpdc-wdt.txt 2024-03-27T06:45:13.444 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ingenic,jz4740-wdt.txt 2024-03-27T06:45:13.446 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/marvel.txt 2024-03-27T06:45:13.448 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/men-a021-wdt.txt 2024-03-27T06:45:13.449 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/meson6-wdt.txt 2024-03-27T06:45:13.451 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/moxa,moxart-watchdog.txt 2024-03-27T06:45:13.453 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/mtk-wdt.txt 2024-03-27T06:45:13.455 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/of-xilinx-wdt.txt 2024-03-27T06:45:13.456 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/omap-wdt.txt 2024-03-27T06:45:13.458 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/pnx4008-wdt.txt 2024-03-27T06:45:13.460 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qca-ar7130-wdt.txt 2024-03-27T06:45:13.462 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qcom-wdt.txt 2024-03-27T06:45:13.464 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/rt2880-wdt.txt 2024-03-27T06:45:13.465 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/samsung-wdt.txt 2024-03-27T06:45:13.467 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sirfsoc_wdt.txt 2024-03-27T06:45:13.469 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/stericsson-coh901327.txt 2024-03-27T06:45:13.471 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sunxi-wdt.txt 2024-03-27T06:45:13.473 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/twl4030-wdt.txt 2024-03-27T06:45:13.474 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ 2024-03-27T06:45:13.475 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ce4100.txt 2024-03-27T06:45:13.476 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/interrupt.txt 2024-03-27T06:45:13.477 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/timer.txt 2024-03-27T06:45:13.478 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/xilinx.txt 2024-03-27T06:45:13.479 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/ 2024-03-27T06:45:13.480 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/xillybus.txt 2024-03-27T06:45:13.481 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/booting-without-of.txt 2024-03-27T06:45:13.483 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/changesets.txt 2024-03-27T06:45:13.484 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/dynamic-resolution-notes.txt 2024-03-27T06:45:13.485 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/of_selftest.txt 2024-03-27T06:45:13.486 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/overlay-notes.txt 2024-03-27T06:45:13.487 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/todo.txt 2024-03-27T06:45:13.488 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/devicetree/usage-model.txt 2024-03-27T06:45:13.489 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/digsig.txt 2024-03-27T06:45:13.491 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dma-buf-sharing.txt 2024-03-27T06:45:13.492 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dmaengine/ 2024-03-27T06:45:13.494 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dmaengine/00-INDEX 2024-03-27T06:45:13.496 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dmaengine/client.txt 2024-03-27T06:45:13.497 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dmaengine/dmatest.txt 2024-03-27T06:45:13.499 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dmaengine/provider.txt 2024-03-27T06:45:13.501 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dontdiff 2024-03-27T06:45:13.502 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/driver-model/ 2024-03-27T06:45:13.504 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/driver-model/binding.txt 2024-03-27T06:45:13.506 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/driver-model/bus.txt 2024-03-27T06:45:13.507 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/driver-model/class.txt 2024-03-27T06:45:13.508 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/driver-model/design-patterns.txt 2024-03-27T06:45:13.509 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/driver-model/device.txt 2024-03-27T06:45:13.510 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/driver-model/devres.txt 2024-03-27T06:45:13.512 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/driver-model/driver.txt 2024-03-27T06:45:13.514 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/driver-model/overview.txt 2024-03-27T06:45:13.515 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/driver-model/platform.txt 2024-03-27T06:45:13.517 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/driver-model/porting.txt 2024-03-27T06:45:13.519 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/ 2024-03-27T06:45:13.520 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/README.dvb-usb 2024-03-27T06:45:13.522 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/avermedia.txt 2024-03-27T06:45:13.524 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/bt8xx.txt 2024-03-27T06:45:13.525 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/cards.txt 2024-03-27T06:45:13.527 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/ci.txt 2024-03-27T06:45:13.529 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/contributors.txt 2024-03-27T06:45:13.531 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/faq.txt 2024-03-27T06:45:13.532 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/get_dvb_firmware 2024-03-27T06:45:13.533 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/lmedm04.txt 2024-03-27T06:45:13.535 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/opera-firmware.txt 2024-03-27T06:45:13.536 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/readme.txt 2024-03-27T06:45:13.537 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/technisat.txt 2024-03-27T06:45:13.538 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/ttusb-dec.txt 2024-03-27T06:45:13.539 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dvb/udev.txt 2024-03-27T06:45:13.540 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/dynamic-debug-howto.txt 2024-03-27T06:45:13.542 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/early-userspace/ 2024-03-27T06:45:13.543 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/early-userspace/README 2024-03-27T06:45:13.544 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/early-userspace/buffer-format.txt 2024-03-27T06:45:13.545 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/edac.txt 2024-03-27T06:45:13.546 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/efi-stub.txt 2024-03-27T06:45:13.547 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/eisa.txt 2024-03-27T06:45:13.548 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/email-clients.txt 2024-03-27T06:45:13.549 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/extcon/ 2024-03-27T06:45:13.550 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/extcon/porting-android-switch-class 2024-03-27T06:45:13.551 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fault-injection/ 2024-03-27T06:45:13.553 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fault-injection/fault-injection.txt 2024-03-27T06:45:13.554 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fault-injection/notifier-error-inject.txt 2024-03-27T06:45:13.555 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fault-injection/provoke-crashes.txt 2024-03-27T06:45:13.556 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/ 2024-03-27T06:45:13.557 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/00-INDEX 2024-03-27T06:45:13.558 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/api.txt 2024-03-27T06:45:13.559 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/arkfb.txt 2024-03-27T06:45:13.560 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/aty128fb.txt 2024-03-27T06:45:13.561 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/cirrusfb.txt 2024-03-27T06:45:13.562 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/cmap_xfbdev.txt 2024-03-27T06:45:13.563 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/deferred_io.txt 2024-03-27T06:45:13.564 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/efifb.txt 2024-03-27T06:45:13.565 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/ep93xx-fb.txt 2024-03-27T06:45:13.566 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/fbcon.txt 2024-03-27T06:45:13.567 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/framebuffer.txt 2024-03-27T06:45:13.568 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/gxfb.txt 2024-03-27T06:45:13.570 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/intel810.txt 2024-03-27T06:45:13.572 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/intelfb.txt 2024-03-27T06:45:13.573 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/internals.txt 2024-03-27T06:45:13.574 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/lxfb.txt 2024-03-27T06:45:13.575 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/matroxfb.txt 2024-03-27T06:45:13.576 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/metronomefb.txt 2024-03-27T06:45:13.577 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/modedb.txt 2024-03-27T06:45:13.578 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/pvr2fb.txt 2024-03-27T06:45:13.579 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/pxafb.txt 2024-03-27T06:45:13.580 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/s3fb.txt 2024-03-27T06:45:13.581 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/sa1100fb.txt 2024-03-27T06:45:13.582 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/sh7760fb.txt 2024-03-27T06:45:13.583 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/sisfb.txt 2024-03-27T06:45:13.584 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/sm501.txt 2024-03-27T06:45:13.585 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/sstfb.txt 2024-03-27T06:45:13.586 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/tgafb.txt 2024-03-27T06:45:13.587 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/tridentfb.txt 2024-03-27T06:45:13.588 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/udlfb.txt 2024-03-27T06:45:13.589 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/uvesafb.txt 2024-03-27T06:45:13.590 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/vesafb.txt 2024-03-27T06:45:13.591 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/viafb.modes 2024-03-27T06:45:13.592 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/viafb.txt 2024-03-27T06:45:13.593 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fb/vt8623fb.txt 2024-03-27T06:45:13.594 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ 2024-03-27T06:45:13.595 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/.gitignore 2024-03-27T06:45:13.596 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/00-INDEX 2024-03-27T06:45:13.597 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/9p.txt 2024-03-27T06:45:13.598 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/Locking 2024-03-27T06:45:13.599 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/Makefile 2024-03-27T06:45:13.600 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/adfs.txt 2024-03-27T06:45:13.601 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/affs.txt 2024-03-27T06:45:13.602 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/afs.txt 2024-03-27T06:45:13.603 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/autofs4-mount-control.txt 2024-03-27T06:45:13.604 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/autofs4.txt 2024-03-27T06:45:13.605 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/automount-support.txt 2024-03-27T06:45:13.606 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/befs.txt 2024-03-27T06:45:13.607 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/bfs.txt 2024-03-27T06:45:13.608 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/btrfs.txt 2024-03-27T06:45:13.609 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/caching/ 2024-03-27T06:45:13.610 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/caching/backend-api.txt 2024-03-27T06:45:13.611 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/caching/cachefiles.txt 2024-03-27T06:45:13.612 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/caching/fscache.txt 2024-03-27T06:45:13.613 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/caching/netfs-api.txt 2024-03-27T06:45:13.614 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/caching/object.txt 2024-03-27T06:45:13.615 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/caching/operations.txt 2024-03-27T06:45:13.616 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ceph.txt 2024-03-27T06:45:13.617 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/cifs/ 2024-03-27T06:45:13.618 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/cifs/AUTHORS 2024-03-27T06:45:13.619 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/cifs/CHANGES 2024-03-27T06:45:13.620 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/cifs/README 2024-03-27T06:45:13.621 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/cifs/TODO 2024-03-27T06:45:13.622 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/cifs/cifs.txt 2024-03-27T06:45:13.623 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/cifs/winucase_convert.pl 2024-03-27T06:45:13.624 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/coda.txt 2024-03-27T06:45:13.625 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/configfs/ 2024-03-27T06:45:13.626 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/configfs/Makefile 2024-03-27T06:45:13.627 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs.txt 2024-03-27T06:45:13.628 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_explicit.c 2024-03-27T06:45:13.629 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_macros.c 2024-03-27T06:45:13.630 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/cramfs.txt 2024-03-27T06:45:13.631 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/dax.txt 2024-03-27T06:45:13.632 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/debugfs.txt 2024-03-27T06:45:13.633 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/devpts.txt 2024-03-27T06:45:13.634 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/directory-locking 2024-03-27T06:45:13.635 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/dlmfs.txt 2024-03-27T06:45:13.636 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/dnotify.txt 2024-03-27T06:45:13.637 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/dnotify_test.c 2024-03-27T06:45:13.639 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ecryptfs.txt 2024-03-27T06:45:13.640 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/efivarfs.txt 2024-03-27T06:45:13.642 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/exofs.txt 2024-03-27T06:45:13.644 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ext2.txt 2024-03-27T06:45:13.646 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ext3.txt 2024-03-27T06:45:13.647 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ext4.txt 2024-03-27T06:45:13.649 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/f2fs.txt 2024-03-27T06:45:13.651 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/fiemap.txt 2024-03-27T06:45:13.653 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/files.txt 2024-03-27T06:45:13.654 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/fuse.txt 2024-03-27T06:45:13.656 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/gfs2-glocks.txt 2024-03-27T06:45:13.658 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/gfs2-uevents.txt 2024-03-27T06:45:13.659 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/gfs2.txt 2024-03-27T06:45:13.661 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/hfs.txt 2024-03-27T06:45:13.663 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/hfsplus.txt 2024-03-27T06:45:13.665 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/hpfs.txt 2024-03-27T06:45:13.666 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/inotify.txt 2024-03-27T06:45:13.668 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/isofs.txt 2024-03-27T06:45:13.669 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/jfs.txt 2024-03-27T06:45:13.671 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/locks.txt 2024-03-27T06:45:13.673 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/logfs.txt 2024-03-27T06:45:13.674 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/mandatory-locking.txt 2024-03-27T06:45:13.676 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ncpfs.txt 2024-03-27T06:45:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/ 2024-03-27T06:45:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/00-INDEX 2024-03-27T06:45:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/Exporting 2024-03-27T06:45:13.683 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/fault_injection.txt 2024-03-27T06:45:13.684 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/idmapper.txt 2024-03-27T06:45:13.686 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/knfsd-stats.txt 2024-03-27T06:45:13.688 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs-rdma.txt 2024-03-27T06:45:13.689 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs.txt 2024-03-27T06:45:13.691 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs41-server.txt 2024-03-27T06:45:13.693 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsd-admin-interfaces.txt 2024-03-27T06:45:13.694 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsroot.txt 2024-03-27T06:45:13.696 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs-block-server.txt 2024-03-27T06:45:13.698 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs.txt 2024-03-27T06:45:13.699 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-cache.txt 2024-03-27T06:45:13.701 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-server-gss.txt 2024-03-27T06:45:13.702 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/nilfs2.txt 2024-03-27T06:45:13.704 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ntfs.txt 2024-03-27T06:45:13.706 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ocfs2.txt 2024-03-27T06:45:13.707 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/omfs.txt 2024-03-27T06:45:13.709 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/overlayfs.txt 2024-03-27T06:45:13.711 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/path-lookup.txt 2024-03-27T06:45:13.712 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/ 2024-03-27T06:45:13.714 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/design_notes.txt 2024-03-27T06:45:13.719 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/info.txt 2024-03-27T06:45:13.719 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/network_protocol.txt 2024-03-27T06:45:13.719 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/porting 2024-03-27T06:45:13.720 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/proc.txt 2024-03-27T06:45:13.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:13 smithi156 ceph-mon[36261]: pgmap v52: 97 pgs: 97 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 47 op/s 2024-03-27T06:45:13.722 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/qnx6.txt 2024-03-27T06:45:13.724 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/quota.txt 2024-03-27T06:45:13.726 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ramfs-rootfs-initramfs.txt 2024-03-27T06:45:13.727 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/relay.txt 2024-03-27T06:45:13.729 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/romfs.txt 2024-03-27T06:45:13.731 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/seq_file.txt 2024-03-27T06:45:13.733 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/sharedsubtree.txt 2024-03-27T06:45:13.734 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/spufs.txt 2024-03-27T06:45:13.736 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/squashfs.txt 2024-03-27T06:45:13.737 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/sysfs-pci.txt 2024-03-27T06:45:13.739 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/sysfs-tagging.txt 2024-03-27T06:45:13.741 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/sysfs.txt 2024-03-27T06:45:13.742 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/sysv-fs.txt 2024-03-27T06:45:13.744 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/tmpfs.txt 2024-03-27T06:45:13.746 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ubifs.txt 2024-03-27T06:45:13.747 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/udf.txt 2024-03-27T06:45:13.749 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/ufs.txt 2024-03-27T06:45:13.751 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/vfat.txt 2024-03-27T06:45:13.753 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/vfs.txt 2024-03-27T06:45:13.754 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/xfs-delayed-logging-design.txt 2024-03-27T06:45:13.756 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/xfs-self-describing-metadata.txt 2024-03-27T06:45:13.758 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/filesystems/xfs.txt 2024-03-27T06:45:13.759 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/firmware_class/ 2024-03-27T06:45:13.761 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/firmware_class/README 2024-03-27T06:45:13.763 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/firmware_class/hotplug-script 2024-03-27T06:45:13.764 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/flexible-arrays.txt 2024-03-27T06:45:13.766 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/ 2024-03-27T06:45:13.768 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/00-INDEX 2024-03-27T06:45:13.769 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/API.txt 2024-03-27T06:45:13.771 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/FMC-and-SDB.txt 2024-03-27T06:45:13.773 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/carrier.txt 2024-03-27T06:45:13.774 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/fmc-chardev.txt 2024-03-27T06:45:13.776 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/fmc-fakedev.txt 2024-03-27T06:45:13.777 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/fmc-trivial.txt 2024-03-27T06:45:13.779 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/fmc-write-eeprom.txt 2024-03-27T06:45:13.781 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/identifiers.txt 2024-03-27T06:45:13.782 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/mezzanine.txt 2024-03-27T06:45:13.784 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/fmc/parameters.txt 2024-03-27T06:45:13.785 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/frv/ 2024-03-27T06:45:13.787 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/frv/README.txt 2024-03-27T06:45:13.789 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/frv/atomic-ops.txt 2024-03-27T06:45:13.790 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/frv/booting.txt 2024-03-27T06:45:13.792 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/frv/clock.txt 2024-03-27T06:45:13.793 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/frv/configuring.txt 2024-03-27T06:45:13.795 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/frv/features.txt 2024-03-27T06:45:13.797 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/frv/gdbinit 2024-03-27T06:45:13.798 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/frv/gdbstub.txt 2024-03-27T06:45:13.800 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/frv/kernel-ABI.txt 2024-03-27T06:45:13.802 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/frv/mmu-layout.txt 2024-03-27T06:45:13.803 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/futex-requeue-pi.txt 2024-03-27T06:45:13.805 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/gcov.txt 2024-03-27T06:45:13.807 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/gdb-kernel-debugging.txt 2024-03-27T06:45:13.809 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/gpio/ 2024-03-27T06:45:13.810 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/gpio/00-INDEX 2024-03-27T06:45:13.812 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/gpio/board.txt 2024-03-27T06:45:13.813 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/gpio/consumer.txt 2024-03-27T06:45:13.815 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/gpio/driver.txt 2024-03-27T06:45:13.817 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/gpio/gpio-legacy.txt 2024-03-27T06:45:13.818 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/gpio/gpio.txt 2024-03-27T06:45:13.820 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/gpio/sysfs.txt 2024-03-27T06:45:13.821 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hid/ 2024-03-27T06:45:13.823 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hid/hid-sensor.txt 2024-03-27T06:45:13.824 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hid/hid-transport.txt 2024-03-27T06:45:13.826 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hid/hiddev.txt 2024-03-27T06:45:13.828 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hid/hidraw.txt 2024-03-27T06:45:13.829 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hid/uhid.txt 2024-03-27T06:45:13.831 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/highuid.txt 2024-03-27T06:45:13.833 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hsi.txt 2024-03-27T06:45:13.834 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hw_random.txt 2024-03-27T06:45:13.835 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ 2024-03-27T06:45:13.836 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ab8500 2024-03-27T06:45:13.838 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/abituguru 2024-03-27T06:45:13.839 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/abituguru-datasheet 2024-03-27T06:45:13.840 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/abituguru3 2024-03-27T06:45:13.840 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/abx500 2024-03-27T06:45:13.842 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/acpi_power_meter 2024-03-27T06:45:13.843 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ad7314 2024-03-27T06:45:13.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:13 smithi170 ceph-mon[36438]: pgmap v52: 97 pgs: 97 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 47 op/s 2024-03-27T06:45:13.844 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adc128d818 2024-03-27T06:45:13.845 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adm1021 2024-03-27T06:45:13.846 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adm1025 2024-03-27T06:45:13.848 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adm1026 2024-03-27T06:45:13.849 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adm1031 2024-03-27T06:45:13.851 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adm1275 2024-03-27T06:45:13.853 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adm9240 2024-03-27T06:45:13.855 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ads1015 2024-03-27T06:45:13.856 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ads7828 2024-03-27T06:45:13.857 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adt7410 2024-03-27T06:45:13.858 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adt7411 2024-03-27T06:45:13.859 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adt7462 2024-03-27T06:45:13.860 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adt7470 2024-03-27T06:45:13.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:13 smithi016 ceph-mon[31908]: pgmap v52: 97 pgs: 97 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 47 op/s 2024-03-27T06:45:13.861 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/adt7475 2024-03-27T06:45:13.862 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/amc6821 2024-03-27T06:45:13.863 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/asb100 2024-03-27T06:45:13.864 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/asc7621 2024-03-27T06:45:13.865 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/coretemp 2024-03-27T06:45:13.866 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/da9052 2024-03-27T06:45:13.867 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/da9055 2024-03-27T06:45:13.868 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/dme1737 2024-03-27T06:45:13.870 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ds1621 2024-03-27T06:45:13.871 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ds620 2024-03-27T06:45:13.873 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/emc1403 2024-03-27T06:45:13.875 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/emc2103 2024-03-27T06:45:13.876 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/emc6w201 2024-03-27T06:45:13.878 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/f71805f 2024-03-27T06:45:13.880 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/f71882fg 2024-03-27T06:45:13.881 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/fam15h_power 2024-03-27T06:45:13.883 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/g760a 2024-03-27T06:45:13.884 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/g762 2024-03-27T06:45:13.886 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/gl518sm 2024-03-27T06:45:13.888 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/hih6130 2024-03-27T06:45:13.890 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/htu21 2024-03-27T06:45:13.891 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/hwmon-kernel-api.txt 2024-03-27T06:45:13.893 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ibmaem 2024-03-27T06:45:13.894 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ibmpowernv 2024-03-27T06:45:13.895 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ina209 2024-03-27T06:45:13.897 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ina2xx 2024-03-27T06:45:13.899 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/it87 2024-03-27T06:45:13.900 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/jc42 2024-03-27T06:45:13.902 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/k10temp 2024-03-27T06:45:13.903 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/k8temp 2024-03-27T06:45:13.905 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lineage-pem 2024-03-27T06:45:13.907 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm25066 2024-03-27T06:45:13.908 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm63 2024-03-27T06:45:13.910 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm70 2024-03-27T06:45:13.912 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm73 2024-03-27T06:45:13.913 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm75 2024-03-27T06:45:13.915 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm77 2024-03-27T06:45:13.916 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm78 2024-03-27T06:45:13.918 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm80 2024-03-27T06:45:13.920 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm83 2024-03-27T06:45:13.921 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm85 2024-03-27T06:45:13.923 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm87 2024-03-27T06:45:13.924 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm90 2024-03-27T06:45:13.926 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm92 2024-03-27T06:45:13.928 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm93 2024-03-27T06:45:13.929 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm95234 2024-03-27T06:45:13.931 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/lm95245 2024-03-27T06:45:13.932 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ltc2945 2024-03-27T06:45:13.933 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ltc2978 2024-03-27T06:45:13.934 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ltc4151 2024-03-27T06:45:13.935 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ltc4215 2024-03-27T06:45:13.936 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ltc4245 2024-03-27T06:45:13.937 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ltc4260 2024-03-27T06:45:13.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ltc4261 2024-03-27T06:45:13.939 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/max16064 2024-03-27T06:45:13.940 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/max16065 2024-03-27T06:45:13.941 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/max1619 2024-03-27T06:45:13.942 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/max1668 2024-03-27T06:45:13.943 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/max197 2024-03-27T06:45:13.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/max34440 2024-03-27T06:45:13.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/max6639 2024-03-27T06:45:13.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/max6642 2024-03-27T06:45:13.947 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/max6650 2024-03-27T06:45:13.948 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/max6697 2024-03-27T06:45:13.949 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/max8688 2024-03-27T06:45:13.950 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/mc13783-adc 2024-03-27T06:45:13.951 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/mcp3021 2024-03-27T06:45:13.952 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/menf21bmc 2024-03-27T06:45:13.953 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/nct6683 2024-03-27T06:45:13.954 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/nct6775 2024-03-27T06:45:13.955 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/nct7802 2024-03-27T06:45:13.956 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ntc_thermistor 2024-03-27T06:45:13.957 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/pc87360 2024-03-27T06:45:13.958 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/pc87427 2024-03-27T06:45:13.959 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/pcf8591 2024-03-27T06:45:13.960 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/pmbus 2024-03-27T06:45:13.961 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/pmbus-core 2024-03-27T06:45:13.962 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/powr1220 2024-03-27T06:45:13.963 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/pwm-fan 2024-03-27T06:45:13.965 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/sch5627 2024-03-27T06:45:13.966 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/sch5636 2024-03-27T06:45:13.967 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/sht15 2024-03-27T06:45:13.968 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/sht21 2024-03-27T06:45:13.970 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/shtc1 2024-03-27T06:45:13.970 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/sis5595 2024-03-27T06:45:13.972 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/smm665 2024-03-27T06:45:13.972 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/smsc47b397 2024-03-27T06:45:13.973 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/smsc47m1 2024-03-27T06:45:13.974 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/smsc47m192 2024-03-27T06:45:13.975 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/submitting-patches 2024-03-27T06:45:13.976 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/sysfs-interface 2024-03-27T06:45:13.978 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/thmc50 2024-03-27T06:45:13.979 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/tmp102 2024-03-27T06:45:13.980 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/tmp103 2024-03-27T06:45:13.981 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/tmp401 2024-03-27T06:45:13.982 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/tmp421 2024-03-27T06:45:13.983 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/tps40422 2024-03-27T06:45:13.984 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/twl4030-madc-hwmon 2024-03-27T06:45:13.985 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ucd9000 2024-03-27T06:45:13.987 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/ucd9200 2024-03-27T06:45:13.988 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/userspace-tools 2024-03-27T06:45:13.989 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/vexpress 2024-03-27T06:45:13.990 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/via686a 2024-03-27T06:45:13.991 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/vt1211 2024-03-27T06:45:13.992 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/w83627ehf 2024-03-27T06:45:13.993 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/w83627hf 2024-03-27T06:45:13.994 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/w83781d 2024-03-27T06:45:13.995 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/w83791d 2024-03-27T06:45:13.997 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/w83792d 2024-03-27T06:45:13.998 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/w83793 2024-03-27T06:45:14.000 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/w83795 2024-03-27T06:45:14.002 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/w83l785ts 2024-03-27T06:45:14.003 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/w83l786ng 2024-03-27T06:45:14.005 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/wm831x 2024-03-27T06:45:14.007 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/wm8350 2024-03-27T06:45:14.008 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwmon/zl6100 2024-03-27T06:45:14.010 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/hwspinlock.txt 2024-03-27T06:45:14.012 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/ 2024-03-27T06:45:14.014 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/ 2024-03-27T06:45:14.015 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1535 2024-03-27T06:45:14.017 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1563 2024-03-27T06:45:14.018 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali15x3 2024-03-27T06:45:14.020 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd756 2024-03-27T06:45:14.021 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd8111 2024-03-27T06:45:14.022 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-diolan-u2c 2024-03-27T06:45:14.023 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-i801 2024-03-27T06:45:14.024 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ismt 2024-03-27T06:45:14.025 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-nforce2 2024-03-27T06:45:14.026 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ocores 2024-03-27T06:45:14.027 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport 2024-03-27T06:45:14.028 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport-light 2024-03-27T06:45:14.029 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-pca-isa 2024-03-27T06:45:14.030 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-piix4 2024-03-27T06:45:14.031 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis5595 2024-03-27T06:45:14.032 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis630 2024-03-27T06:45:14.033 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis96x 2024-03-27T06:45:14.034 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-taos-evm 2024-03-27T06:45:14.035 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-via 2024-03-27T06:45:14.036 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-viapro 2024-03-27T06:45:14.037 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/busses/scx200_acb 2024-03-27T06:45:14.038 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/dev-interface 2024-03-27T06:45:14.039 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/fault-codes 2024-03-27T06:45:14.040 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/functionality 2024-03-27T06:45:14.041 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/i2c-protocol 2024-03-27T06:45:14.043 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/i2c-stub 2024-03-27T06:45:14.044 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/instantiating-devices 2024-03-27T06:45:14.045 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/muxes/ 2024-03-27T06:45:14.047 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/muxes/i2c-mux-gpio 2024-03-27T06:45:14.048 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/old-module-parameters 2024-03-27T06:45:14.050 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/smbus-protocol 2024-03-27T06:45:14.052 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/summary 2024-03-27T06:45:14.054 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/ten-bit-addresses 2024-03-27T06:45:14.055 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/upgrading-clients 2024-03-27T06:45:14.057 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2c/writing-clients 2024-03-27T06:45:14.059 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2o/ 2024-03-27T06:45:14.060 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2o/README 2024-03-27T06:45:14.062 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/i2o/ioctl 2024-03-27T06:45:14.063 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/ 2024-03-27T06:45:14.065 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/.gitignore 2024-03-27T06:45:14.067 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/IRQ-redir.txt 2024-03-27T06:45:14.069 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/Makefile 2024-03-27T06:45:14.070 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/README 2024-03-27T06:45:14.072 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/aliasing-test.c 2024-03-27T06:45:14.073 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/aliasing.txt 2024-03-27T06:45:14.104 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:13 smithi170 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-03-27T06:45:14.104 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:13 smithi170 systemd[1]: Stopped Ceph mds.f for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:45:14.104 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:13 smithi170 systemd[1]: Starting Ceph mds.f for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:45:14.104 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:14 smithi170 podman[71001]: 2024-03-27 06:45:13.995393089 +0000 UTC m=+0.022005083 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:45:14.104 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:14 smithi170 podman[71001]: 2024-03-27 06:45:14.109904665 +0000 UTC m=+0.136516641 container create 0bba2c438fb3e88e7ece84b5cdf148ddd21dca12f63e129e82f06a7ac69a5582 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-27T06:45:14.323 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/efirtc.txt 2024-03-27T06:45:14.325 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/err_inject.txt 2024-03-27T06:45:14.327 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/fsys.txt 2024-03-27T06:45:14.329 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/mca.txt 2024-03-27T06:45:14.331 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/serial.txt 2024-03-27T06:45:14.333 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ia64/xen.txt 2024-03-27T06:45:14.335 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ide/ 2024-03-27T06:45:14.340 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ide/00-INDEX 2024-03-27T06:45:14.342 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-cd.1994-2004 2024-03-27T06:45:14.344 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-floppy.1996-2002 2024-03-27T06:45:14.345 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-tape.1995-2002 2024-03-27T06:45:14.347 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ide/ide-tape.txt 2024-03-27T06:45:14.349 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ide/ide.txt 2024-03-27T06:45:14.350 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ide/warm-plug-howto.txt 2024-03-27T06:45:14.352 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/infiniband/ 2024-03-27T06:45:14.354 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/infiniband/core_locking.txt 2024-03-27T06:45:14.355 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/infiniband/ipoib.txt 2024-03-27T06:45:14.357 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/infiniband/sysfs.txt 2024-03-27T06:45:14.358 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/infiniband/user_mad.txt 2024-03-27T06:45:14.360 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/infiniband/user_verbs.txt 2024-03-27T06:45:14.361 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/init.txt 2024-03-27T06:45:14.363 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/initrd.txt 2024-03-27T06:45:14.365 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/ 2024-03-27T06:45:14.366 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:14 smithi170 podman[71001]: 2024-03-27 06:45:14.346185528 +0000 UTC m=+0.372797516 container init 0bba2c438fb3e88e7ece84b5cdf148ddd21dca12f63e129e82f06a7ac69a5582 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-27T06:45:14.366 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:14 smithi170 podman[71001]: 2024-03-27 06:45:14.349687223 +0000 UTC m=+0.376299195 container start 0bba2c438fb3e88e7ece84b5cdf148ddd21dca12f63e129e82f06a7ac69a5582 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-27T06:45:14.367 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/alps.txt 2024-03-27T06:45:14.368 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/amijoy.txt 2024-03-27T06:45:14.370 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/appletouch.txt 2024-03-27T06:45:14.371 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/atarikbd.txt 2024-03-27T06:45:14.373 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/bcm5974.txt 2024-03-27T06:45:14.374 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/cd32.txt 2024-03-27T06:45:14.376 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/cma3000_d0x.txt 2024-03-27T06:45:14.377 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/cs461x.txt 2024-03-27T06:45:14.379 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/edt-ft5x06.txt 2024-03-27T06:45:14.381 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/elantech.txt 2024-03-27T06:45:14.382 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/event-codes.txt 2024-03-27T06:45:14.384 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/ff.txt 2024-03-27T06:45:14.385 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/gamepad.txt 2024-03-27T06:45:14.387 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/gameport-programming.txt 2024-03-27T06:45:14.389 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/gpio-tilt.txt 2024-03-27T06:45:14.390 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/iforce-protocol.txt 2024-03-27T06:45:14.392 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/input-programming.txt 2024-03-27T06:45:14.393 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/input.txt 2024-03-27T06:45:14.395 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/interactive.fig 2024-03-27T06:45:14.397 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/joystick-api.txt 2024-03-27T06:45:14.398 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/joystick-parport.txt 2024-03-27T06:45:14.399 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/joystick.txt 2024-03-27T06:45:14.400 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/multi-touch-protocol.txt 2024-03-27T06:45:14.401 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/notifier.txt 2024-03-27T06:45:14.402 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/ntrig.txt 2024-03-27T06:45:14.403 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/rotary-encoder.txt 2024-03-27T06:45:14.404 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/sentelic.txt 2024-03-27T06:45:14.405 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/shape.fig 2024-03-27T06:45:14.406 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/walkera0701.txt 2024-03-27T06:45:14.407 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/xpad.txt 2024-03-27T06:45:14.408 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/input/yealink.txt 2024-03-27T06:45:14.410 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/intel_txt.txt 2024-03-27T06:45:14.411 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/io-mapping.txt 2024-03-27T06:45:14.412 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/io_ordering.txt 2024-03-27T06:45:14.414 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ioctl/ 2024-03-27T06:45:14.415 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ioctl/00-INDEX 2024-03-27T06:45:14.416 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ioctl/botching-up-ioctls.txt 2024-03-27T06:45:14.418 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ioctl/cdrom.txt 2024-03-27T06:45:14.420 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ioctl/hdio.txt 2024-03-27T06:45:14.421 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ioctl/ioctl-decoding.txt 2024-03-27T06:45:14.423 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ioctl/ioctl-number.txt 2024-03-27T06:45:14.425 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/iostats.txt 2024-03-27T06:45:14.427 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/irqflags-tracing.txt 2024-03-27T06:45:14.429 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isapnp.txt 2024-03-27T06:45:14.431 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/ 2024-03-27T06:45:14.433 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/00-INDEX 2024-03-27T06:45:14.434 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/CREDITS 2024-03-27T06:45:14.435 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/HiSax.cert 2024-03-27T06:45:14.437 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/INTERFACE 2024-03-27T06:45:14.438 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.CAPI 2024-03-27T06:45:14.439 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.fax 2024-03-27T06:45:14.440 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README 2024-03-27T06:45:14.442 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.FAQ 2024-03-27T06:45:14.443 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.HiSax 2024-03-27T06:45:14.445 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.act2000 2024-03-27T06:45:14.447 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.audio 2024-03-27T06:45:14.448 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.avmb1 2024-03-27T06:45:14.449 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.concap 2024-03-27T06:45:14.450 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.diversion 2024-03-27T06:45:14.454 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.fax 2024-03-27T06:45:14.498 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.gigaset 2024-03-27T06:45:14.513 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.hfc-pci 2024-03-27T06:45:14.518 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.hysdn 2024-03-27T06:45:14.519 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.icn 2024-03-27T06:45:14.521 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.mISDN 2024-03-27T06:45:14.522 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.pcbit 2024-03-27T06:45:14.524 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.sc 2024-03-27T06:45:14.525 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.syncppp 2024-03-27T06:45:14.527 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/README.x25 2024-03-27T06:45:14.529 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/isdn/syncPPP.FAQ 2024-03-27T06:45:14.530 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ja_JP/ 2024-03-27T06:45:14.532 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ja_JP/HOWTO 2024-03-27T06:45:14.533 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ja_JP/SubmitChecklist 2024-03-27T06:45:14.534 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ja_JP/SubmittingPatches 2024-03-27T06:45:14.535 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ja_JP/stable_api_nonsense.txt 2024-03-27T06:45:14.536 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ja_JP/stable_kernel_rules.txt 2024-03-27T06:45:14.538 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/java.txt 2024-03-27T06:45:14.539 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kasan.txt 2024-03-27T06:45:14.540 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kbuild/ 2024-03-27T06:45:14.542 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kbuild/00-INDEX 2024-03-27T06:45:14.543 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kbuild/headers_install.txt 2024-03-27T06:45:14.544 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kbuild/kbuild.txt 2024-03-27T06:45:14.545 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kbuild/kconfig-language.txt 2024-03-27T06:45:14.546 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kbuild/kconfig.txt 2024-03-27T06:45:14.548 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kbuild/makefiles.txt 2024-03-27T06:45:14.549 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kbuild/modules.txt 2024-03-27T06:45:14.550 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kdump/ 2024-03-27T06:45:14.551 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kdump/gdbmacros.txt 2024-03-27T06:45:14.552 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kdump/kdump.txt 2024-03-27T06:45:14.553 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kernel-doc-nano-HOWTO.txt 2024-03-27T06:45:14.554 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kernel-docs.txt 2024-03-27T06:45:14.555 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kernel-parameters.txt 2024-03-27T06:45:14.558 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kernel-per-CPU-kthreads.txt 2024-03-27T06:45:14.559 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kmemcheck.txt 2024-03-27T06:45:14.561 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kmemleak.txt 2024-03-27T06:45:14.563 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ko_KR/ 2024-03-27T06:45:14.565 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ko_KR/HOWTO 2024-03-27T06:45:14.568 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ko_KR/stable_api_nonsense.txt 2024-03-27T06:45:14.570 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kobject.txt 2024-03-27T06:45:14.572 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kprobes.txt 2024-03-27T06:45:14.576 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kref.txt 2024-03-27T06:45:14.580 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/kselftest.txt 2024-03-27T06:45:14.582 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/ 2024-03-27T06:45:14.584 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/.gitignore 2024-03-27T06:45:14.586 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/00-INDEX 2024-03-27T06:45:14.588 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/Makefile 2024-03-27T06:45:14.590 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/asus-laptop.txt 2024-03-27T06:45:14.592 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/disk-shock-protection.txt 2024-03-27T06:45:14.594 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/dslm.c 2024-03-27T06:45:14.596 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/freefall.c 2024-03-27T06:45:14.598 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/laptop-mode.txt 2024-03-27T06:45:14.600 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/sony-laptop.txt 2024-03-27T06:45:14.602 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/sonypi.txt 2024-03-27T06:45:14.604 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/thinkpad-acpi.txt 2024-03-27T06:45:14.606 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/laptops/toshiba_haps.txt 2024-03-27T06:45:14.608 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ldm.txt 2024-03-27T06:45:14.611 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/ 2024-03-27T06:45:14.613 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/00-INDEX 2024-03-27T06:45:14.615 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/leds-blinkm.txt 2024-03-27T06:45:14.617 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/leds-class.txt 2024-03-27T06:45:14.618 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/leds-lm3556.txt 2024-03-27T06:45:14.620 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/leds-lp3944.txt 2024-03-27T06:45:14.622 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/leds-lp5521.txt 2024-03-27T06:45:14.624 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/leds-lp5523.txt 2024-03-27T06:45:14.626 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/leds-lp5562.txt 2024-03-27T06:45:14.628 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/leds-lp55xx.txt 2024-03-27T06:45:14.631 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/ledtrig-oneshot.txt 2024-03-27T06:45:14.633 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/leds/ledtrig-transient.txt 2024-03-27T06:45:14.635 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/local_ops.txt 2024-03-27T06:45:14.637 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/locking/ 2024-03-27T06:45:14.639 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/locking/00-INDEX 2024-03-27T06:45:14.641 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/locking/lglock.txt 2024-03-27T06:45:14.643 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/locking/lockdep-design.txt 2024-03-27T06:45:14.644 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/locking/lockstat.txt 2024-03-27T06:45:14.646 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/locking/locktorture.txt 2024-03-27T06:45:14.647 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/locking/mutex-design.txt 2024-03-27T06:45:14.649 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/locking/rt-mutex-design.txt 2024-03-27T06:45:14.650 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/locking/rt-mutex.txt 2024-03-27T06:45:14.652 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/locking/spinlocks.txt 2024-03-27T06:45:14.654 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/locking/ww-mutex-design.txt 2024-03-27T06:45:14.656 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/lockup-watchdogs.txt 2024-03-27T06:45:14.658 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/logo.gif 2024-03-27T06:45:14.662 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/logo.txt 2024-03-27T06:45:14.665 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/lzo.txt 2024-03-27T06:45:14.667 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/m68k/ 2024-03-27T06:45:14.669 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/m68k/00-INDEX 2024-03-27T06:45:14.670 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/m68k/README.buddha 2024-03-27T06:45:14.672 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/m68k/kernel-options.txt 2024-03-27T06:45:14.674 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/magic-number.txt 2024-03-27T06:45:14.675 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mailbox.txt 2024-03-27T06:45:14.677 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/md.txt 2024-03-27T06:45:14.679 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/media-framework.txt 2024-03-27T06:45:14.680 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/memory-barriers.txt 2024-03-27T06:45:14.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:14 smithi156 ceph-mon[36261]: daemon mds.f restarted 2024-03-27T06:45:14.739 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/memory-devices/ 2024-03-27T06:45:14.739 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/memory-devices/ti-emif.txt 2024-03-27T06:45:14.739 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/memory-hotplug.txt 2024-03-27T06:45:14.739 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/metag/ 2024-03-27T06:45:14.740 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/metag/00-INDEX 2024-03-27T06:45:14.741 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/metag/kernel-ABI.txt 2024-03-27T06:45:14.742 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mic/ 2024-03-27T06:45:14.743 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mic/Makefile 2024-03-27T06:45:14.744 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mic/mic_overview.txt 2024-03-27T06:45:14.745 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mic/mpssd/ 2024-03-27T06:45:14.746 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mic/mpssd/.gitignore 2024-03-27T06:45:14.747 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mic/mpssd/Makefile 2024-03-27T06:45:14.748 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mic/mpssd/micctrl 2024-03-27T06:45:14.750 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mic/mpssd/mpss 2024-03-27T06:45:14.752 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.c 2024-03-27T06:45:14.753 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.h 2024-03-27T06:45:14.755 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mic/mpssd/sysfs.c 2024-03-27T06:45:14.756 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mips/ 2024-03-27T06:45:14.758 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mips/00-INDEX 2024-03-27T06:45:14.759 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mips/AU1xxx_IDE.README 2024-03-27T06:45:14.761 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/ 2024-03-27T06:45:14.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:14 smithi016 ceph-mon[31908]: daemon mds.f restarted 2024-03-27T06:45:14.763 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/Makefile 2024-03-27T06:45:14.764 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/ad525x_dpot.txt 2024-03-27T06:45:14.765 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/apds990x.txt 2024-03-27T06:45:14.767 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/bh1770glc.txt 2024-03-27T06:45:14.769 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/c2port.txt 2024-03-27T06:45:14.770 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/eeprom 2024-03-27T06:45:14.771 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/ics932s401 2024-03-27T06:45:14.772 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/isl29003 2024-03-27T06:45:14.774 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/lis3lv02d 2024-03-27T06:45:14.775 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/max6875 2024-03-27T06:45:14.776 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/mei/ 2024-03-27T06:45:14.777 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/mei/.gitignore 2024-03-27T06:45:14.778 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/mei/Makefile 2024-03-27T06:45:14.779 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/mei/TODO 2024-03-27T06:45:14.780 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-amt-version.c 2024-03-27T06:45:14.781 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-client-bus.txt 2024-03-27T06:45:14.783 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei.txt 2024-03-27T06:45:14.784 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/misc-devices/spear-pcie-gadget.txt 2024-03-27T06:45:14.785 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mmc/ 2024-03-27T06:45:14.787 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mmc/00-INDEX 2024-03-27T06:45:14.788 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mmc/mmc-async-req.txt 2024-03-27T06:45:14.790 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-attrs.txt 2024-03-27T06:45:14.791 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-parts.txt 2024-03-27T06:45:14.792 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mn10300/ 2024-03-27T06:45:14.794 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mn10300/ABI.txt 2024-03-27T06:45:14.795 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mn10300/compartmentalisation.txt 2024-03-27T06:45:14.796 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/module-signing.txt 2024-03-27T06:45:14.798 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mono.txt 2024-03-27T06:45:14.799 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mtd/ 2024-03-27T06:45:14.801 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mtd/nand/ 2024-03-27T06:45:14.802 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt 2024-03-27T06:45:14.803 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mtd/nand_ecc.txt 2024-03-27T06:45:14.805 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/mtd/spi-nor.txt 2024-03-27T06:45:14.806 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/namespaces/ 2024-03-27T06:45:14.807 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/namespaces/compatibility-list.txt 2024-03-27T06:45:14.809 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/namespaces/resource-control.txt 2024-03-27T06:45:14.809 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/netlabel/ 2024-03-27T06:45:14.811 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/netlabel/00-INDEX 2024-03-27T06:45:14.811 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/netlabel/cipso_ipv4.txt 2024-03-27T06:45:14.813 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/netlabel/draft-ietf-cipso-ipsecurity-01.txt 2024-03-27T06:45:14.814 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/netlabel/introduction.txt 2024-03-27T06:45:14.816 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/netlabel/lsm_interface.txt 2024-03-27T06:45:14.817 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ 2024-03-27T06:45:14.819 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/00-INDEX 2024-03-27T06:45:14.820 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/3c509.txt 2024-03-27T06:45:14.821 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/6pack.txt 2024-03-27T06:45:14.823 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/LICENSE.qla3xxx 2024-03-27T06:45:14.824 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlcnic 2024-03-27T06:45:14.826 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlge 2024-03-27T06:45:14.827 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/Makefile 2024-03-27T06:45:14.828 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/PLIP.txt 2024-03-27T06:45:14.830 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/README.ipw2100 2024-03-27T06:45:14.831 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/README.ipw2200 2024-03-27T06:45:14.833 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/README.sb1000 2024-03-27T06:45:14.834 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/alias.txt 2024-03-27T06:45:14.836 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/altera_tse.txt 2024-03-27T06:45:14.837 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/arcnet-hardware.txt 2024-03-27T06:45:14.839 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/arcnet.txt 2024-03-27T06:45:14.840 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/atm.txt 2024-03-27T06:45:14.841 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ax25.txt 2024-03-27T06:45:14.842 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/batman-adv.txt 2024-03-27T06:45:14.843 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/baycom.txt 2024-03-27T06:45:14.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:14 smithi170 ceph-mon[36438]: daemon mds.f restarted 2024-03-27T06:45:14.844 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:14 smithi170 ceph-mds[71020]: -- 172.21.15.170:0/3158945129 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b56392da40 con 0x55b563901000 2024-03-27T06:45:14.844 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:14 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[71016]: starting mds.f at 2024-03-27T06:45:14.844 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:14 smithi170 bash[71001]: 0bba2c438fb3e88e7ece84b5cdf148ddd21dca12f63e129e82f06a7ac69a5582 2024-03-27T06:45:14.844 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:45:14 smithi170 systemd[1]: Started Ceph mds.f for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:45:14.844 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/bonding.txt 2024-03-27T06:45:14.845 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/bridge.txt 2024-03-27T06:45:14.846 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/caif/ 2024-03-27T06:45:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/caif/Linux-CAIF.txt 2024-03-27T06:45:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/caif/README 2024-03-27T06:45:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/caif/spi_porting.txt 2024-03-27T06:45:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/can.txt 2024-03-27T06:45:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/cdc_mbim.txt 2024-03-27T06:45:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/cops.txt 2024-03-27T06:45:14.853 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/cs89x0.txt 2024-03-27T06:45:14.855 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/cxacru-cf.py 2024-03-27T06:45:14.856 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/cxacru.txt 2024-03-27T06:45:14.858 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/cxgb.txt 2024-03-27T06:45:14.859 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/dccp.txt 2024-03-27T06:45:14.860 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/dctcp.txt 2024-03-27T06:45:14.861 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/de4x5.txt 2024-03-27T06:45:14.862 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/decnet.txt 2024-03-27T06:45:14.863 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/dl2k.txt 2024-03-27T06:45:14.864 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/dm9000.txt 2024-03-27T06:45:14.865 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/dmfe.txt 2024-03-27T06:45:14.866 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/dns_resolver.txt 2024-03-27T06:45:14.867 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/driver.txt 2024-03-27T06:45:14.868 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/e100.txt 2024-03-27T06:45:14.868 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/e1000.txt 2024-03-27T06:45:14.869 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/e1000e.txt 2024-03-27T06:45:14.870 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/eql.txt 2024-03-27T06:45:14.871 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/fib_trie.txt 2024-03-27T06:45:14.872 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/filter.txt 2024-03-27T06:45:14.873 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/fore200e.txt 2024-03-27T06:45:14.874 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/framerelay.txt 2024-03-27T06:45:14.875 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/gen_stats.txt 2024-03-27T06:45:14.876 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/generic-hdlc.txt 2024-03-27T06:45:14.876 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/generic_netlink.txt 2024-03-27T06:45:14.878 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/gianfar.txt 2024-03-27T06:45:14.878 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/i40e.txt 2024-03-27T06:45:14.879 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/i40evf.txt 2024-03-27T06:45:14.880 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ieee802154.txt 2024-03-27T06:45:14.881 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/igb.txt 2024-03-27T06:45:14.882 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/igbvf.txt 2024-03-27T06:45:14.883 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ip-sysctl.txt 2024-03-27T06:45:14.884 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ip_dynaddr.txt 2024-03-27T06:45:14.885 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ipddp.txt 2024-03-27T06:45:14.886 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/iphase.txt 2024-03-27T06:45:14.887 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ipsec.txt 2024-03-27T06:45:14.888 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ipv6.txt 2024-03-27T06:45:14.889 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ipvlan.txt 2024-03-27T06:45:14.890 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ipvs-sysctl.txt 2024-03-27T06:45:14.890 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/irda.txt 2024-03-27T06:45:14.891 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ixgb.txt 2024-03-27T06:45:14.892 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ixgbe.txt 2024-03-27T06:45:14.893 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ixgbevf.txt 2024-03-27T06:45:14.894 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/l2tp.txt 2024-03-27T06:45:14.895 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/lapb-module.txt 2024-03-27T06:45:14.896 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ltpc.txt 2024-03-27T06:45:14.897 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/mac80211-auth-assoc-deauth.txt 2024-03-27T06:45:14.898 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/mac80211-injection.txt 2024-03-27T06:45:14.899 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/ 2024-03-27T06:45:14.900 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/README 2024-03-27T06:45:14.901 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/hostapd.conf 2024-03-27T06:45:14.902 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/wpa_supplicant.conf 2024-03-27T06:45:14.903 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/multiqueue.txt 2024-03-27T06:45:14.903 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/netconsole.txt 2024-03-27T06:45:14.904 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/netdev-FAQ.txt 2024-03-27T06:45:14.905 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/netdev-features.txt 2024-03-27T06:45:14.906 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/netdevices.txt 2024-03-27T06:45:14.907 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/netif-msg.txt 2024-03-27T06:45:14.908 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/netlink_mmap.txt 2024-03-27T06:45:14.909 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/nf_conntrack-sysctl.txt 2024-03-27T06:45:14.910 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/nfc.txt 2024-03-27T06:45:14.911 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/openvswitch.txt 2024-03-27T06:45:14.911 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/operstates.txt 2024-03-27T06:45:14.912 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/packet_mmap.txt 2024-03-27T06:45:14.913 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/phonet.txt 2024-03-27T06:45:14.914 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/phy.txt 2024-03-27T06:45:14.915 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/pktgen.txt 2024-03-27T06:45:14.917 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/policy-routing.txt 2024-03-27T06:45:14.919 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ppp_generic.txt 2024-03-27T06:45:14.920 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/proc_net_tcp.txt 2024-03-27T06:45:14.922 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/radiotap-headers.txt 2024-03-27T06:45:14.923 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/ray_cs.txt 2024-03-27T06:45:14.925 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/rds.txt 2024-03-27T06:45:14.926 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/regulatory.txt 2024-03-27T06:45:14.928 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/rxrpc.txt 2024-03-27T06:45:14.929 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/s2io.txt 2024-03-27T06:45:14.931 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/scaling.txt 2024-03-27T06:45:14.931 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/sctp.txt 2024-03-27T06:45:14.932 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/secid.txt 2024-03-27T06:45:14.933 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/skfp.txt 2024-03-27T06:45:14.934 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/smc9.txt 2024-03-27T06:45:14.935 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/spider_net.txt 2024-03-27T06:45:14.936 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/stmmac.txt 2024-03-27T06:45:14.937 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/switchdev.txt 2024-03-27T06:45:14.938 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/tc-actions-env-rules.txt 2024-03-27T06:45:14.939 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/tcp-thin.txt 2024-03-27T06:45:14.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/tcp.txt 2024-03-27T06:45:14.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/team.txt 2024-03-27T06:45:14.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/timestamping.txt 2024-03-27T06:45:14.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/timestamping/ 2024-03-27T06:45:14.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/timestamping/.gitignore 2024-03-27T06:45:14.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/timestamping/Makefile 2024-03-27T06:45:14.945 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c 2024-03-27T06:45:14.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/timestamping/timestamping.c 2024-03-27T06:45:14.947 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c 2024-03-27T06:45:14.948 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/tlan.txt 2024-03-27T06:45:14.949 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/tproxy.txt 2024-03-27T06:45:14.950 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/tuntap.txt 2024-03-27T06:45:14.951 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/udplite.txt 2024-03-27T06:45:14.952 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/vortex.txt 2024-03-27T06:45:14.953 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/vxge.txt 2024-03-27T06:45:14.953 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/vxlan.txt 2024-03-27T06:45:14.954 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/x25-iface.txt 2024-03-27T06:45:14.955 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/x25.txt 2024-03-27T06:45:14.956 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/xfrm_proc.txt 2024-03-27T06:45:14.957 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/xfrm_sync.txt 2024-03-27T06:45:14.958 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/xfrm_sysctl.txt 2024-03-27T06:45:14.959 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/networking/z8530drv.txt 2024-03-27T06:45:14.960 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/nfc/ 2024-03-27T06:45:14.961 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/nfc/nfc-hci.txt 2024-03-27T06:45:14.962 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/nfc/nfc-pn544.txt 2024-03-27T06:45:14.962 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/nios2/ 2024-03-27T06:45:14.963 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/nios2/README 2024-03-27T06:45:14.964 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/nommu-mmap.txt 2024-03-27T06:45:14.965 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/numastat.txt 2024-03-27T06:45:14.966 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/oops-tracing.txt 2024-03-27T06:45:14.967 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/padata.txt 2024-03-27T06:45:14.968 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/parisc/ 2024-03-27T06:45:14.969 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/parisc/00-INDEX 2024-03-27T06:45:14.970 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/parisc/debugging 2024-03-27T06:45:14.971 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/parisc/registers 2024-03-27T06:45:14.972 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/parport-lowlevel.txt 2024-03-27T06:45:14.973 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/parport.txt 2024-03-27T06:45:14.974 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pcmcia/ 2024-03-27T06:45:14.975 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pcmcia/.gitignore 2024-03-27T06:45:14.976 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pcmcia/Makefile 2024-03-27T06:45:14.977 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pcmcia/crc32hash.c 2024-03-27T06:45:14.978 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pcmcia/devicetable.txt 2024-03-27T06:45:14.979 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pcmcia/driver-changes.txt 2024-03-27T06:45:14.980 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pcmcia/driver.txt 2024-03-27T06:45:14.980 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pcmcia/locking.txt 2024-03-27T06:45:14.981 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/percpu-rw-semaphore.txt 2024-03-27T06:45:14.982 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/phy.txt 2024-03-27T06:45:14.983 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/phy/ 2024-03-27T06:45:14.984 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/phy/samsung-usb2.txt 2024-03-27T06:45:14.985 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pi-futex.txt 2024-03-27T06:45:14.986 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pinctrl.txt 2024-03-27T06:45:14.987 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/platform/ 2024-03-27T06:45:14.988 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/platform/x86-laptop-drivers.txt 2024-03-27T06:45:14.989 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pnp.txt 2024-03-27T06:45:14.990 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/ 2024-03-27T06:45:14.991 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/00-INDEX 2024-03-27T06:45:14.992 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/apm-acpi.txt 2024-03-27T06:45:14.992 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/basic-pm-debugging.txt 2024-03-27T06:45:14.993 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/charger-manager.txt 2024-03-27T06:45:14.994 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/devices.txt 2024-03-27T06:45:14.995 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/drivers-testing.txt 2024-03-27T06:45:14.996 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/freezing-of-tasks.txt 2024-03-27T06:45:14.997 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/interface.txt 2024-03-27T06:45:14.998 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/notifiers.txt 2024-03-27T06:45:14.999 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/opp.txt 2024-03-27T06:45:14.999 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/pci.txt 2024-03-27T06:45:15.000 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/pm_qos_interface.txt 2024-03-27T06:45:15.001 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/power_supply_class.txt 2024-03-27T06:45:15.002 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/powercap/ 2024-03-27T06:45:15.003 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/powercap/powercap.txt 2024-03-27T06:45:15.004 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/regulator/ 2024-03-27T06:45:15.005 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/regulator/consumer.txt 2024-03-27T06:45:15.005 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/regulator/design.txt 2024-03-27T06:45:15.006 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/regulator/machine.txt 2024-03-27T06:45:15.007 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/regulator/overview.txt 2024-03-27T06:45:15.008 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/regulator/regulator.txt 2024-03-27T06:45:15.009 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/runtime_pm.txt 2024-03-27T06:45:15.010 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/s2ram.txt 2024-03-27T06:45:15.010 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/states.txt 2024-03-27T06:45:15.011 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/suspend-and-cpuhotplug.txt 2024-03-27T06:45:15.012 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/suspend-and-interrupts.txt 2024-03-27T06:45:15.013 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/swsusp-and-swap-files.txt 2024-03-27T06:45:15.014 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/swsusp-dmcrypt.txt 2024-03-27T06:45:15.015 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/swsusp.txt 2024-03-27T06:45:15.016 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/tricks.txt 2024-03-27T06:45:15.017 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/userland-swsusp.txt 2024-03-27T06:45:15.018 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/power/video.txt 2024-03-27T06:45:15.019 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/ 2024-03-27T06:45:15.020 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/00-INDEX 2024-03-27T06:45:15.021 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/bootwrapper.txt 2024-03-27T06:45:15.022 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/cpu_families.txt 2024-03-27T06:45:15.022 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/cpu_features.txt 2024-03-27T06:45:15.023 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/cxl.txt 2024-03-27T06:45:15.024 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/eeh-pci-error-recovery.txt 2024-03-27T06:45:15.025 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/firmware-assisted-dump.txt 2024-03-27T06:45:15.026 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/hvcs.txt 2024-03-27T06:45:15.028 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/mpc52xx.txt 2024-03-27T06:45:15.028 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/pmu-ebb.txt 2024-03-27T06:45:15.029 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/ptrace.txt 2024-03-27T06:45:15.030 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/qe_firmware.txt 2024-03-27T06:45:15.031 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/powerpc/transactional_memory.txt 2024-03-27T06:45:15.032 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pps/ 2024-03-27T06:45:15.033 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pps/pps.txt 2024-03-27T06:45:15.034 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/prctl/ 2024-03-27T06:45:15.035 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/prctl/.gitignore 2024-03-27T06:45:15.035 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/prctl/Makefile 2024-03-27T06:45:15.036 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-ctxt-sw-stress-test.c 2024-03-27T06:45:15.037 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-on-off-stress-test.c 2024-03-27T06:45:15.038 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-test.c 2024-03-27T06:45:15.039 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/prctl/no_new_privs.txt 2024-03-27T06:45:15.040 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/prctl/seccomp_filter.txt 2024-03-27T06:45:15.042 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/preempt-locking.txt 2024-03-27T06:45:15.044 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/printk-formats.txt 2024-03-27T06:45:15.045 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pti/ 2024-03-27T06:45:15.047 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pti/pti_intel_mid.txt 2024-03-27T06:45:15.048 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ptp/ 2024-03-27T06:45:15.050 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ptp/.gitignore 2024-03-27T06:45:15.051 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ptp/Makefile 2024-03-27T06:45:15.052 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ptp/ptp.txt 2024-03-27T06:45:15.054 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ptp/testptp.c 2024-03-27T06:45:15.056 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ptp/testptp.mk 2024-03-27T06:45:15.057 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/pwm.txt 2024-03-27T06:45:15.058 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/ramoops.txt 2024-03-27T06:45:15.059 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/rapidio/ 2024-03-27T06:45:15.060 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/rapidio/rapidio.txt 2024-03-27T06:45:15.061 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/rapidio/sysfs.txt 2024-03-27T06:45:15.061 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/rapidio/tsi721.txt 2024-03-27T06:45:15.062 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/rbtree.txt 2024-03-27T06:45:15.063 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/remoteproc.txt 2024-03-27T06:45:15.064 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/rfkill.txt 2024-03-27T06:45:15.066 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/robust-futex-ABI.txt 2024-03-27T06:45:15.067 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/robust-futexes.txt 2024-03-27T06:45:15.069 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/rpmsg.txt 2024-03-27T06:45:15.070 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/rtc.txt 2024-03-27T06:45:15.071 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/ 2024-03-27T06:45:15.072 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/00-INDEX 2024-03-27T06:45:15.073 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/3270.ChangeLog 2024-03-27T06:45:15.074 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/3270.txt 2024-03-27T06:45:15.074 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/CommonIO 2024-03-27T06:45:15.075 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/DASD 2024-03-27T06:45:15.076 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/Debugging390.txt 2024-03-27T06:45:15.077 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/cds.txt 2024-03-27T06:45:15.078 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/config3270.sh 2024-03-27T06:45:15.079 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/driver-model.txt 2024-03-27T06:45:15.080 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/kvm.txt 2024-03-27T06:45:15.082 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/monreader.txt 2024-03-27T06:45:15.083 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/qeth.txt 2024-03-27T06:45:15.085 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/s390dbf.txt 2024-03-27T06:45:15.086 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/s390/zfcpdump.txt 2024-03-27T06:45:15.088 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scheduler/ 2024-03-27T06:45:15.089 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scheduler/00-INDEX 2024-03-27T06:45:15.091 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scheduler/completion.txt 2024-03-27T06:45:15.092 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scheduler/sched-arch.txt 2024-03-27T06:45:15.094 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scheduler/sched-bwc.txt 2024-03-27T06:45:15.095 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scheduler/sched-deadline.txt 2024-03-27T06:45:15.097 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scheduler/sched-design-CFS.txt 2024-03-27T06:45:15.098 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scheduler/sched-domains.txt 2024-03-27T06:45:15.100 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scheduler/sched-nice-design.txt 2024-03-27T06:45:15.101 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scheduler/sched-rt-group.txt 2024-03-27T06:45:15.103 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scheduler/sched-stats.txt 2024-03-27T06:45:15.104 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ 2024-03-27T06:45:15.106 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/00-INDEX 2024-03-27T06:45:15.107 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/53c700.txt 2024-03-27T06:45:15.109 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/BusLogic.txt 2024-03-27T06:45:15.111 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.1992-1997 2024-03-27T06:45:15.112 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.arcmsr 2024-03-27T06:45:15.114 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ips 2024-03-27T06:45:15.115 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.lpfc 2024-03-27T06:45:15.116 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid 2024-03-27T06:45:15.117 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid_sas 2024-03-27T06:45:15.118 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ncr53c8xx 2024-03-27T06:45:15.119 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx 2024-03-27T06:45:15.120 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx_2 2024-03-27T06:45:15.121 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/FlashPoint.txt 2024-03-27T06:45:15.122 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/LICENSE.FlashPoint 2024-03-27T06:45:15.124 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla2xxx 2024-03-27T06:45:15.125 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla4xxx 2024-03-27T06:45:15.126 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/Mylex.txt 2024-03-27T06:45:15.128 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/NinjaSCSI.txt 2024-03-27T06:45:15.129 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/aacraid.txt 2024-03-27T06:45:15.131 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/advansys.txt 2024-03-27T06:45:15.132 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/aha152x.txt 2024-03-27T06:45:15.133 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/aic79xx.txt 2024-03-27T06:45:15.135 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/aic7xxx.txt 2024-03-27T06:45:15.136 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/arcmsr_spec.txt 2024-03-27T06:45:15.138 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/bfa.txt 2024-03-27T06:45:15.139 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/bnx2fc.txt 2024-03-27T06:45:15.141 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/cxgb3i.txt 2024-03-27T06:45:15.142 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/dc395x.txt 2024-03-27T06:45:15.147 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/dpti.txt 2024-03-27T06:45:15.147 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/dtc3x80.txt 2024-03-27T06:45:15.147 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/g_NCR5380.txt 2024-03-27T06:45:15.148 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/hpsa.txt 2024-03-27T06:45:15.149 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/hptiop.txt 2024-03-27T06:45:15.151 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/in2000.txt 2024-03-27T06:45:15.152 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/libsas.txt 2024-03-27T06:45:15.154 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/link_power_management_policy.txt 2024-03-27T06:45:15.155 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/lpfc.txt 2024-03-27T06:45:15.157 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/megaraid.txt 2024-03-27T06:45:15.158 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ncr53c8xx.txt 2024-03-27T06:45:15.160 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/osd.txt 2024-03-27T06:45:15.161 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/osst.txt 2024-03-27T06:45:15.163 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ppa.txt 2024-03-27T06:45:15.166 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/qlogicfas.txt 2024-03-27T06:45:15.167 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/scsi-changer.txt 2024-03-27T06:45:15.169 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/scsi-generic.txt 2024-03-27T06:45:15.171 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/scsi-parameters.txt 2024-03-27T06:45:15.173 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/scsi.txt 2024-03-27T06:45:15.175 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/scsi_eh.txt 2024-03-27T06:45:15.177 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/scsi_fc_transport.txt 2024-03-27T06:45:15.180 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/scsi_mid_low_api.txt 2024-03-27T06:45:15.182 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/ 2024-03-27T06:45:15.184 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/Makefile 2024-03-27T06:45:15.186 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/rport_state_diagram.dot 2024-03-27T06:45:15.188 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/st.txt 2024-03-27T06:45:15.190 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/sym53c500_cs.txt 2024-03-27T06:45:15.192 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/sym53c8xx_2.txt 2024-03-27T06:45:15.194 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/tmscsim.txt 2024-03-27T06:45:15.196 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/ufs.txt 2024-03-27T06:45:15.198 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/scsi/wd719x.txt 2024-03-27T06:45:15.200 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/ 2024-03-27T06:45:15.203 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/00-INDEX 2024-03-27T06:45:15.205 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/IMA-templates.txt 2024-03-27T06:45:15.207 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/LSM.txt 2024-03-27T06:45:15.209 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/SELinux.txt 2024-03-27T06:45:15.211 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/Smack.txt 2024-03-27T06:45:15.213 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/Yama.txt 2024-03-27T06:45:15.215 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/apparmor.txt 2024-03-27T06:45:15.217 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/credentials.txt 2024-03-27T06:45:15.218 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/keys-ecryptfs.txt 2024-03-27T06:45:15.220 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/keys-request-key.txt 2024-03-27T06:45:15.222 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/keys-trusted-encrypted.txt 2024-03-27T06:45:15.224 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/keys.txt 2024-03-27T06:45:15.226 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/security/tomoyo.txt 2024-03-27T06:45:15.226 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/serial-console.txt 2024-03-27T06:45:15.227 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/serial/ 2024-03-27T06:45:15.229 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/serial/00-INDEX 2024-03-27T06:45:15.230 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/serial/README.cycladesZ 2024-03-27T06:45:15.232 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/serial/driver 2024-03-27T06:45:15.233 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/serial/moxa-smartio 2024-03-27T06:45:15.234 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/serial/n_gsm.txt 2024-03-27T06:45:15.236 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/serial/rocket.txt 2024-03-27T06:45:15.237 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/serial/serial-rs485.txt 2024-03-27T06:45:15.238 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/serial/tty.txt 2024-03-27T06:45:15.240 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sgi-ioc4.txt 2024-03-27T06:45:15.241 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sh/ 2024-03-27T06:45:15.242 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sh/new-machine.txt 2024-03-27T06:45:15.243 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sh/register-banks.txt 2024-03-27T06:45:15.245 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/smsc_ece1099.txt 2024-03-27T06:45:15.246 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/ 2024-03-27T06:45:15.247 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/ 2024-03-27T06:45:15.248 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/ALSA-Configuration.txt 2024-03-27T06:45:15.250 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/Audigy-mixer.txt 2024-03-27T06:45:15.251 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/Audiophile-Usb.txt 2024-03-27T06:45:15.252 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/Bt87x.txt 2024-03-27T06:45:15.254 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/CMIPCI.txt 2024-03-27T06:45:15.255 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/Channel-Mapping-API.txt 2024-03-27T06:45:15.257 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/ControlNames.txt 2024-03-27T06:45:15.259 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Controls.txt 2024-03-27T06:45:15.261 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Models.txt 2024-03-27T06:45:15.263 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio.txt 2024-03-27T06:45:15.265 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/Joystick.txt 2024-03-27T06:45:15.266 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/MIXART.txt 2024-03-27T06:45:15.268 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/OSS-Emulation.txt 2024-03-27T06:45:15.270 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/Procfile.txt 2024-03-27T06:45:15.272 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/README.maya44 2024-03-27T06:45:15.274 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/SB-Live-mixer.txt 2024-03-27T06:45:15.276 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/VIA82xx-mixer.txt 2024-03-27T06:45:15.278 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/alsa-parameters.txt 2024-03-27T06:45:15.280 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/compress_offload.txt 2024-03-27T06:45:15.282 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/emu10k1-jack.txt 2024-03-27T06:45:15.284 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/hda_codec.txt 2024-03-27T06:45:15.286 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/hdspm.txt 2024-03-27T06:45:15.288 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/powersave.txt 2024-03-27T06:45:15.290 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/seq_oss.html 2024-03-27T06:45:15.291 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/serial-u16550.txt 2024-03-27T06:45:15.293 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/soc/ 2024-03-27T06:45:15.294 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt 2024-03-27T06:45:15.295 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt 2024-03-27T06:45:15.297 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt 2024-03-27T06:45:15.298 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/soc/codec.txt 2024-03-27T06:45:15.300 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt 2024-03-27T06:45:15.301 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/soc/jack.txt 2024-03-27T06:45:15.303 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/soc/machine.txt 2024-03-27T06:45:15.304 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/soc/overview.txt 2024-03-27T06:45:15.306 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/soc/platform.txt 2024-03-27T06:45:15.307 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt 2024-03-27T06:45:15.309 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/ 2024-03-27T06:45:15.310 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/ALS 2024-03-27T06:45:15.311 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16 2024-03-27T06:45:15.313 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/CMI8330 2024-03-27T06:45:15.314 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/ESS 2024-03-27T06:45:15.316 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/ESS1868 2024-03-27T06:45:15.317 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/Introduction 2024-03-27T06:45:15.319 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/MultiSound 2024-03-27T06:45:15.321 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/OPL3 2024-03-27T06:45:15.322 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/Opti 2024-03-27T06:45:15.324 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/PAS16 2024-03-27T06:45:15.326 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/PSS 2024-03-27T06:45:15.327 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/PSS-updates 2024-03-27T06:45:15.329 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/README.OSS 2024-03-27T06:45:15.330 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/README.modules 2024-03-27T06:45:15.331 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/README.ymfsb 2024-03-27T06:45:15.332 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/SoundPro 2024-03-27T06:45:15.334 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/Soundblaster 2024-03-27T06:45:15.336 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/Tropez+ 2024-03-27T06:45:15.338 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/VIBRA16 2024-03-27T06:45:15.340 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/WaveArtist 2024-03-27T06:45:15.342 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/btaudio 2024-03-27T06:45:15.344 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/mwave 2024-03-27T06:45:15.346 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/oss-parameters.txt 2024-03-27T06:45:15.350 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sound/oss/ultrasound 2024-03-27T06:45:15.351 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sparse.txt 2024-03-27T06:45:15.352 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/ 2024-03-27T06:45:15.354 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/.gitignore 2024-03-27T06:45:15.356 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/00-INDEX 2024-03-27T06:45:15.358 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/Makefile 2024-03-27T06:45:15.360 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/butterfly 2024-03-27T06:45:15.362 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/ep93xx_spi 2024-03-27T06:45:15.364 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/pxa2xx 2024-03-27T06:45:15.366 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/spi-lm70llp 2024-03-27T06:45:15.368 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/spi-sc18is602 2024-03-27T06:45:15.370 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/spi-summary 2024-03-27T06:45:15.372 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/spidev 2024-03-27T06:45:15.374 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/spidev_fdx.c 2024-03-27T06:45:15.376 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/spi/spidev_test.c 2024-03-27T06:45:15.378 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/stable_api_nonsense.txt 2024-03-27T06:45:15.380 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/stable_kernel_rules.txt 2024-03-27T06:45:15.382 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/static-keys.txt 2024-03-27T06:45:15.385 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/svga.txt 2024-03-27T06:45:15.387 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sysctl/ 2024-03-27T06:45:15.389 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sysctl/00-INDEX 2024-03-27T06:45:15.391 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sysctl/README 2024-03-27T06:45:15.393 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sysctl/abi.txt 2024-03-27T06:45:15.395 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sysctl/fs.txt 2024-03-27T06:45:15.397 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sysctl/kernel.txt 2024-03-27T06:45:15.399 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sysctl/net.txt 2024-03-27T06:45:15.401 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sysctl/sunrpc.txt 2024-03-27T06:45:15.403 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sysctl/vm.txt 2024-03-27T06:45:15.405 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sysfs-rules.txt 2024-03-27T06:45:15.407 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/sysrq.txt 2024-03-27T06:45:15.409 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/target/ 2024-03-27T06:45:15.410 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.py 2024-03-27T06:45:15.413 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.txt 2024-03-27T06:45:15.415 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/target/tcmu-design.txt 2024-03-27T06:45:15.417 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/thermal/ 2024-03-27T06:45:15.419 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/thermal/cpu-cooling-api.txt 2024-03-27T06:45:15.420 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal 2024-03-27T06:45:15.421 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal_emulation 2024-03-27T06:45:15.423 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/thermal/intel_powerclamp.txt 2024-03-27T06:45:15.425 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/thermal/nouveau_thermal 2024-03-27T06:45:15.427 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/thermal/sysfs-api.txt 2024-03-27T06:45:15.429 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/thermal/x86_pkg_temperature_thermal 2024-03-27T06:45:15.431 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/this_cpu_ops.txt 2024-03-27T06:45:15.433 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/ 2024-03-27T06:45:15.435 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/.gitignore 2024-03-27T06:45:15.437 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/00-INDEX 2024-03-27T06:45:15.439 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/Makefile 2024-03-27T06:45:15.440 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/NO_HZ.txt 2024-03-27T06:45:15.442 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/highres.txt 2024-03-27T06:45:15.446 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/hpet.txt 2024-03-27T06:45:15.446 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/hpet_example.c 2024-03-27T06:45:15.448 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/hrtimers.txt 2024-03-27T06:45:15.450 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/timekeeping.txt 2024-03-27T06:45:15.452 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/timer_stats.txt 2024-03-27T06:45:15.454 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/timers/timers-howto.txt 2024-03-27T06:45:15.457 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/tpm/ 2024-03-27T06:45:15.460 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/tpm/xen-tpmfront.txt 2024-03-27T06:45:15.462 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/ 2024-03-27T06:45:15.463 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/coresight.txt 2024-03-27T06:45:15.465 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/events-kmem.txt 2024-03-27T06:45:15.466 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/events-nmi.txt 2024-03-27T06:45:15.468 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/events-power.txt 2024-03-27T06:45:15.469 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/events.txt 2024-03-27T06:45:15.471 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/ftrace-design.txt 2024-03-27T06:45:15.472 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/ftrace.txt 2024-03-27T06:45:15.474 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/function-graph-fold.vim 2024-03-27T06:45:15.475 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/kprobetrace.txt 2024-03-27T06:45:15.477 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/mmiotrace.txt 2024-03-27T06:45:15.478 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/postprocess/ 2024-03-27T06:45:15.479 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-pagealloc-postprocess.pl 2024-03-27T06:45:15.481 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-vmscan-postprocess.pl 2024-03-27T06:45:15.483 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/ring-buffer-design.txt 2024-03-27T06:45:15.485 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/tracepoint-analysis.txt 2024-03-27T06:45:15.486 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/tracepoints.txt 2024-03-27T06:45:15.488 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/trace/uprobetracer.txt 2024-03-27T06:45:15.490 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/unaligned-memory-access.txt 2024-03-27T06:45:15.491 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/unicode.txt 2024-03-27T06:45:15.493 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/unshare.txt 2024-03-27T06:45:15.494 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/ 2024-03-27T06:45:15.496 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/CREDITS 2024-03-27T06:45:15.499 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/URB.txt 2024-03-27T06:45:15.501 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/WUSB-Design-overview.txt 2024-03-27T06:45:15.503 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/acm.txt 2024-03-27T06:45:15.505 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/anchors.txt 2024-03-27T06:45:15.507 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/authorization.txt 2024-03-27T06:45:15.509 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/bulk-streams.txt 2024-03-27T06:45:15.510 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/callbacks.txt 2024-03-27T06:45:15.511 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/chipidea.txt 2024-03-27T06:45:15.512 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/dma.txt 2024-03-27T06:45:15.513 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/dwc3.txt 2024-03-27T06:45:15.515 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/ehci.txt 2024-03-27T06:45:15.516 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/error-codes.txt 2024-03-27T06:45:15.517 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/functionfs.txt 2024-03-27T06:45:15.518 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/gadget-testing.txt 2024-03-27T06:45:15.519 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/gadget_configfs.txt 2024-03-27T06:45:15.520 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/gadget_hid.txt 2024-03-27T06:45:15.522 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/gadget_multi.txt 2024-03-27T06:45:15.524 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/gadget_printer.txt 2024-03-27T06:45:15.525 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/gadget_serial.txt 2024-03-27T06:45:15.526 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/hotplug.txt 2024-03-27T06:45:15.527 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/iuu_phoenix.txt 2024-03-27T06:45:15.528 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/linux-cdc-acm.inf 2024-03-27T06:45:15.530 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/linux.inf 2024-03-27T06:45:15.531 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/mass-storage.txt 2024-03-27T06:45:15.532 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/misc_usbsevseg.txt 2024-03-27T06:45:15.533 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/mtouchusb.txt 2024-03-27T06:45:15.535 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/ohci.txt 2024-03-27T06:45:15.536 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/persist.txt 2024-03-27T06:45:15.537 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/power-management.txt 2024-03-27T06:45:15.538 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/proc_usb_info.txt 2024-03-27T06:45:15.539 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/rio.txt 2024-03-27T06:45:15.540 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/usb-help.txt 2024-03-27T06:45:15.542 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/usb-serial.txt 2024-03-27T06:45:15.544 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/usbmon.txt 2024-03-27T06:45:15.546 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/usb/wusb-cbaf 2024-03-27T06:45:15.548 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vDSO/ 2024-03-27T06:45:15.550 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vDSO/.gitignore 2024-03-27T06:45:15.554 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vDSO/Makefile 2024-03-27T06:45:15.555 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vDSO/parse_vdso.c 2024-03-27T06:45:15.556 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vDSO/vdso_standalone_test_x86.c 2024-03-27T06:45:15.558 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vDSO/vdso_test.c 2024-03-27T06:45:15.560 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vfio.txt 2024-03-27T06:45:15.562 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vgaarbiter.txt 2024-03-27T06:45:15.564 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video-output.txt 2024-03-27T06:45:15.566 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/ 2024-03-27T06:45:15.568 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/4CCs.txt 2024-03-27T06:45:15.569 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/API.html 2024-03-27T06:45:15.571 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.au0828 2024-03-27T06:45:15.572 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.bttv 2024-03-27T06:45:15.575 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx23885 2024-03-27T06:45:15.576 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx88 2024-03-27T06:45:15.578 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.em28xx 2024-03-27T06:45:15.580 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.ivtv 2024-03-27T06:45:15.582 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7134 2024-03-27T06:45:15.584 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7164 2024-03-27T06:45:15.586 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tm6000 2024-03-27T06:45:15.588 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tuner 2024-03-27T06:45:15.589 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.usbvision 2024-03-27T06:45:15.591 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/Makefile 2024-03-27T06:45:15.592 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/README.cpia2 2024-03-27T06:45:15.593 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/README.cx88 2024-03-27T06:45:15.594 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/README.davinci-vpbe 2024-03-27T06:45:15.596 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/README.ir 2024-03-27T06:45:15.597 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/README.ivtv 2024-03-27T06:45:15.598 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/README.pvrusb2 2024-03-27T06:45:15.599 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/README.saa7134 2024-03-27T06:45:15.600 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/Zoran 2024-03-27T06:45:15.601 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/ 2024-03-27T06:45:15.603 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/CONTRIBUTORS 2024-03-27T06:45:15.604 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/Cards 2024-03-27T06:45:15.605 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/ICs 2024-03-27T06:45:15.606 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/Insmod-options 2024-03-27T06:45:15.608 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/MAKEDEV 2024-03-27T06:45:15.610 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modprobe.conf 2024-03-27T06:45:15.612 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modules.conf 2024-03-27T06:45:15.614 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/PROBLEMS 2024-03-27T06:45:15.615 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/README 2024-03-27T06:45:15.617 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.WINVIEW 2024-03-27T06:45:15.619 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.freeze 2024-03-27T06:45:15.620 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.quirks 2024-03-27T06:45:15.621 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/Sound-FAQ 2024-03-27T06:45:15.622 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/Specs 2024-03-27T06:45:15.623 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/THANKS 2024-03-27T06:45:15.625 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/bttv/Tuners 2024-03-27T06:45:15.628 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cafe_ccic 2024-03-27T06:45:15.630 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cpia2_overview.txt 2024-03-27T06:45:15.632 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx18.txt 2024-03-27T06:45:15.633 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/ 2024-03-27T06:45:15.635 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.hm12 2024-03-27T06:45:15.637 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.vbi 2024-03-27T06:45:15.638 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-calling.txt 2024-03-27T06:45:15.640 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-api.txt 2024-03-27T06:45:15.641 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-regs.txt 2024-03-27T06:45:15.643 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-dma.txt 2024-03-27T06:45:15.644 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-encoder-api.txt 2024-03-27T06:45:15.646 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-memory.txt 2024-03-27T06:45:15.648 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-osd-api.txt 2024-03-27T06:45:15.649 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-upload.txt 2024-03-27T06:45:15.651 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx88/ 2024-03-27T06:45:15.653 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/cx88/hauppauge-wintv-cx88-ir.txt 2024-03-27T06:45:15.654 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/extract_xc3028.pl 2024-03-27T06:45:15.656 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/fimc.txt 2024-03-27T06:45:15.657 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/gspca.txt 2024-03-27T06:45:15.659 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/hauppauge-wintv-cx88-ir.txt 2024-03-27T06:45:15.660 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/lifeview.txt 2024-03-27T06:45:15.662 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/meye.txt 2024-03-27T06:45:15.664 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/not-in-cx2388x-datasheet.txt 2024-03-27T06:45:15.665 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/omap3isp.txt 2024-03-27T06:45:15.667 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/omap4_camera.txt 2024-03-27T06:45:15.669 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/pxa_camera.txt 2024-03-27T06:45:15.671 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/radiotrack.txt 2024-03-27T06:45:15.673 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/sh_mobile_ceu_camera.txt 2024-03-27T06:45:15.674 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/si470x.txt 2024-03-27T06:45:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 -- 172.21.15.16:0/4058824787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c1059f0 msgr2=0x7f925c105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 --2- 172.21.15.16:0/4058824787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c1059f0 0x7f925c105dd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9250009900 tx=0x7f925002bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 -- 172.21.15.16:0/4058824787 shutdown_connections 2024-03-27T06:45:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 --2- 172.21.15.16:0/4058824787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f925c101660 0x7f925c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 --2- 172.21.15.16:0/4058824787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f925c100ca0 0x7f925c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 --2- 172.21.15.16:0/4058824787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c1059f0 0x7f925c105dd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 -- 172.21.15.16:0/4058824787 >> 172.21.15.16:0/4058824787 conn(0x7f925c0778a0 msgr2=0x7f925c077cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 -- 172.21.15.16:0/4058824787 shutdown_connections 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 -- 172.21.15.16:0/4058824787 wait complete. 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 Processor -- start 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 -- start start 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f925c100ca0 0x7f925c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c101660 0x7f925c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f925c1059f0 0x7f925c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f925c110ff0 con 0x7f925c101660 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f925c110e70 con 0x7f925c100ca0 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.680+0000 7f9262efa640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f925c111170 con 0x7f925c1059f0 2024-03-27T06:45:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f925bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c101660 0x7f925c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f925bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c101660 0x7f925c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58742/0 (socket says 172.21.15.16:58742) 2024-03-27T06:45:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f925bfff640 1 -- 172.21.15.16:0/387954746 learned_addr learned my addr 172.21.15.16:0/387954746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f9260c96640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f925c100ca0 0x7f925c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f9261497640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f925c1059f0 0x7f925c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f925bfff640 1 -- 172.21.15.16:0/387954746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f925c1059f0 msgr2=0x7f925c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f925bfff640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f925c1059f0 0x7f925c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f925bfff640 1 -- 172.21.15.16:0/387954746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f925c100ca0 msgr2=0x7f925c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f925bfff640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f925c100ca0 0x7f925c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f925bfff640 1 -- 172.21.15.16:0/387954746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c19d4b0 con 0x7f925c101660 2024-03-27T06:45:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f925bfff640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c101660 0x7f925c198a00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f924800e9f0 tx=0x7f924800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:15.678 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/si4713.txt 2024-03-27T06:45:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f9259ffb640 1 -- 172.21.15.16:0/387954746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924800cd80 con 0x7f925c101660 2024-03-27T06:45:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f9259ffb640 1 -- 172.21.15.16:0/387954746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9248010930 con 0x7f925c101660 2024-03-27T06:45:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f9262efa640 1 -- 172.21.15.16:0/387954746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c19d7a0 con 0x7f925c101660 2024-03-27T06:45:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.681+0000 7f9259ffb640 1 -- 172.21.15.16:0/387954746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248018b00 con 0x7f925c101660 2024-03-27T06:45:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.682+0000 7f9262efa640 1 -- 172.21.15.16:0/387954746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c1a5030 con 0x7f925c101660 2024-03-27T06:45:15.680 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/si476x.txt 2024-03-27T06:45:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.684+0000 7f9259ffb640 1 -- 172.21.15.16:0/387954746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f9248018ca0 con 0x7f925c101660 2024-03-27T06:45:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.684+0000 7f9259ffb640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f924c0a38a0 0x7f924c0a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.684+0000 7f9260c96640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f924c0a38a0 0x7f924c0a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.684+0000 7f9259ffb640 1 -- 172.21.15.16:0/387954746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f92480d03f0 con 0x7f925c101660 2024-03-27T06:45:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.685+0000 7f9260c96640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f924c0a38a0 0x7f924c0a5d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9250006010 tx=0x7f9250005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:15.681 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/soc-camera.txt 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.685+0000 7f9265bb7740 1 Processor -- start 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.685+0000 7f9265bb7740 1 -- start start 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.685+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e620f0 0x555b84e624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e6b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e6b7e0 0x555b84e6dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9265bb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b84e28cd0 con 0x555b84e62a10 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9265bb7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555b84d6d950 con 0x555b84e620f0 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9265bb7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555b84dc5e50 con 0x555b84e6b7e0 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f925bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e6b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9260c96640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e620f0 0x555b84e624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f925bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e6b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58744/0 (socket says 172.21.15.16:58744) 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f925bfff640 1 -- 172.21.15.16:0/597730940 learned_addr learned my addr 172.21.15.16:0/597730940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9261497640 1 --2- 172.21.15.16:0/597730940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e6b7e0 0x555b84e6dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9260c96640 1 -- 172.21.15.16:0/597730940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e6b7e0 msgr2=0x555b84e6dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9260c96640 1 --2- 172.21.15.16:0/597730940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e6b7e0 0x555b84e6dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9260c96640 1 -- 172.21.15.16:0/597730940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 msgr2=0x555b84e6b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9260c96640 1 --2- 172.21.15.16:0/597730940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e6b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9260c96640 1 -- 172.21.15.16:0/597730940 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b84ada210 con 0x555b84e620f0 2024-03-27T06:45:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f925bfff640 1 --2- 172.21.15.16:0/597730940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e6b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9261497640 1 --2- 172.21.15.16:0/597730940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e6b7e0 0x555b84e6dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9260c96640 1 --2- 172.21.15.16:0/597730940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e620f0 0x555b84e624d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9250000c00 tx=0x7f925003e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3842e3c8fde6701c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9246ffd640 1 -- 172.21.15.16:0/597730940 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250050070 con 0x555b84e620f0 2024-03-27T06:45:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9246ffd640 1 -- 172.21.15.16:0/597730940 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9250005540 con 0x555b84e620f0 2024-03-27T06:45:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.686+0000 7f9246ffd640 1 -- 172.21.15.16:0/597730940 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250047040 con 0x555b84e620f0 2024-03-27T06:45:15.685 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/uvcvideo.txt 2024-03-27T06:45:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.687+0000 7f9265bb7740 1 -- 172.21.15.16:0/597730940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e620f0 msgr2=0x555b84e624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.687+0000 7f9265bb7740 1 --2- 172.21.15.16:0/597730940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e620f0 0x555b84e624d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9250000c00 tx=0x7f925003e000 comp rx=0 tx=0).stop 2024-03-27T06:45:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.687+0000 7f9265bb7740 1 -- 172.21.15.16:0/597730940 shutdown_connections 2024-03-27T06:45:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.687+0000 7f9265bb7740 1 --2- 172.21.15.16:0/597730940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e6b7e0 0x555b84e6dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.687+0000 7f9265bb7740 1 --2- 172.21.15.16:0/597730940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e6b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.687+0000 7f9265bb7740 1 --2- 172.21.15.16:0/597730940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e620f0 0x555b84e624d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.688+0000 7f9265bb7740 1 -- 172.21.15.16:0/597730940 >> 172.21.15.16:0/597730940 conn(0x555b84e5ca10 msgr2=0x555b84e5cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.688+0000 7f9265bb7740 1 -- 172.21.15.16:0/597730940 shutdown_connections 2024-03-27T06:45:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.688+0000 7f9265bb7740 1 -- 172.21.15.16:0/597730940 wait complete. 2024-03-27T06:45:15.687 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/v4l2-controls.txt 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.688+0000 7f9265bb7740 1 -- start start 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.688+0000 7f9265bb7740 1 Processor -- start 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e68800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e62a10 0x555b84e68d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e6b7e0 0x555b84e69280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9265bb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b84dc1e00 con 0x555b84e6b7e0 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9265bb7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555b84d6e520 con 0x555b84e62a10 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9265bb7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555b84d818e0 con 0x555b84e620f0 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9265bb7740 10 client.0 _ll_register_callbacks cb 0x555b84dbca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9261497640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e6b7e0 0x555b84e69280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f925bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e62a10 0x555b84e68d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9260c96640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e68800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9260c96640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e68800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:42962/0 (socket says 172.21.15.16:42962) 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f925bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e62a10 0x555b84e68d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:54434/0 (socket says 172.21.15.16:54434) 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9261497640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e6b7e0 0x555b84e69280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58746/0 (socket says 172.21.15.16:58746) 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f925bfff640 1 -- 172.21.15.16:0/366275334 learned_addr learned my addr 172.21.15.16:0/366275334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f925bfff640 1 -- 172.21.15.16:0/366275334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 msgr2=0x555b84e68800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f925bfff640 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e68800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f925bfff640 1 -- 172.21.15.16:0/366275334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e6b7e0 msgr2=0x555b84e69280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f925bfff640 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e6b7e0 0x555b84e69280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f925bfff640 1 -- 172.21.15.16:0/366275334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b84ba6620 con 0x555b84e62a10 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9260c96640 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e68800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f925bfff640 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e62a10 0x555b84e68d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9248099030 tx=0x7f924809c260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98f7fae31d1358b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f923bfff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-27T06:45:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f923bfff640 1 -- 172.21.15.16:0/366275334 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92480b5060 con 0x555b84e62a10 2024-03-27T06:45:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f9265bb7740 1 -- 172.21.15.16:0/366275334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9250005540 con 0x555b84e62a10 2024-03-27T06:45:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.689+0000 7f923bfff640 1 -- 172.21.15.16:0/366275334 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f9248010ae0 con 0x555b84e62a10 2024-03-27T06:45:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f923bfff640 1 -- 172.21.15.16:0/366275334 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92480b0e60 con 0x555b84e62a10 2024-03-27T06:45:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f9265bb7740 10 client.25862 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f9265bb7740 1 -- 172.21.15.16:0/366275334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b84e7a3a0 con 0x555b84e62a10 2024-03-27T06:45:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f923bfff640 1 -- 172.21.15.16:0/366275334 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9250005540 con 0x555b84e62a10 2024-03-27T06:45:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f923bfff640 1 -- 172.21.15.16:0/366275334 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f9248085ba0 con 0x555b84e62a10 2024-03-27T06:45:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f9265bb7740 10 client.25862 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f9265bb7740 10 client.25862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f9265bb7740 20 client.25862 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f9265bb7740 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555b84e7b8b0 0x555b84e7dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.692 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/v4l2-framework.txt 2024-03-27T06:45:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f9265bb7740 4 client.25862 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f9265bb7740 1 -- 172.21.15.16:0/366275334 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b84dd3390 con 0x555b84e7b8b0 2024-03-27T06:45:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.690+0000 7f9260c96640 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555b84e7b8b0 0x555b84e7dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.691+0000 7f9260c96640 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555b84e7b8b0 0x555b84e7dd00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25499eef739d2947 server_cookie=2616194bd1559cb0 in_seq=0 out_seq=0 2024-03-27T06:45:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.691+0000 7f923bfff640 10 client.25862 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:15.694 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/v4l2-pci-skeleton.c 2024-03-27T06:45:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.693+0000 7f923bfff640 1 -- 172.21.15.16:0/366275334 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555b84dd3390 con 0x555b84e7b8b0 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.693+0000 7f923bfff640 10 client.25862 handle_command_reply: tid=0 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.693+0000 7f9265bb7740 1 client.25862 shutdown 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.693+0000 7f9265bb7740 1 -- 172.21.15.16:0/366275334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e62a10 msgr2=0x555b84e68d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.693+0000 7f9265bb7740 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e62a10 0x555b84e68d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9248099030 tx=0x7f924809c260 comp rx=0 tx=0).stop 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.693+0000 7f9265bb7740 1 -- 172.21.15.16:0/366275334 shutdown_connections 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.693+0000 7f9265bb7740 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555b84e7b8b0 0x555b84e7dd00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.693+0000 7f9265bb7740 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e6b7e0 0x555b84e69280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.694+0000 7f9265bb7740 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e62a10 0x555b84e68d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.694+0000 7f9265bb7740 1 --2- 172.21.15.16:0/366275334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e68800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.694+0000 7f923bfff640 0 client.25862 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.694+0000 7f9265bb7740 1 -- 172.21.15.16:0/366275334 >> 172.21.15.16:0/366275334 conn(0x555b84e5ca10 msgr2=0x555b84e5e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.694+0000 7f9265bb7740 1 -- 172.21.15.16:0/366275334 shutdown_connections 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.694+0000 7f9265bb7740 1 -- 172.21.15.16:0/366275334 wait complete. 2024-03-27T06:45:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.694+0000 7f9265bb7740 20 client.25862 trim_cache size 0 max 16384 2024-03-27T06:45:15.696 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/videobuf 2024-03-27T06:45:15.696 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/vivid.txt 2024-03-27T06:45:15.696 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/video4linux/zr364xx.txt 2024-03-27T06:45:15.696 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/ 2024-03-27T06:45:15.697 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/00-INDEX 2024-03-27T06:45:15.699 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/ 2024-03-27T06:45:15.700 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/00-INDEX 2024-03-27T06:45:15.702 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/api.txt 2024-03-27T06:45:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f9265bb7740 1 Processor -- start 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f9265bb7740 1 -- start start 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e76e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e6b7e0 0x555b84e65a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f9265bb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1ad720 con 0x555b84e62a10 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f9265bb7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1af320 con 0x555b84e6b7e0 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f9265bb7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1addf0 con 0x555b84e620f0 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f925bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f925bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e835d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51686/0 (socket says 172.21.15.16:51686) 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f925bfff640 1 -- 172.21.15.16:0/351249714 learned_addr learned my addr 172.21.15.16:0/351249714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f9261497640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e6b7e0 0x555b84e65a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.709+0000 7f9260c96640 1 --2- 172.21.15.16:0/351249714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e76e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.710+0000 7f9260c96640 1 -- 172.21.15.16:0/351249714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e6b7e0 msgr2=0x555b84e65a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.710+0000 7f9260c96640 1 --2- 172.21.15.16:0/351249714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e6b7e0 0x555b84e65a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.710+0000 7f9260c96640 1 -- 172.21.15.16:0/351249714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 msgr2=0x555b84e835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.710+0000 7f9260c96640 1 --2- 172.21.15.16:0/351249714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.710+0000 7f9260c96640 1 -- 172.21.15.16:0/351249714 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b84e77390 con 0x555b84e620f0 2024-03-27T06:45:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.710+0000 7f925bfff640 1 --2- 172.21.15.16:0/351249714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.710+0000 7f9260c96640 1 --2- 172.21.15.16:0/351249714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e76e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9250036040 tx=0x7f92500049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2aa12d08ccddc68 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:15.706 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/cpuid.txt 2024-03-27T06:45:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.710+0000 7f92467fc640 1 -- 172.21.15.16:0/351249714 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250047040 con 0x555b84e620f0 2024-03-27T06:45:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.710+0000 7f92467fc640 1 -- 172.21.15.16:0/351249714 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f92500044d0 con 0x555b84e620f0 2024-03-27T06:45:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.710+0000 7f92467fc640 1 -- 172.21.15.16:0/351249714 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250050040 con 0x555b84e620f0 2024-03-27T06:45:15.707 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/ 2024-03-27T06:45:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.711+0000 7f9265bb7740 1 -- 172.21.15.16:0/351249714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 msgr2=0x555b84e76e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.711+0000 7f9265bb7740 1 --2- 172.21.15.16:0/351249714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e76e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9250036040 tx=0x7f92500049b0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.711+0000 7f9265bb7740 1 -- 172.21.15.16:0/351249714 shutdown_connections 2024-03-27T06:45:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.711+0000 7f9265bb7740 1 --2- 172.21.15.16:0/351249714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e6b7e0 0x555b84e65a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.711+0000 7f9265bb7740 1 --2- 172.21.15.16:0/351249714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.711+0000 7f9265bb7740 1 --2- 172.21.15.16:0/351249714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e76e50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.711+0000 7f9265bb7740 1 -- 172.21.15.16:0/351249714 >> 172.21.15.16:0/351249714 conn(0x555b84e5ca10 msgr2=0x555b84e5e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.711+0000 7f9265bb7740 1 -- 172.21.15.16:0/351249714 shutdown_connections 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.711+0000 7f9265bb7740 1 -- 172.21.15.16:0/351249714 wait complete. 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.711+0000 7f9265bb7740 1 -- start start 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f9265bb7740 1 Processor -- start 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e73770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e70450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f9265bb7740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e6b7e0 0x555b84e709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f9265bb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1aec50 con 0x555b84e62a10 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f9265bb7740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1ae4c0 con 0x555b84e6b7e0 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f9265bb7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1ab700 con 0x555b84e620f0 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f9265bb7740 10 client.0 _ll_register_callbacks cb 0x7f924800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f925bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e70450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f9260c96640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e73770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f925bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e70450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51690/0 (socket says 172.21.15.16:51690) 2024-03-27T06:45:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f925bfff640 1 -- 172.21.15.16:0/3258667665 learned_addr learned my addr 172.21.15.16:0/3258667665 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.712+0000 7f9261497640 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e6b7e0 0x555b84e709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.711 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/README 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f925bfff640 1 -- 172.21.15.16:0/3258667665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 msgr2=0x555b84e73770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f925bfff640 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e73770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f925bfff640 1 -- 172.21.15.16:0/3258667665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e6b7e0 msgr2=0x555b84e709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f925bfff640 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e6b7e0 0x555b84e709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f925bfff640 1 -- 172.21.15.16:0/3258667665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b84e71200 con 0x555b84e62a10 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f9260c96640 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e73770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f925bfff640 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e70450 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f924809c5f0 tx=0x7f9248085dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb254f42b05c061e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f923bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f923bfff640 1 -- 172.21.15.16:0/3258667665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248098900 con 0x555b84e62a10 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f923bfff640 1 -- 172.21.15.16:0/3258667665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f924809bd80 con 0x555b84e62a10 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f9265bb7740 1 -- 172.21.15.16:0/3258667665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92500044d0 con 0x555b84e62a10 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f923bfff640 1 -- 172.21.15.16:0/3258667665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9248010cf0 con 0x555b84e62a10 2024-03-27T06:45:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f9265bb7740 10 client.16038 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f9265bb7740 1 -- 172.21.15.16:0/3258667665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b84ee7090 con 0x555b84e62a10 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f923bfff640 1 -- 172.21.15.16:0/3258667665 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f92500044d0 con 0x555b84e62a10 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f923bfff640 1 -- 172.21.15.16:0/3258667665 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f92480b5b40 con 0x555b84e62a10 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f9265bb7740 10 client.16038 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f9265bb7740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f9265bb7740 20 client.16038 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f9265bb7740 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555b84ee7890 0x555b84ee9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f9265bb7740 4 client.16038 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.713+0000 7f9265bb7740 1 -- 172.21.15.16:0/3258667665 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b84dcd6e0 con 0x555b84ee7890 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.714+0000 7f9260c96640 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555b84ee7890 0x555b84ee9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.714+0000 7f9260c96640 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555b84ee7890 0x555b84ee9cb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738e4ab0375ba74c server_cookie=443191d613f5f668 in_seq=0 out_seq=0 2024-03-27T06:45:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:15.714+0000 7f923bfff640 10 client.16038 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:15.715 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt 2024-03-27T06:45:15.715 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/mpic.txt 2024-03-27T06:45:15.715 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/s390_flic.txt 2024-03-27T06:45:15.715 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vfio.txt 2024-03-27T06:45:15.718 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vm.txt 2024-03-27T06:45:15.719 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/xics.txt 2024-03-27T06:45:15.721 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/hypercalls.txt 2024-03-27T06:45:15.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:15 smithi156 ceph-mon[36261]: pgmap v53: 97 pgs: 97 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 47 op/s 2024-03-27T06:45:15.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:15 smithi156 ceph-mon[36261]: osdmap e90: 12 total, 12 up, 12 in 2024-03-27T06:45:15.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:15 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6834/681514902,v1:172.21.15.170:6835/681514902] up:boot 2024-03-27T06:45:15.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:15 smithi156 ceph-mon[36261]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-27T06:45:15.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:15 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:45:15.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:15 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:15.723 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/locking.txt 2024-03-27T06:45:15.724 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/mmu.txt 2024-03-27T06:45:15.726 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/msr.txt 2024-03-27T06:45:15.727 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/nested-vmx.txt 2024-03-27T06:45:15.729 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/ppc-pv.txt 2024-03-27T06:45:15.730 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/review-checklist.txt 2024-03-27T06:45:15.732 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/s390-diag.txt 2024-03-27T06:45:15.734 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/kvm/timekeeping.txt 2024-03-27T06:45:15.736 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/paravirt_ops.txt 2024-03-27T06:45:15.738 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/uml/ 2024-03-27T06:45:15.739 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt 2024-03-27T06:45:15.742 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/ 2024-03-27T06:45:15.746 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/.gitignore 2024-03-27T06:45:15.748 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/00-INDEX 2024-03-27T06:45:15.750 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/active_mm.txt 2024-03-27T06:45:15.752 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/balance 2024-03-27T06:45:15.755 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/cleancache.txt 2024-03-27T06:45:15.759 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/frontswap.txt 2024-03-27T06:45:15.759 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/highmem.txt 2024-03-27T06:45:15.760 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/hugetlbpage.txt 2024-03-27T06:45:15.761 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/hwpoison.txt 2024-03-27T06:45:15.763 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/ksm.txt 2024-03-27T06:45:15.765 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/numa 2024-03-27T06:45:15.767 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/numa_memory_policy.txt 2024-03-27T06:45:15.769 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/overcommit-accounting 2024-03-27T06:45:15.771 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/page_migration 2024-03-27T06:45:15.773 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/page_owner.txt 2024-03-27T06:45:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/pagemap.txt 2024-03-27T06:45:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/remap_file_pages.txt 2024-03-27T06:45:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/slub.txt 2024-03-27T06:45:15.781 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/soft-dirty.txt 2024-03-27T06:45:15.783 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/split_page_table_lock 2024-03-27T06:45:15.785 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/transhuge.txt 2024-03-27T06:45:15.788 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/unevictable-lru.txt 2024-03-27T06:45:15.790 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vm/zswap.txt 2024-03-27T06:45:15.792 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/vme_api.txt 2024-03-27T06:45:15.794 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/volatile-considered-harmful.txt 2024-03-27T06:45:15.796 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/ 2024-03-27T06:45:15.798 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/00-INDEX 2024-03-27T06:45:15.800 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/masters/ 2024-03-27T06:45:15.802 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/masters/00-INDEX 2024-03-27T06:45:15.804 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/masters/ds2482 2024-03-27T06:45:15.806 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/masters/ds2490 2024-03-27T06:45:15.807 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/masters/mxc-w1 2024-03-27T06:45:15.809 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/masters/omap-hdq 2024-03-27T06:45:15.811 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/masters/w1-gpio 2024-03-27T06:45:15.812 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/slaves/ 2024-03-27T06:45:15.814 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/slaves/00-INDEX 2024-03-27T06:45:15.815 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2406 2024-03-27T06:45:15.817 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2423 2024-03-27T06:45:15.819 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds28e04 2024-03-27T06:45:15.821 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/slaves/w1_therm 2024-03-27T06:45:15.823 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/w1.generic 2024-03-27T06:45:15.824 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/w1/w1.netlink 2024-03-27T06:45:15.826 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/ 2024-03-27T06:45:15.828 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/Makefile 2024-03-27T06:45:15.830 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/convert_drivers_to_kernel_api.txt 2024-03-27T06:45:15.831 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/hpwdt.txt 2024-03-27T06:45:15.834 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/pcwd-watchdog.txt 2024-03-27T06:45:15.836 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/src/ 2024-03-27T06:45:15.838 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/src/.gitignore 2024-03-27T06:45:15.840 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/src/Makefile 2024-03-27T06:45:15.842 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-simple.c 2024-03-27T06:45:15.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:15 smithi170 ceph-mon[36438]: pgmap v53: 97 pgs: 97 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 47 op/s 2024-03-27T06:45:15.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:15 smithi170 ceph-mon[36438]: osdmap e90: 12 total, 12 up, 12 in 2024-03-27T06:45:15.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:15 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6834/681514902,v1:172.21.15.170:6835/681514902] up:boot 2024-03-27T06:45:15.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:15 smithi170 ceph-mon[36438]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-27T06:45:15.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:15 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:45:15.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:15 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:15.845 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-test.c 2024-03-27T06:45:15.847 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/watchdog-api.txt 2024-03-27T06:45:15.849 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/watchdog-kernel-api.txt 2024-03-27T06:45:15.851 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/watchdog-parameters.txt 2024-03-27T06:45:15.853 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/watchdog/wdt.txt 2024-03-27T06:45:15.855 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/wimax/ 2024-03-27T06:45:15.857 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/wimax/README.i2400m 2024-03-27T06:45:15.859 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/wimax/README.wimax 2024-03-27T06:45:15.861 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/workqueue.txt 2024-03-27T06:45:15.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:15 smithi016 ceph-mon[31908]: pgmap v53: 97 pgs: 97 active+clean; 82 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 47 op/s 2024-03-27T06:45:15.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:15 smithi016 ceph-mon[31908]: osdmap e90: 12 total, 12 up, 12 in 2024-03-27T06:45:15.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:15 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6834/681514902,v1:172.21.15.170:6835/681514902] up:boot 2024-03-27T06:45:15.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:15 smithi016 ceph-mon[31908]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-27T06:45:15.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:15 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:45:15.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:15 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:15.863 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/ 2024-03-27T06:45:15.865 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/00-INDEX 2024-03-27T06:45:15.867 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/boot.txt 2024-03-27T06:45:15.869 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/early-microcode.txt 2024-03-27T06:45:15.871 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/earlyprintk.txt 2024-03-27T06:45:15.872 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/entry_64.txt 2024-03-27T06:45:15.874 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/exception-tables.txt 2024-03-27T06:45:15.875 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/i386/ 2024-03-27T06:45:15.877 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/i386/IO-APIC.txt 2024-03-27T06:45:15.878 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/intel_mpx.txt 2024-03-27T06:45:15.879 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/mtrr.txt 2024-03-27T06:45:15.881 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/pat.txt 2024-03-27T06:45:15.883 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/tlb.txt 2024-03-27T06:45:15.885 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/usb-legacy-support.txt 2024-03-27T06:45:15.887 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/x86_64/ 2024-03-27T06:45:15.889 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/x86_64/00-INDEX 2024-03-27T06:45:15.891 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/x86_64/boot-options.txt 2024-03-27T06:45:15.893 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/x86_64/cpu-hotplug-spec 2024-03-27T06:45:15.895 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/x86_64/fake-numa-for-cpusets 2024-03-27T06:45:15.897 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/x86_64/kernel-stacks 2024-03-27T06:45:15.899 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/x86_64/machinecheck 2024-03-27T06:45:15.901 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/x86_64/mm.txt 2024-03-27T06:45:15.903 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/x86_64/uefi.txt 2024-03-27T06:45:15.905 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/x86/zero-page.txt 2024-03-27T06:45:15.908 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/xillybus.txt 2024-03-27T06:45:15.910 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/xtensa/ 2024-03-27T06:45:15.912 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/xtensa/atomctl.txt 2024-03-27T06:45:15.914 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/xtensa/mmu.txt 2024-03-27T06:45:15.916 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/xz.txt 2024-03-27T06:45:15.919 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/ 2024-03-27T06:45:15.921 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/CodingStyle 2024-03-27T06:45:15.923 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/HOWTO 2024-03-27T06:45:15.925 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/IRQ.txt 2024-03-27T06:45:15.927 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/SecurityBugs 2024-03-27T06:45:15.929 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingDrivers 2024-03-27T06:45:15.932 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingPatches 2024-03-27T06:45:15.933 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/arm/ 2024-03-27T06:45:15.935 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/arm/Booting 2024-03-27T06:45:15.937 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/arm/kernel_user_helpers.txt 2024-03-27T06:45:15.939 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/arm64/ 2024-03-27T06:45:15.940 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/arm64/booting.txt 2024-03-27T06:45:15.942 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/arm64/memory.txt 2024-03-27T06:45:15.944 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/arm64/tagged-pointers.txt 2024-03-27T06:45:15.946 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/basic_profiling.txt 2024-03-27T06:45:15.948 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/email-clients.txt 2024-03-27T06:45:15.950 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/ 2024-03-27T06:45:15.953 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/sysfs.txt 2024-03-27T06:45:15.954 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/gpio.txt 2024-03-27T06:45:15.956 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/io_ordering.txt 2024-03-27T06:45:15.958 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/magic-number.txt 2024-03-27T06:45:15.963 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/oops-tracing.txt 2024-03-27T06:45:15.964 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/sparse.txt 2024-03-27T06:45:15.964 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/stable_api_nonsense.txt 2024-03-27T06:45:15.965 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/stable_kernel_rules.txt 2024-03-27T06:45:15.967 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/ 2024-03-27T06:45:15.968 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/omap3isp.txt 2024-03-27T06:45:15.970 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/v4l2-framework.txt 2024-03-27T06:45:15.972 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zh_CN/volatile-considered-harmful.txt 2024-03-27T06:45:15.973 INFO:tasks.workunit.client.0.smithi016.stdout:linux-4.0.5/Documentation/zorro.txt 2024-03-27T06:45:16.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T06:45:16.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T06:45:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.845+0000 7f209828f640 1 -- 172.21.15.16:0/146033593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090107ad0 msgr2=0x7f2090107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.845+0000 7f209828f640 1 --2- 172.21.15.16:0/146033593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090107ad0 0x7f2090107eb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2080009930 tx=0x7f208002bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.845+0000 7f209828f640 1 -- 172.21.15.16:0/146033593 shutdown_connections 2024-03-27T06:45:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.845+0000 7f209828f640 1 --2- 172.21.15.16:0/146033593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20900696f0 0x7f209010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.845+0000 7f209828f640 1 --2- 172.21.15.16:0/146033593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2090068d30 0x7f20900691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.845+0000 7f209828f640 1 --2- 172.21.15.16:0/146033593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090107ad0 0x7f2090107eb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.845+0000 7f209828f640 1 -- 172.21.15.16:0/146033593 >> 172.21.15.16:0/146033593 conn(0x7f2090077b50 msgr2=0x7f2090077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209828f640 1 -- 172.21.15.16:0/146033593 shutdown_connections 2024-03-27T06:45:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209828f640 1 -- 172.21.15.16:0/146033593 wait complete. 2024-03-27T06:45:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209828f640 1 Processor -- start 2024-03-27T06:45:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209828f640 1 -- start start 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209828f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090068d30 0x7f2090198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209828f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20900696f0 0x7f2090198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209828f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2090107ad0 0x7f209019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209828f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2090111f80 con 0x7f2090068d30 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209828f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f2090111e00 con 0x7f2090107ad0 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209828f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2090112100 con 0x7f20900696f0 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209602b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090068d30 0x7f2090198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209602b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090068d30 0x7f2090198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51716/0 (socket says 172.21.15.16:51716) 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209602b640 1 -- 172.21.15.16:0/1146778028 learned_addr learned my addr 172.21.15.16:0/1146778028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.846+0000 7f209682c640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2090107ad0 0x7f209019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f209582a640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20900696f0 0x7f2090198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f209602b640 1 -- 172.21.15.16:0/1146778028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20900696f0 msgr2=0x7f2090198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f209602b640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20900696f0 0x7f2090198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f209602b640 1 -- 172.21.15.16:0/1146778028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2090107ad0 msgr2=0x7f209019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f209602b640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2090107ad0 0x7f209019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f209602b640 1 -- 172.21.15.16:0/1146778028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209019d4e0 con 0x7f2090068d30 2024-03-27T06:45:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f209602b640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090068d30 0x7f2090198540 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2080009230 tx=0x7f208002dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f20877fe640 1 -- 172.21.15.16:0/1146778028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080043020 con 0x7f2090068d30 2024-03-27T06:45:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f209828f640 1 -- 172.21.15.16:0/1146778028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209019d770 con 0x7f2090068d30 2024-03-27T06:45:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f20877fe640 1 -- 172.21.15.16:0/1146778028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2080004250 con 0x7f2090068d30 2024-03-27T06:45:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f20877fe640 1 -- 172.21.15.16:0/1146778028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080003970 con 0x7f2090068d30 2024-03-27T06:45:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.847+0000 7f209828f640 1 -- 172.21.15.16:0/1146778028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901a5060 con 0x7f2090068d30 2024-03-27T06:45:16.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.849+0000 7f20877fe640 1 -- 172.21.15.16:0/1146778028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f2080046020 con 0x7f2090068d30 2024-03-27T06:45:16.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209ae7e740 1 Processor -- start 2024-03-27T06:45:16.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209ae7e740 1 -- start start 2024-03-27T06:45:16.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe4a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:16.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4ab70 0x555fefe53210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 0x555fefe55b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209ae7e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555fefdae860 con 0x555fefe53750 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209ae7e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555fefd568d0 con 0x555fefe4ab70 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209ae7e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555fefdaa810 con 0x555fefe4a250 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209682c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe4a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209602b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4ab70 0x555fefe53210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209682c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe4a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:33340/0 (socket says 172.21.15.16:33340) 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.850+0000 7f209682c640 1 -- 172.21.15.16:0/2395394095 learned_addr learned my addr 172.21.15.16:0/2395394095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f20877fe640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f207c0a3780 0x7f207c0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f209582a640 1 --2- 172.21.15.16:0/2395394095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 0x555fefe55b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f209682c640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f207c0a3780 0x7f207c0a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f20877fe640 1 -- 172.21.15.16:0/1146778028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f20800f1ff0 con 0x7f2090068d30 2024-03-27T06:45:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f209602b640 1 -- 172.21.15.16:0/2395394095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 msgr2=0x555fefe4a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f209602b640 1 --2- 172.21.15.16:0/2395394095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe4a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f209602b640 1 -- 172.21.15.16:0/2395394095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 msgr2=0x555fefe55b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f209602b640 1 --2- 172.21.15.16:0/2395394095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 0x555fefe55b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f209602b640 1 -- 172.21.15.16:0/2395394095 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fefd61120 con 0x555fefe4ab70 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f209682c640 1 --2- 172.21.15.16:0/2395394095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe4a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f209602b640 1 --2- 172.21.15.16:0/2395394095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4ab70 0x555fefe53210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f20800ef820 tx=0x7f208002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d31bbbb4373516d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f209682c640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f207c0a3780 0x7f207c0a5c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f20880049d0 tx=0x7f2088004ab0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f2084bfe640 1 -- 172.21.15.16:0/2395394095 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20800efe60 con 0x555fefe4ab70 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f2084bfe640 1 -- 172.21.15.16:0/2395394095 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f20800be960 con 0x555fefe4ab70 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.851+0000 7f2084bfe640 1 -- 172.21.15.16:0/2395394095 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20800ef990 con 0x555fefe4ab70 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.852+0000 7f209ae7e740 1 -- 172.21.15.16:0/2395394095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4ab70 msgr2=0x555fefe53210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.852+0000 7f209ae7e740 1 --2- 172.21.15.16:0/2395394095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4ab70 0x555fefe53210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f20800ef820 tx=0x7f208002bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.852+0000 7f209ae7e740 1 -- 172.21.15.16:0/2395394095 shutdown_connections 2024-03-27T06:45:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.852+0000 7f209ae7e740 1 --2- 172.21.15.16:0/2395394095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 0x555fefe55b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.852+0000 7f209ae7e740 1 --2- 172.21.15.16:0/2395394095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4ab70 0x555fefe53210 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.852+0000 7f209ae7e740 1 --2- 172.21.15.16:0/2395394095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe4a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.852+0000 7f209ae7e740 1 -- 172.21.15.16:0/2395394095 >> 172.21.15.16:0/2395394095 conn(0x555fefe44980 msgr2=0x555fefe44d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.852+0000 7f209ae7e740 1 -- 172.21.15.16:0/2395394095 shutdown_connections 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.852+0000 7f209ae7e740 1 -- 172.21.15.16:0/2395394095 wait complete. 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209ae7e740 1 -- start start 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209ae7e740 1 Processor -- start 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4a250 0x555fefe50770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4ab70 0x555fefe50cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 0x555fefe511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209ae7e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555fefe5a710 con 0x555fefe53750 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209ae7e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x555fefd69cd0 con 0x555fefe4a250 2024-03-27T06:45:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209ae7e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555fefe5a890 con 0x555fefe4ab70 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209ae7e740 10 client.0 _ll_register_callbacks cb 0x555fefd5bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209682c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 0x555fefe511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209602b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4a250 0x555fefe50770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209602b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4a250 0x555fefe50770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:55330/0 (socket says 172.21.15.16:55330) 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209682c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 0x555fefe511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51732/0 (socket says 172.21.15.16:51732) 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209602b640 1 -- 172.21.15.16:0/2916455710 learned_addr learned my addr 172.21.15.16:0/2916455710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209582a640 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4ab70 0x555fefe50cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209602b640 1 -- 172.21.15.16:0/2916455710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4ab70 msgr2=0x555fefe50cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209602b640 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4ab70 0x555fefe50cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209602b640 1 -- 172.21.15.16:0/2916455710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 msgr2=0x555fefe511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.853+0000 7f209602b640 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 0x555fefe511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f209602b640 1 -- 172.21.15.16:0/2916455710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fefb8c7d0 con 0x555fefe4a250 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f209682c640 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 0x555fefe511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f209582a640 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4ab70 0x555fefe50cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f209602b640 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4a250 0x555fefe50770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20800bac30 tx=0x7f20800bc770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=441c568db07cb9dd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f2079ffb640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f2079ffb640 1 -- 172.21.15.16:0/2916455710 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20800efe60 con 0x555fefe4a250 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f2079ffb640 1 -- 172.21.15.16:0/2916455710 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f20800c6c40 con 0x555fefe4a250 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f209ae7e740 1 -- 172.21.15.16:0/2916455710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20800be960 con 0x555fefe4a250 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f2079ffb640 1 -- 172.21.15.16:0/2916455710 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20800bdbc0 con 0x555fefe4a250 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f209ae7e740 10 client.25880 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f209ae7e740 1 -- 172.21.15.16:0/2916455710 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555fefac23f0 con 0x555fefe4a250 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f2079ffb640 1 -- 172.21.15.16:0/2916455710 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f20800be960 con 0x555fefe4a250 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.854+0000 7f2079ffb640 1 -- 172.21.15.16:0/2916455710 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f20800c2160 con 0x555fefe4a250 2024-03-27T06:45:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.855+0000 7f209ae7e740 10 client.25880 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.855+0000 7f209ae7e740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.855+0000 7f209ae7e740 20 client.25880 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.855+0000 7f209ae7e740 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555fefe67a80 0x555fefe69e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.855+0000 7f209ae7e740 4 client.25880 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.855+0000 7f209ae7e740 1 -- 172.21.15.16:0/2916455710 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fefdbbda0 con 0x555fefe67a80 2024-03-27T06:45:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.855+0000 7f209582a640 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555fefe67a80 0x555fefe69e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.855+0000 7f2079ffb640 10 client.25880 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:16.855+0000 7f209582a640 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555fefe67a80 0x555fefe69e70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d52aaad8f3c5d8df server_cookie=c85e074008e73684 in_seq=0 out_seq=0 2024-03-27T06:45:17.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:17 smithi156 ceph-mon[36261]: pgmap v55: 97 pgs: 97 active+clean; 88 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 8.7 MiB/s wr, 43 op/s 2024-03-27T06:45:17.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:17 smithi170 ceph-mon[36438]: pgmap v55: 97 pgs: 97 active+clean; 88 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 8.7 MiB/s wr, 43 op/s 2024-03-27T06:45:17.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:17 smithi016 ceph-mon[31908]: pgmap v55: 97 pgs: 97 active+clean; 88 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 8.7 MiB/s wr, 43 op/s 2024-03-27T06:45:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f923bfff640 1 -- 172.21.15.16:0/3258667665 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b84dcd6e0 con 0x555b84ee7890 2024-03-27T06:45:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f923bfff640 10 client.16038 handle_command_reply: tid=0 2024-03-27T06:45:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 1 client.16038 shutdown 2024-03-27T06:45:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 1 -- 172.21.15.16:0/3258667665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 msgr2=0x555b84e70450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e70450 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f924809c5f0 tx=0x7f9248085dd0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 1 -- 172.21.15.16:0/3258667665 shutdown_connections 2024-03-27T06:45:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555b84ee7890 0x555b84ee9cb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555b84e6b7e0 0x555b84e709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b84e62a10 0x555b84e70450 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 1 --2- 172.21.15.16:0/3258667665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555b84e620f0 0x555b84e73770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f923bfff640 0 client.16038 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 1 -- 172.21.15.16:0/3258667665 >> 172.21.15.16:0/3258667665 conn(0x555b84e5ca10 msgr2=0x555b84e5b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 1 -- 172.21.15.16:0/3258667665 shutdown_connections 2024-03-27T06:45:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 1 -- 172.21.15.16:0/3258667665 wait complete. 2024-03-27T06:45:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f9265bb7740 20 client.16038 trim_cache size 0 max 16384 2024-03-27T06:45:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f2079ffb640 1 -- 172.21.15.16:0/2916455710 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555fefdbbda0 con 0x555fefe67a80 2024-03-27T06:45:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f2079ffb640 10 client.25880 handle_command_reply: tid=0 2024-03-27T06:45:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f209ae7e740 1 client.25880 shutdown 2024-03-27T06:45:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:45:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:45:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:45:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:45:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f209ae7e740 1 -- 172.21.15.16:0/2916455710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4a250 msgr2=0x555fefe50770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f209ae7e740 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4a250 0x555fefe50770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20800bac30 tx=0x7f20800bc770 comp rx=0 tx=0).stop 2024-03-27T06:45:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f209ae7e740 1 -- 172.21.15.16:0/2916455710 shutdown_connections 2024-03-27T06:45:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f209ae7e740 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555fefe67a80 0x555fefe69e70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f209ae7e740 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe53750 0x555fefe511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f209ae7e740 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4ab70 0x555fefe50cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f209ae7e740 1 --2- 172.21.15.16:0/2916455710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe4a250 0x555fefe50770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f2079ffb640 0 client.25880 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f209ae7e740 1 -- 172.21.15.16:0/2916455710 >> 172.21.15.16:0/2916455710 conn(0x555fefe44980 msgr2=0x555fefe466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.645+0000 7f209ae7e740 1 -- 172.21.15.16:0/2916455710 shutdown_connections 2024-03-27T06:45:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f209ae7e740 1 -- 172.21.15.16:0/2916455710 wait complete. 2024-03-27T06:45:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f209ae7e740 20 client.25880 trim_cache size 0 max 16384 2024-03-27T06:45:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 -- 172.21.15.16:0/387954746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f924c0a38a0 msgr2=0x7f924c0a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f924c0a38a0 0x7f924c0a5d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9250006010 tx=0x7f9250005e40 comp rx=0 tx=0).stop 2024-03-27T06:45:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 -- 172.21.15.16:0/387954746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c101660 msgr2=0x7f925c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c101660 0x7f925c198a00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f924800e9f0 tx=0x7f924800eeb0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 -- 172.21.15.16:0/387954746 shutdown_connections 2024-03-27T06:45:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f924c0a38a0 0x7f924c0a5d60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f925c1059f0 0x7f925c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f925c101660 0x7f925c198a00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 --2- 172.21.15.16:0/387954746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f925c100ca0 0x7f925c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 -- 172.21.15.16:0/387954746 >> 172.21.15.16:0/387954746 conn(0x7f925c0778a0 msgr2=0x7f925c104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 -- 172.21.15.16:0/387954746 shutdown_connections 2024-03-27T06:45:18.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:18.646+0000 7f9262efa640 1 -- 172.21.15.16:0/387954746 wait complete. 2024-03-27T06:45:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209ae7e740 1 Processor -- start 2024-03-27T06:45:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209ae7e740 1 -- start start 2024-03-27T06:45:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe52ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe53750 0x555fefe6f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209ae7e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20901ab660 con 0x555fefe4ab70 2024-03-27T06:45:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209ae7e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f20901ad260 con 0x555fefe53750 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209ae7e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f20901abd30 con 0x555fefe4a250 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209582a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209582a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51736/0 (socket says 172.21.15.16:51736) 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209582a640 1 -- 172.21.15.16:0/4103591600 learned_addr learned my addr 172.21.15.16:0/4103591600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209682c640 1 --2- 172.21.15.16:0/4103591600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe53750 0x555fefe6f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209602b640 1 --2- 172.21.15.16:0/4103591600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe52ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209582a640 1 -- 172.21.15.16:0/4103591600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 msgr2=0x555fefe52ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209582a640 1 --2- 172.21.15.16:0/4103591600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe52ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209582a640 1 -- 172.21.15.16:0/4103591600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe53750 msgr2=0x555fefe6f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209582a640 1 --2- 172.21.15.16:0/4103591600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe53750 0x555fefe6f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209582a640 1 -- 172.21.15.16:0/4103591600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fefe4a9a0 con 0x555fefe4ab70 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.665+0000 7f209682c640 1 --2- 172.21.15.16:0/4103591600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe53750 0x555fefe6f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.666+0000 7f209582a640 1 --2- 172.21.15.16:0/4103591600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe633b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x555fefe45b10 tx=0x7f208c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b74925db6bfd20 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.666+0000 7f207bfff640 1 -- 172.21.15.16:0/4103591600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c005020 con 0x555fefe4ab70 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.666+0000 7f207bfff640 1 -- 172.21.15.16:0/4103591600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f208c022170 con 0x555fefe4ab70 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.666+0000 7f207bfff640 1 -- 172.21.15.16:0/4103591600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c004a90 con 0x555fefe4ab70 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.666+0000 7f209ae7e740 1 -- 172.21.15.16:0/4103591600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 msgr2=0x555fefe633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.666+0000 7f209ae7e740 1 --2- 172.21.15.16:0/4103591600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe633b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x555fefe45b10 tx=0x7f208c005f90 comp rx=0 tx=0).stop 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 -- 172.21.15.16:0/4103591600 shutdown_connections 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 --2- 172.21.15.16:0/4103591600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe53750 0x555fefe6f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 --2- 172.21.15.16:0/4103591600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe633b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 --2- 172.21.15.16:0/4103591600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe52ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 -- 172.21.15.16:0/4103591600 >> 172.21.15.16:0/4103591600 conn(0x555fefe44980 msgr2=0x555fefe4ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 -- 172.21.15.16:0/4103591600 shutdown_connections 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 -- 172.21.15.16:0/4103591600 wait complete. 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 -- start start 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 Processor -- start 2024-03-27T06:45:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe5aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe5b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe53750 0x555fefecbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20901acb90 con 0x555fefe4ab70 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f20901ac400 con 0x555fefe53750 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f20901a9620 con 0x555fefe4a250 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209ae7e740 10 client.0 _ll_register_callbacks cb 0x7f208c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209582a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe5b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209582a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe5b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51748/0 (socket says 172.21.15.16:51748) 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209582a640 1 -- 172.21.15.16:0/3834535331 learned_addr learned my addr 172.21.15.16:0/3834535331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209682c640 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe53750 0x555fefecbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.667+0000 7f209602b640 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe5aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209582a640 1 -- 172.21.15.16:0/3834535331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 msgr2=0x555fefe5aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209582a640 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe5aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209582a640 1 -- 172.21.15.16:0/3834535331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe53750 msgr2=0x555fefecbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209582a640 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe53750 0x555fefecbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209582a640 1 -- 172.21.15.16:0/3834535331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fefa820f0 con 0x555fefe4ab70 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209582a640 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe5b000 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f208c012f30 tx=0x7f208c012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3163cf18d9d45e30 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f2079ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f2079ffb640 1 -- 172.21.15.16:0/3834535331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c004230 con 0x555fefe4ab70 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f2079ffb640 1 -- 172.21.15.16:0/3834535331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f208c0043d0 con 0x555fefe4ab70 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209ae7e740 1 -- 172.21.15.16:0/3834535331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f208c022170 con 0x555fefe4ab70 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f2079ffb640 1 -- 172.21.15.16:0/3834535331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f208c022320 con 0x555fefe4ab70 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209ae7e740 10 client.16056 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209ae7e740 1 -- 172.21.15.16:0/3834535331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555fefecd4a0 con 0x555fefe4ab70 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f2079ffb640 1 -- 172.21.15.16:0/3834535331 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f208c022170 con 0x555fefe4ab70 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f2079ffb640 1 -- 172.21.15.16:0/3834535331 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f208c00fd00 con 0x555fefe4ab70 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209ae7e740 10 client.16056 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209ae7e740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.668+0000 7f209ae7e740 20 client.16056 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555fefecdcd0 0x555fefed00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 4 client.16056 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 -- 172.21.15.16:0/3834535331 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555fefdb60f0 con 0x555fefecdcd0 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209602b640 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555fefecdcd0 0x555fefed00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209602b640 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555fefecdcd0 0x555fefed00f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c9f124d8c807dc server_cookie=90f0c86a9c2ace80 in_seq=0 out_seq=0 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f2079ffb640 10 client.16056 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f2079ffb640 1 -- 172.21.15.16:0/3834535331 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555fefdb60f0 con 0x555fefecdcd0 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f2079ffb640 10 client.16056 handle_command_reply: tid=0 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 client.16056 shutdown 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 -- 172.21.15.16:0/3834535331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 msgr2=0x555fefe5b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe5b000 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f208c012f30 tx=0x7f208c012690 comp rx=0 tx=0).stop 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 -- 172.21.15.16:0/3834535331 shutdown_connections 2024-03-27T06:45:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x555fefecdcd0 0x555fefed00f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fefe53750 0x555fefecbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fefe4ab70 0x555fefe5b000 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f2079ffb640 0 client.16056 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 --2- 172.21.15.16:0/3834535331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555fefe4a250 0x555fefe5aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 -- 172.21.15.16:0/3834535331 >> 172.21.15.16:0/3834535331 conn(0x555fefe44980 msgr2=0x555fefe43090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.669+0000 7f209ae7e740 1 -- 172.21.15.16:0/3834535331 shutdown_connections 2024-03-27T06:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209ae7e740 1 -- 172.21.15.16:0/3834535331 wait complete. 2024-03-27T06:45:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209ae7e740 20 client.16056 trim_cache size 0 max 16384 2024-03-27T06:45:18.671 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:45:18.671 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-03-27T06:45:18.672 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-03-27T06:45:18.672 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:45:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 -- 172.21.15.16:0/1146778028 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f207c0a3780 msgr2=0x7f207c0a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f207c0a3780 0x7f207c0a5c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f20880049d0 tx=0x7f2088004ab0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 -- 172.21.15.16:0/1146778028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090068d30 msgr2=0x7f2090198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090068d30 0x7f2090198540 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2080009230 tx=0x7f208002dc90 comp rx=0 tx=0).stop 2024-03-27T06:45:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 -- 172.21.15.16:0/1146778028 shutdown_connections 2024-03-27T06:45:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2090107ad0 0x7f209019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f207c0a3780 0x7f207c0a5c40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20900696f0 0x7f2090198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 --2- 172.21.15.16:0/1146778028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090068d30 0x7f2090198540 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 -- 172.21.15.16:0/1146778028 >> 172.21.15.16:0/1146778028 conn(0x7f2090077b50 msgr2=0x7f20900fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 -- 172.21.15.16:0/1146778028 shutdown_connections 2024-03-27T06:45:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.670+0000 7f209828f640 1 -- 172.21.15.16:0/1146778028 wait complete. 2024-03-27T06:45:18.676 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-27T06:45:18.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:45:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.785+0000 7f74d58fb640 1 -- 172.21.15.16:0/2619018203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0102dc0 msgr2=0x7f74d01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.785+0000 7f74d58fb640 1 --2- 172.21.15.16:0/2619018203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0102dc0 0x7f74d01031a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009900 tx=0x7f74c002bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.785+0000 7f74d58fb640 1 -- 172.21.15.16:0/2619018203 shutdown_connections 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.785+0000 7f74d58fb640 1 --2- 172.21.15.16:0/2619018203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74d010c190 0x7f74d010e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.785+0000 7f74d58fb640 1 --2- 172.21.15.16:0/2619018203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74d01036e0 0x7f74d010bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.785+0000 7f74d58fb640 1 --2- 172.21.15.16:0/2619018203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d0102dc0 0x7f74d01031a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.785+0000 7f74d58fb640 1 -- 172.21.15.16:0/2619018203 >> 172.21.15.16:0/2619018203 conn(0x7f74d0077b50 msgr2=0x7f74d0077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.785+0000 7f74d58fb640 1 -- 172.21.15.16:0/2619018203 shutdown_connections 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.785+0000 7f74d58fb640 1 -- 172.21.15.16:0/2619018203 wait complete. 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74d58fb640 1 Processor -- start 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74d58fb640 1 -- start start 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74d58fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74d0102dc0 0x7f74d0100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74d58fb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74d01036e0 0x7f74d0100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74d58fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d010c190 0x7f74d0101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74d58fb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0112060 con 0x7f74d010c190 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74d58fb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0111ee0 con 0x7f74d0102dc0 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74d58fb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01121e0 con 0x7f74d01036e0 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d010c190 0x7f74d0101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d010c190 0x7f74d0101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51774/0 (socket says 172.21.15.16:51774) 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74cf7fe640 1 -- 172.21.15.16:0/613648781 learned_addr learned my addr 172.21.15.16:0/613648781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74ceffd640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74d0102dc0 0x7f74d0100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.786+0000 7f74c67fc640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74d01036e0 0x7f74d0100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74cf7fe640 1 -- 172.21.15.16:0/613648781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74d01036e0 msgr2=0x7f74d0100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74cf7fe640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74d01036e0 0x7f74d0100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74cf7fe640 1 -- 172.21.15.16:0/613648781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74d0102dc0 msgr2=0x7f74d0100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74cf7fe640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74d0102dc0 0x7f74d0100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74cf7fe640 1 -- 172.21.15.16:0/613648781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d01a69e0 con 0x7f74d010c190 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74ceffd640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74d0102dc0 0x7f74d0100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74cf7fe640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d010c190 0x7f74d0101490 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f74c800efd0 tx=0x7f74c800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74d48f9640 1 -- 172.21.15.16:0/613648781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8003b70 con 0x7f74d010c190 2024-03-27T06:45:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74d58fb640 1 -- 172.21.15.16:0/613648781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d01a6cd0 con 0x7f74d010c190 2024-03-27T06:45:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74d48f9640 1 -- 172.21.15.16:0/613648781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f74c8003d10 con 0x7f74d010c190 2024-03-27T06:45:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74d48f9640 1 -- 172.21.15.16:0/613648781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8010a90 con 0x7f74d010c190 2024-03-27T06:45:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.787+0000 7f74d58fb640 1 -- 172.21.15.16:0/613648781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d01a71d0 con 0x7f74d010c190 2024-03-27T06:45:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.788+0000 7f74d58fb640 1 -- 172.21.15.16:0/613648781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d0068940 con 0x7f74d010c190 2024-03-27T06:45:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.790+0000 7f74d48f9640 1 -- 172.21.15.16:0/613648781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f74c8004b40 con 0x7f74d010c190 2024-03-27T06:45:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.790+0000 7f74d48f9640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f74bc0a37b0 0x7f74bc0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.791+0000 7f74ceffd640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f74bc0a37b0 0x7f74bc0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.791+0000 7f74d48f9640 1 -- 172.21.15.16:0/613648781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f74c80d0c00 con 0x7f74d010c190 2024-03-27T06:45:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.791+0000 7f74ceffd640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f74bc0a37b0 0x7f74bc0a5c70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f74c0006010 tx=0x7f74c0005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:18.794+0000 7f74d48f9640 1 -- 172.21.15.16:0/613648781 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f74c809d4c0 con 0x7f74d010c190 2024-03-27T06:45:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.022+0000 7f74d58fb640 1 -- 172.21.15.16:0/613648781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74d0101ef0 con 0x7f74d010c190 2024-03-27T06:45:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.023+0000 7f74d48f9640 1 -- 172.21.15.16:0/613648781 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11936 (secure 0 0 0) 0x7f74d0101ef0 con 0x7f74d010c190 2024-03-27T06:45:19.018 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:45:19.018 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:45:14.458585+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25852":{"gid":25852,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/681514902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":681514902},{"type":"v1","addr":"172.21.15.170:6835","nonce":681514902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527,24467,24500,24493,24472]},"id":1}]} 2024-03-27T06:45:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.025+0000 7f74d58fb640 1 -- 172.21.15.16:0/613648781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f74bc0a37b0 msgr2=0x7f74bc0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.025+0000 7f74d58fb640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f74bc0a37b0 0x7f74bc0a5c70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f74c0006010 tx=0x7f74c0005e40 comp rx=0 tx=0).stop 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.026+0000 7f74d58fb640 1 -- 172.21.15.16:0/613648781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d010c190 msgr2=0x7f74d0101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.026+0000 7f74d58fb640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d010c190 0x7f74d0101490 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f74c800efd0 tx=0x7f74c800c6a0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.026+0000 7f74d58fb640 1 -- 172.21.15.16:0/613648781 shutdown_connections 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.026+0000 7f74d58fb640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d010c190 0x7f74d0101490 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.026+0000 7f74d58fb640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f74bc0a37b0 0x7f74bc0a5c70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.026+0000 7f74d58fb640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74d01036e0 0x7f74d0100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.026+0000 7f74d58fb640 1 --2- 172.21.15.16:0/613648781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74d0102dc0 0x7f74d0100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.026+0000 7f74d58fb640 1 -- 172.21.15.16:0/613648781 >> 172.21.15.16:0/613648781 conn(0x7f74d0077b50 msgr2=0x7f74d00fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.026+0000 7f74d58fb640 1 -- 172.21.15.16:0/613648781 shutdown_connections 2024-03-27T06:45:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.026+0000 7f74d58fb640 1 -- 172.21.15.16:0/613648781 wait complete. 2024-03-27T06:45:19.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-27T06:45:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.144+0000 7fae03925640 1 -- 172.21.15.16:0/1178335115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadfc06a870 msgr2=0x7fadfc10e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.144+0000 7fae03925640 1 --2- 172.21.15.16:0/1178335115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadfc06a870 0x7fadfc10e9d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009cd0 tx=0x7fadf402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.144+0000 7fae03925640 1 -- 172.21.15.16:0/1178335115 shutdown_connections 2024-03-27T06:45:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.144+0000 7fae03925640 1 --2- 172.21.15.16:0/1178335115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadfc06a870 0x7fadfc10e9d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.144+0000 7fae03925640 1 --2- 172.21.15.16:0/1178335115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadfc069eb0 0x7fadfc06a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.144+0000 7fae03925640 1 --2- 172.21.15.16:0/1178335115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadfc107ba0 0x7fadfc107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.144+0000 7fae03925640 1 -- 172.21.15.16:0/1178335115 >> 172.21.15.16:0/1178335115 conn(0x7fadfc077ae0 msgr2=0x7fadfc077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.144+0000 7fae03925640 1 -- 172.21.15.16:0/1178335115 shutdown_connections 2024-03-27T06:45:19.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.144+0000 7fae03925640 1 -- 172.21.15.16:0/1178335115 wait complete. 2024-03-27T06:45:19.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae03925640 1 Processor -- start 2024-03-27T06:45:19.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae03925640 1 -- start start 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae03925640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadfc069eb0 0x7fadfc100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae03925640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadfc06a870 0x7fadfc100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae03925640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadfc107ba0 0x7fadfc101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae03925640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc112010 con 0x7fadfc107ba0 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae03925640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc111e90 con 0x7fadfc06a870 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae03925640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc112190 con 0x7fadfc069eb0 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae01ec2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadfc107ba0 0x7fadfc101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae016c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadfc069eb0 0x7fadfc100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae01ec2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadfc107ba0 0x7fadfc101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51796/0 (socket says 172.21.15.16:51796) 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae01ec2640 1 -- 172.21.15.16:0/4105211330 learned_addr learned my addr 172.21.15.16:0/4105211330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.145+0000 7fae00ec0640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadfc06a870 0x7fadfc100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.146+0000 7fae01ec2640 1 -- 172.21.15.16:0/4105211330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadfc069eb0 msgr2=0x7fadfc100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.146+0000 7fae01ec2640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadfc069eb0 0x7fadfc100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.146+0000 7fae01ec2640 1 -- 172.21.15.16:0/4105211330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadfc06a870 msgr2=0x7fadfc100fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.146+0000 7fae01ec2640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadfc06a870 0x7fadfc100fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.146+0000 7fae01ec2640 1 -- 172.21.15.16:0/4105211330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc1a68d0 con 0x7fadfc107ba0 2024-03-27T06:45:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.146+0000 7fae01ec2640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadfc107ba0 0x7fadfc101500 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fadf4004af0 tx=0x7fadf4032550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.146+0000 7fadf27fc640 1 -- 172.21.15.16:0/4105211330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4039040 con 0x7fadfc107ba0 2024-03-27T06:45:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.146+0000 7fadf27fc640 1 -- 172.21.15.16:0/4105211330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fadf4044070 con 0x7fadfc107ba0 2024-03-27T06:45:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.146+0000 7fae03925640 1 -- 172.21.15.16:0/4105211330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadfc1a6b60 con 0x7fadfc107ba0 2024-03-27T06:45:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.146+0000 7fadf27fc640 1 -- 172.21.15.16:0/4105211330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf40042a0 con 0x7fadfc107ba0 2024-03-27T06:45:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.146+0000 7fae03925640 1 -- 172.21.15.16:0/4105211330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadfc1a7050 con 0x7fadfc107ba0 2024-03-27T06:45:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.148+0000 7fadf27fc640 1 -- 172.21.15.16:0/4105211330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fadf4003970 con 0x7fadfc107ba0 2024-03-27T06:45:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.148+0000 7fae065ef740 1 Processor -- start 2024-03-27T06:45:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae065ef740 1 -- start start 2024-03-27T06:45:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae065ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984617b0 0x55e598461b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae065ef740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 0x55e59846a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae065ef740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e59846acb0 0x55e59846d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae065ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e5983c5710 con 0x55e5984617b0 2024-03-27T06:45:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae065ef740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e59836daf0 con 0x55e59846acb0 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae065ef740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e5983c16c0 con 0x55e5984620d0 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae00ec0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984617b0 0x55e598461b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae016c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 0x55e59846a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae016c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 0x55e59846a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:33412/0 (socket says 172.21.15.16:33412) 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae00ec0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984617b0 0x55e598461b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51804/0 (socket says 172.21.15.16:51804) 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae016c1640 1 -- 172.21.15.16:0/1197218533 learned_addr learned my addr 172.21.15.16:0/1197218533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae00ec0640 1 --2- 172.21.15.16:0/1197218533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e59846acb0 0x55e59846d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fadf27fc640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fade80a37b0 0x7fade80a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae016c1640 1 -- 172.21.15.16:0/1197218533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e59846acb0 msgr2=0x55e59846d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae01ec2640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fade80a37b0 0x7fade80a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae016c1640 1 --2- 172.21.15.16:0/1197218533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e59846acb0 0x55e59846d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae016c1640 1 -- 172.21.15.16:0/1197218533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984617b0 msgr2=0x55e598461b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae016c1640 1 --2- 172.21.15.16:0/1197218533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984617b0 0x55e598461b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fae016c1640 1 -- 172.21.15.16:0/1197218533 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5980d93f0 con 0x55e5984620d0 2024-03-27T06:45:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.149+0000 7fadf27fc640 1 -- 172.21.15.16:0/4105211330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fadf40f2430 con 0x7fadfc107ba0 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.150+0000 7fae016c1640 1 --2- 172.21.15.16:0/1197218533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 0x55e59846a770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009c20 tx=0x7fadf8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f20787c5d0b4d8a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.150+0000 7fade77fe640 1 -- 172.21.15.16:0/1197218533 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8004220 con 0x55e5984620d0 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.150+0000 7fade77fe640 1 -- 172.21.15.16:0/1197218533 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fadf80043c0 con 0x55e5984620d0 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.150+0000 7fade77fe640 1 -- 172.21.15.16:0/1197218533 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf80197b0 con 0x55e5984620d0 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.150+0000 7fae01ec2640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fade80a37b0 0x7fade80a5c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fadf40efd70 tx=0x7fadf40e9620 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.150+0000 7fae065ef740 1 -- 172.21.15.16:0/1197218533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 msgr2=0x55e59846a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.150+0000 7fae065ef740 1 --2- 172.21.15.16:0/1197218533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 0x55e59846a770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009c20 tx=0x7fadf8011810 comp rx=0 tx=0).stop 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 -- 172.21.15.16:0/1197218533 shutdown_connections 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 --2- 172.21.15.16:0/1197218533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e59846acb0 0x55e59846d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 --2- 172.21.15.16:0/1197218533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 0x55e59846a770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 --2- 172.21.15.16:0/1197218533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984617b0 0x55e598461b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 -- 172.21.15.16:0/1197218533 >> 172.21.15.16:0/1197218533 conn(0x55e59845bee0 msgr2=0x55e59845c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 -- 172.21.15.16:0/1197218533 shutdown_connections 2024-03-27T06:45:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 -- 172.21.15.16:0/1197218533 wait complete. 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 -- start start 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 Processor -- start 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e598467c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e5984681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59846acb0 0x55e598468710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e59846daa0 con 0x55e5984620d0 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e598380eb0 con 0x55e5984617b0 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e59846dc20 con 0x55e59846acb0 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.151+0000 7fae065ef740 10 client.0 _ll_register_callbacks cb 0x55e5983bd6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae016c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e5984681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae016c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e5984681d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51820/0 (socket says 172.21.15.16:51820) 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae016c1640 1 -- 172.21.15.16:0/3933541700 learned_addr learned my addr 172.21.15.16:0/3933541700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae00ec0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e598467c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae00ec0640 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59846acb0 0x55e598468710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae016c1640 1 -- 172.21.15.16:0/3933541700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59846acb0 msgr2=0x55e598468710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae016c1640 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59846acb0 0x55e598468710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae016c1640 1 -- 172.21.15.16:0/3933541700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 msgr2=0x55e598467c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae016c1640 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e598467c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae016c1640 1 -- 172.21.15.16:0/3933541700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5981a37d0 con 0x55e5984620d0 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae016c1640 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e5984681d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fadf801b040 tx=0x7fadf80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12bc7cd25e5c2138 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fade4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fade4ff9640 1 -- 172.21.15.16:0/3933541700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8028070 con 0x55e5984620d0 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fade4ff9640 1 -- 172.21.15.16:0/3933541700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fadf8005040 con 0x55e5984620d0 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae065ef740 1 -- 172.21.15.16:0/3933541700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf80043c0 con 0x55e5984620d0 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fade4ff9640 1 -- 172.21.15.16:0/3933541700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fadf8034050 con 0x55e5984620d0 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae065ef740 10 client.16074 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.152+0000 7fae065ef740 1 -- 172.21.15.16:0/3933541700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e598479d50 con 0x55e5984620d0 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.153+0000 7fade4ff9640 1 -- 172.21.15.16:0/3933541700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fadf80043c0 con 0x55e5984620d0 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.153+0000 7fade4ff9640 1 -- 172.21.15.16:0/3933541700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fadf8004a10 con 0x55e5984620d0 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.153+0000 7fae065ef740 10 client.16074 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.153+0000 7fae065ef740 10 client.16074 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-27T06:45:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.153+0000 7fae065ef740 20 client.16074 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.153+0000 7fae065ef740 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55e59847b4b0 0x55e59847d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.153+0000 7fae065ef740 4 client.16074 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.153+0000 7fae065ef740 1 -- 172.21.15.16:0/3933541700 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5983d2c50 con 0x55e59847b4b0 2024-03-27T06:45:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.153+0000 7fae00ec0640 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55e59847b4b0 0x55e59847d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.153+0000 7fade4ff9640 10 client.16074 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:45:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.153+0000 7fae00ec0640 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55e59847b4b0 0x55e59847d8a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8b3a8c4e34bad2e7 server_cookie=b86844b2a94e3f17 in_seq=0 out_seq=0 2024-03-27T06:45:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fade4ff9640 1 -- 172.21.15.16:0/3933541700 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e5983d2c50 con 0x55e59847b4b0 2024-03-27T06:45:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fade4ff9640 10 client.16074 handle_command_reply: tid=0 2024-03-27T06:45:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fae065ef740 1 client.16074 shutdown 2024-03-27T06:45:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fae065ef740 1 -- 172.21.15.16:0/3933541700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 msgr2=0x55e5984681d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fae065ef740 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e5984681d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fadf801b040 tx=0x7fadf80038a0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fae065ef740 1 -- 172.21.15.16:0/3933541700 shutdown_connections 2024-03-27T06:45:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fae065ef740 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55e59847b4b0 0x55e59847d8a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fae065ef740 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59846acb0 0x55e598468710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fade4ff9640 0 client.16074 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:45:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fae065ef740 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e5984681d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fae065ef740 1 --2- 172.21.15.16:0/3933541700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e598467c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fae065ef740 1 -- 172.21.15.16:0/3933541700 >> 172.21.15.16:0/3933541700 conn(0x55e59845bee0 msgr2=0x55e59845dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fae065ef740 1 -- 172.21.15.16:0/3933541700 shutdown_connections 2024-03-27T06:45:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.154+0000 7fae065ef740 1 -- 172.21.15.16:0/3933541700 wait complete. 2024-03-27T06:45:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.155+0000 7fae065ef740 20 client.16074 trim_cache size 0 max 16384 2024-03-27T06:45:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.168+0000 7fae065ef740 1 Processor -- start 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae065ef740 1 -- start start 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae065ef740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e59846a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae065ef740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 0x55e59845d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae065ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59846acb0 0x55e5984808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae065ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1afa10 con 0x55e59846acb0 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae065ef740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1b1610 con 0x55e5984617b0 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae065ef740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1b00e0 con 0x55e5984620d0 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae00ec0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59846acb0 0x55e5984808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae00ec0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59846acb0 0x55e5984808c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51832/0 (socket says 172.21.15.16:51832) 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae00ec0640 1 -- 172.21.15.16:0/3283147876 learned_addr learned my addr 172.21.15.16:0/3283147876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae016c1640 1 --2- 172.21.15.16:0/3283147876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 0x55e59845d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae00ec0640 1 --2- 172.21.15.16:0/3283147876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e59846a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae00ec0640 1 -- 172.21.15.16:0/3283147876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 msgr2=0x55e59845d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae00ec0640 1 --2- 172.21.15.16:0/3283147876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 0x55e59845d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae00ec0640 1 -- 172.21.15.16:0/3283147876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 msgr2=0x55e59846a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae00ec0640 1 --2- 172.21.15.16:0/3283147876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e59846a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae00ec0640 1 -- 172.21.15.16:0/3283147876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e59845bd10 con 0x55e59846acb0 2024-03-27T06:45:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae016c1640 1 --2- 172.21.15.16:0/3283147876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 0x55e59845d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.169+0000 7fae00ec0640 1 --2- 172.21.15.16:0/3283147876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59846acb0 0x55e5984808c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fadec0058c0 tx=0x7fadec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf9b2243c573eef server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fade6ffd640 1 -- 172.21.15.16:0/3283147876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec014e70 con 0x55e59846acb0 2024-03-27T06:45:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fade6ffd640 1 -- 172.21.15.16:0/3283147876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fadec0270d0 con 0x55e59846acb0 2024-03-27T06:45:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fade6ffd640 1 -- 172.21.15.16:0/3283147876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec0257d0 con 0x55e59846acb0 2024-03-27T06:45:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fae065ef740 1 -- 172.21.15.16:0/3283147876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59846acb0 msgr2=0x55e5984808c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fae065ef740 1 --2- 172.21.15.16:0/3283147876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59846acb0 0x55e5984808c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fadec0058c0 tx=0x7fadec0046c0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fae065ef740 1 -- 172.21.15.16:0/3283147876 shutdown_connections 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fae065ef740 1 --2- 172.21.15.16:0/3283147876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59846acb0 0x55e5984808c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fae065ef740 1 --2- 172.21.15.16:0/3283147876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5984620d0 0x55e59845d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fae065ef740 1 --2- 172.21.15.16:0/3283147876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e59846a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fae065ef740 1 -- 172.21.15.16:0/3283147876 >> 172.21.15.16:0/3283147876 conn(0x55e59845bee0 msgr2=0x55e59845d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fae065ef740 1 -- 172.21.15.16:0/3283147876 shutdown_connections 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.170+0000 7fae065ef740 1 -- 172.21.15.16:0/3283147876 wait complete. 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae065ef740 1 -- start start 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae065ef740 1 Processor -- start 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae065ef740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e59847e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae065ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e59847e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae065ef740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59846acb0 0x55e59847ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae065ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1b0f40 con 0x55e5984620d0 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae065ef740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1b07b0 con 0x55e5984617b0 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae065ef740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc1ad9f0 con 0x55e59846acb0 2024-03-27T06:45:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae065ef740 10 client.0 _ll_register_callbacks cb 0x7fadec009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae016c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e59847e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae00ec0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e59847e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae016c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e59847e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51846/0 (socket says 172.21.15.16:51846) 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae016c1640 1 -- 172.21.15.16:0/2989518502 learned_addr learned my addr 172.21.15.16:0/2989518502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.171+0000 7fae00ec0640 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59846acb0 0x55e59847ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fae016c1640 1 -- 172.21.15.16:0/2989518502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59846acb0 msgr2=0x55e59847ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fae016c1640 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59846acb0 0x55e59847ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fae016c1640 1 -- 172.21.15.16:0/2989518502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 msgr2=0x55e59847e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fae016c1640 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e59847e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fae016c1640 1 -- 172.21.15.16:0/2989518502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5984e0450 con 0x55e5984620d0 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fae00ec0640 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e59847e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fae016c1640 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e59847e6f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55e598469260 tx=0x7fadf80096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613d06c6561cb1a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fade4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fade4ff9640 1 -- 172.21.15.16:0/2989518502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf803c070 con 0x55e5984620d0 2024-03-27T06:45:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fade4ff9640 1 -- 172.21.15.16:0/2989518502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fadf8004280 con 0x55e5984620d0 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fae065ef740 1 -- 172.21.15.16:0/2989518502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadec0270d0 con 0x55e5984620d0 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fade4ff9640 1 -- 172.21.15.16:0/2989518502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fadf8047050 con 0x55e5984620d0 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fae065ef740 10 client.16080 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fae065ef740 1 -- 172.21.15.16:0/2989518502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e5984e16b0 con 0x55e5984620d0 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.172+0000 7fade4ff9640 1 -- 172.21.15.16:0/2989518502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fadec0270d0 con 0x55e5984620d0 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fade4ff9640 1 -- 172.21.15.16:0/2989518502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fadf80051a0 con 0x55e5984620d0 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 10 client.16080 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 10 client.16080 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 20 client.16080 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55e5984e1e00 0x55e5984e4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 4 client.16080 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 1 -- 172.21.15.16:0/2989518502 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5983ccfa0 con 0x55e5984e1e00 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae00ec0640 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55e5984e1e00 0x55e5984e4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fade4ff9640 10 client.16080 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae00ec0640 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55e5984e1e00 0x55e5984e4220 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=506f563db951bf61 server_cookie=e8b504b8e5c1c950 in_seq=0 out_seq=0 2024-03-27T06:45:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fade4ff9640 1 -- 172.21.15.16:0/2989518502 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5983ccfa0 con 0x55e5984e1e00 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fade4ff9640 10 client.16080 handle_command_reply: tid=0 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 1 client.16080 shutdown 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 1 -- 172.21.15.16:0/2989518502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 msgr2=0x55e59847e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e59847e6f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55e598469260 tx=0x7fadf80096b0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 1 -- 172.21.15.16:0/2989518502 shutdown_connections 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x55e5984e1e00 0x55e5984e4220 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fae065ef740 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e59846acb0 0x55e59847ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae065ef740 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5984620d0 0x55e59847e6f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae065ef740 1 --2- 172.21.15.16:0/2989518502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e5984617b0 0x55e59847e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.173+0000 7fade4ff9640 0 client.16080 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae065ef740 1 -- 172.21.15.16:0/2989518502 >> 172.21.15.16:0/2989518502 conn(0x55e59845bee0 msgr2=0x55e59845b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae065ef740 1 -- 172.21.15.16:0/2989518502 shutdown_connections 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae065ef740 1 -- 172.21.15.16:0/2989518502 wait complete. 2024-03-27T06:45:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae065ef740 20 client.16080 trim_cache size 0 max 16384 2024-03-27T06:45:19.175 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:45:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 -- 172.21.15.16:0/4105211330 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fade80a37b0 msgr2=0x7fade80a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fade80a37b0 0x7fade80a5c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fadf40efd70 tx=0x7fadf40e9620 comp rx=0 tx=0).stop 2024-03-27T06:45:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 -- 172.21.15.16:0/4105211330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadfc107ba0 msgr2=0x7fadfc101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadfc107ba0 0x7fadfc101500 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fadf4004af0 tx=0x7fadf4032550 comp rx=0 tx=0).stop 2024-03-27T06:45:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 -- 172.21.15.16:0/4105211330 shutdown_connections 2024-03-27T06:45:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadfc107ba0 0x7fadfc101500 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadfc06a870 0x7fadfc100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fade80a37b0 0x7fade80a5c70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 --2- 172.21.15.16:0/4105211330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadfc069eb0 0x7fadfc100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 -- 172.21.15.16:0/4105211330 >> 172.21.15.16:0/4105211330 conn(0x7fadfc077ae0 msgr2=0x7fadfc0ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 -- 172.21.15.16:0/4105211330 shutdown_connections 2024-03-27T06:45:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.174+0000 7fae03925640 1 -- 172.21.15.16:0/4105211330 wait complete. 2024-03-27T06:45:19.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-27T06:45:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.290+0000 7faa2cd8a640 1 -- 172.21.15.16:0/727878300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28107090 msgr2=0x7faa2810e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.290+0000 7faa2cd8a640 1 --2- 172.21.15.16:0/727878300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28107090 0x7faa2810e950 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faa18009ca0 tx=0x7faa1802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.290+0000 7faa2cd8a640 1 -- 172.21.15.16:0/727878300 shutdown_connections 2024-03-27T06:45:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.290+0000 7faa2cd8a640 1 --2- 172.21.15.16:0/727878300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28107090 0x7faa2810e950 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.290+0000 7faa2cd8a640 1 --2- 172.21.15.16:0/727878300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa281066d0 0x7faa28106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.290+0000 7faa2cd8a640 1 --2- 172.21.15.16:0/727878300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa28069130 0x7faa28069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.290+0000 7faa2cd8a640 1 -- 172.21.15.16:0/727878300 >> 172.21.15.16:0/727878300 conn(0x7faa28077b50 msgr2=0x7faa28077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.290+0000 7faa2cd8a640 1 -- 172.21.15.16:0/727878300 shutdown_connections 2024-03-27T06:45:19.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.290+0000 7faa2cd8a640 1 -- 172.21.15.16:0/727878300 wait complete. 2024-03-27T06:45:19.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.291+0000 7faa2cd8a640 1 Processor -- start 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.291+0000 7faa2cd8a640 1 -- start start 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa2cd8a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa28069130 0x7faa281a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa2cd8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281066d0 0x7faa281a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa2cd8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28107090 0x7faa2810da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa2cd8a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa28111ff0 con 0x7faa28107090 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa2cd8a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7faa28111e70 con 0x7faa28069130 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa26d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28107090 0x7faa2810da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa2cd8a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faa28112170 con 0x7faa281066d0 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa26d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28107090 0x7faa2810da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51872/0 (socket says 172.21.15.16:51872) 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa26d9d640 1 -- 172.21.15.16:0/1362938580 learned_addr learned my addr 172.21.15.16:0/1362938580 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa25d9b640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281066d0 0x7faa281a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa2659c640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa28069130 0x7faa281a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa26d9d640 1 -- 172.21.15.16:0/1362938580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281066d0 msgr2=0x7faa281a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa26d9d640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281066d0 0x7faa281a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa26d9d640 1 -- 172.21.15.16:0/1362938580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa28069130 msgr2=0x7faa281a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa26d9d640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa28069130 0x7faa281a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa26d9d640 1 -- 172.21.15.16:0/1362938580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2810e1f0 con 0x7faa28107090 2024-03-27T06:45:19.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa2659c640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa28069130 0x7faa281a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:45:19.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.292+0000 7faa26d9d640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28107090 0x7faa2810da70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faa1802c700 tx=0x7faa180093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.293+0000 7faa1f7fe640 1 -- 172.21.15.16:0/1362938580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18039040 con 0x7faa28107090 2024-03-27T06:45:19.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.293+0000 7faa2cd8a640 1 -- 172.21.15.16:0/1362938580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2810e480 con 0x7faa28107090 2024-03-27T06:45:19.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.293+0000 7faa1f7fe640 1 -- 172.21.15.16:0/1362938580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faa18044070 con 0x7faa28107090 2024-03-27T06:45:19.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.293+0000 7faa1f7fe640 1 -- 172.21.15.16:0/1362938580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18004440 con 0x7faa28107090 2024-03-27T06:45:19.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.293+0000 7faa2cd8a640 1 -- 172.21.15.16:0/1362938580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa281ab5a0 con 0x7faa28107090 2024-03-27T06:45:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa1f7fe640 1 -- 172.21.15.16:0/1362938580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7faa18003b00 con 0x7faa28107090 2024-03-27T06:45:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa2f972740 1 Processor -- start 2024-03-27T06:45:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa2f972740 1 -- start start 2024-03-27T06:45:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa2f972740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32dcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa2f972740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa2f972740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa2f972740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561ef3240710 con 0x561ef32dc7b0 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa2f972740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x561ef31e8af0 con 0x561ef32dd0d0 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa2f972740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561ef323c6c0 con 0x561ef32e5cb0 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32dcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32dcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51884/0 (socket says 172.21.15.16:51884) 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa25d9b640 1 -- 172.21.15.16:0/1744337059 learned_addr learned my addr 172.21.15.16:0/1744337059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa2659c640 1 --2- 172.21.15.16:0/1744337059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.296+0000 7faa25d9b640 1 --2- 172.21.15.16:0/1744337059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa1f7fe640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7faa140a3780 0x7faa140a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa25d9b640 1 -- 172.21.15.16:0/1744337059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 msgr2=0x561ef32e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa25d9b640 1 --2- 172.21.15.16:0/1744337059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32e80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa25d9b640 1 -- 172.21.15.16:0/1744337059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 msgr2=0x561ef32e5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa25d9b640 1 --2- 172.21.15.16:0/1744337059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32e5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa26d9d640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7faa140a3780 0x7faa140a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa25d9b640 1 -- 172.21.15.16:0/1744337059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef2f543f0 con 0x561ef32dc7b0 2024-03-27T06:45:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa1f7fe640 1 -- 172.21.15.16:0/1362938580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7faa180f2480 con 0x7faa28107090 2024-03-27T06:45:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa25d9b640 1 --2- 172.21.15.16:0/1744337059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32dcb90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faa10009fd0 tx=0x7faa100096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0881f1fcf497395 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa1cff9640 1 -- 172.21.15.16:0/1744337059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10004050 con 0x561ef32dc7b0 2024-03-27T06:45:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa1cff9640 1 -- 172.21.15.16:0/1744337059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faa100041f0 con 0x561ef32dc7b0 2024-03-27T06:45:19.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.297+0000 7faa1cff9640 1 -- 172.21.15.16:0/1744337059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1000f730 con 0x561ef32dc7b0 2024-03-27T06:45:19.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.298+0000 7faa2f972740 1 -- 172.21.15.16:0/1744337059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 msgr2=0x561ef32dcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.298+0000 7faa26d9d640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7faa140a3780 0x7faa140a5c40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa180efdc0 tx=0x7faa180b3be0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.298+0000 7faa2f972740 1 --2- 172.21.15.16:0/1744337059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32dcb90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faa10009fd0 tx=0x7faa100096c0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.298+0000 7faa2f972740 1 -- 172.21.15.16:0/1744337059 shutdown_connections 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.298+0000 7faa2f972740 1 --2- 172.21.15.16:0/1744337059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.298+0000 7faa2f972740 1 --2- 172.21.15.16:0/1744337059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32e5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.298+0000 7faa2f972740 1 --2- 172.21.15.16:0/1744337059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32dcb90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.298+0000 7faa2f972740 1 -- 172.21.15.16:0/1744337059 >> 172.21.15.16:0/1744337059 conn(0x561ef32d6ee0 msgr2=0x561ef32d72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.298+0000 7faa2f972740 1 -- 172.21.15.16:0/1744337059 shutdown_connections 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.298+0000 7faa2f972740 1 -- 172.21.15.16:0/1744337059 wait complete. 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa2f972740 1 -- start start 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa2f972740 1 Processor -- start 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa2f972740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32dc7b0 0x561ef32e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa2f972740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32e31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa2f972740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32e5cb0 0x561ef32e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa2f972740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561ef32e8aa0 con 0x561ef32e5cb0 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa2f972740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x561ef31fbeb0 con 0x561ef32dd0d0 2024-03-27T06:45:19.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa2f972740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561ef32e8c20 con 0x561ef32dc7b0 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa2f972740 10 client.0 _ll_register_callbacks cb 0x561ef32386a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32e5cb0 0x561ef32e36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32e5cb0 0x561ef32e36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51896/0 (socket says 172.21.15.16:51896) 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa25d9b640 1 -- 172.21.15.16:0/2037531960 learned_addr learned my addr 172.21.15.16:0/2037531960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.299+0000 7faa25d9b640 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32dc7b0 0x561ef32e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa2659c640 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32e31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa25d9b640 1 -- 172.21.15.16:0/2037531960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32dc7b0 msgr2=0x561ef32e2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa25d9b640 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32dc7b0 0x561ef32e2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa25d9b640 1 -- 172.21.15.16:0/2037531960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 msgr2=0x561ef32e31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa25d9b640 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32e31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa25d9b640 1 -- 172.21.15.16:0/2037531960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef301e7d0 con 0x561ef32e5cb0 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa2659c640 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32e31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa25d9b640 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32e5cb0 0x561ef32e36f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faa10004190 tx=0x7faa10003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c69875757001bb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa0dffb640 1 -- 172.21.15.16:0/2037531960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10004460 con 0x561ef32e5cb0 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa0dffb640 1 -- 172.21.15.16:0/2037531960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7faa10013170 con 0x561ef32e5cb0 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa2f972740 1 -- 172.21.15.16:0/2037531960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa100041f0 con 0x561ef32e5cb0 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa0dffb640 1 -- 172.21.15.16:0/2037531960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa10005020 con 0x561ef32e5cb0 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa2f972740 10 client.16092 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa2f972740 1 -- 172.21.15.16:0/2037531960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ef32f4d50 con 0x561ef32e5cb0 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa0dffb640 1 -- 172.21.15.16:0/2037531960 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faa100041f0 con 0x561ef32e5cb0 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.300+0000 7faa0dffb640 1 -- 172.21.15.16:0/2037531960 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7faa10004c90 con 0x561ef32e5cb0 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.301+0000 7faa2f972740 10 client.16092 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.301+0000 7faa2f972740 10 client.16092 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.301+0000 7faa2f972740 20 client.16092 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.301+0000 7faa2f972740 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561ef32f60a0 0x561ef32f8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.301+0000 7faa2f972740 4 client.16092 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.301+0000 7faa2f972740 1 -- 172.21.15.16:0/2037531960 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ef324dc50 con 0x561ef32f60a0 2024-03-27T06:45:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.301+0000 7faa2659c640 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561ef32f60a0 0x561ef32f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.301+0000 7faa0dffb640 10 client.16092 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.301+0000 7faa2659c640 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561ef32f60a0 0x561ef32f8490 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d75a38f6eb3e49c9 server_cookie=e29f0f1966b7e2d1 in_seq=0 out_seq=0 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa0dffb640 1 -- 172.21.15.16:0/2037531960 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561ef324dc50 con 0x561ef32f60a0 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa0dffb640 10 client.16092 handle_command_reply: tid=0 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa2f972740 1 client.16092 shutdown 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa2f972740 1 -- 172.21.15.16:0/2037531960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32e5cb0 msgr2=0x561ef32e36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa2f972740 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32e5cb0 0x561ef32e36f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faa10004190 tx=0x7faa10003ed0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa2f972740 1 -- 172.21.15.16:0/2037531960 shutdown_connections 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa2f972740 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561ef32f60a0 0x561ef32f8490 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa2f972740 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32e5cb0 0x561ef32e36f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa2f972740 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32e31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa0dffb640 0 client.16092 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa2f972740 1 --2- 172.21.15.16:0/2037531960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32dc7b0 0x561ef32e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.302+0000 7faa2f972740 1 -- 172.21.15.16:0/2037531960 >> 172.21.15.16:0/2037531960 conn(0x561ef32d6ee0 msgr2=0x561ef32d8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.303+0000 7faa2f972740 1 -- 172.21.15.16:0/2037531960 shutdown_connections 2024-03-27T06:45:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.303+0000 7faa2f972740 1 -- 172.21.15.16:0/2037531960 wait complete. 2024-03-27T06:45:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.303+0000 7faa2f972740 20 client.16092 trim_cache size 0 max 16384 2024-03-27T06:45:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa2f972740 1 Processor -- start 2024-03-27T06:45:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa2f972740 1 -- start start 2024-03-27T06:45:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa2f972740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32e53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa2f972740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa2f972740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32e1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa2f972740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa281b3eb0 con 0x561ef32dc7b0 2024-03-27T06:45:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa2f972740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7faa281b5ab0 con 0x561ef32dd0d0 2024-03-27T06:45:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa2f972740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faa281b4580 con 0x561ef32e5cb0 2024-03-27T06:45:19.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32e53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32e53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51900/0 (socket says 172.21.15.16:51900) 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa25d9b640 1 -- 172.21.15.16:0/1732630643 learned_addr learned my addr 172.21.15.16:0/1732630643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa2659c640 1 --2- 172.21.15.16:0/1732630643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa25d9b640 1 --2- 172.21.15.16:0/1732630643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32e1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa25d9b640 1 -- 172.21.15.16:0/1732630643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 msgr2=0x561ef32e1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa25d9b640 1 --2- 172.21.15.16:0/1732630643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32e1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa25d9b640 1 -- 172.21.15.16:0/1732630643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 msgr2=0x561ef32f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa25d9b640 1 --2- 172.21.15.16:0/1732630643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.317+0000 7faa25d9b640 1 -- 172.21.15.16:0/1732630643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef32d6d10 con 0x561ef32dc7b0 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.318+0000 7faa25d9b640 1 --2- 172.21.15.16:0/1732630643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32e53a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7faa10009820 tx=0x7faa10005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ea8686f8a5b5ff server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.318+0000 7faa0ffff640 1 -- 172.21.15.16:0/1732630643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10024070 con 0x561ef32dc7b0 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.318+0000 7faa0ffff640 1 -- 172.21.15.16:0/1732630643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faa10003a20 con 0x561ef32dc7b0 2024-03-27T06:45:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.318+0000 7faa0ffff640 1 -- 172.21.15.16:0/1732630643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1000f9f0 con 0x561ef32dc7b0 2024-03-27T06:45:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 -- 172.21.15.16:0/1732630643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 msgr2=0x561ef32e53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 --2- 172.21.15.16:0/1732630643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32e53a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7faa10009820 tx=0x7faa10005f90 comp rx=0 tx=0).stop 2024-03-27T06:45:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 -- 172.21.15.16:0/1732630643 shutdown_connections 2024-03-27T06:45:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 --2- 172.21.15.16:0/1732630643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32e1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 --2- 172.21.15.16:0/1732630643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dd0d0 0x561ef32f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 --2- 172.21.15.16:0/1732630643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dc7b0 0x561ef32e53a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 -- 172.21.15.16:0/1732630643 >> 172.21.15.16:0/1732630643 conn(0x561ef32d6ee0 msgr2=0x561ef32d5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 -- 172.21.15.16:0/1732630643 shutdown_connections 2024-03-27T06:45:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 -- 172.21.15.16:0/1732630643 wait complete. 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 -- start start 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 Processor -- start 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dc7b0 0x561ef32e9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dd0d0 0x561ef32e9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32ea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa281b53e0 con 0x561ef32dd0d0 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7faa281b4c50 con 0x561ef32dc7b0 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.319+0000 7faa2f972740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faa281b1e90 con 0x561ef32e5cb0 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa2f972740 10 client.0 _ll_register_callbacks cb 0x7faa20009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa2659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dd0d0 0x561ef32e9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dc7b0 0x561ef32e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa2659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dd0d0 0x561ef32e9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51910/0 (socket says 172.21.15.16:51910) 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa2659c640 1 -- 172.21.15.16:0/3587431742 learned_addr learned my addr 172.21.15.16:0/3587431742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dc7b0 0x561ef32e9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:55474/0 (socket says 172.21.15.16:55474) 2024-03-27T06:45:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa25d9b640 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32ea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa2659c640 1 -- 172.21.15.16:0/3587431742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 msgr2=0x561ef32ea1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa2659c640 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32ea1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa2659c640 1 -- 172.21.15.16:0/3587431742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dc7b0 msgr2=0x561ef32e9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa2659c640 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dc7b0 0x561ef32e9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa2659c640 1 -- 172.21.15.16:0/3587431742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef335b2d0 con 0x561ef32dd0d0 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa2659c640 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dd0d0 0x561ef32e9c60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x561ef32dc280 tx=0x7faa2001e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e90f108c13e0b14 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa0dffb640 1 -- 172.21.15.16:0/3587431742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa20014b80 con 0x561ef32dd0d0 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa2f972740 1 -- 172.21.15.16:0/3587431742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa10003a20 con 0x561ef32dd0d0 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.320+0000 7faa0dffb640 1 -- 172.21.15.16:0/3587431742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7faa20014d20 con 0x561ef32dd0d0 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa0dffb640 1 -- 172.21.15.16:0/3587431742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa20012450 con 0x561ef32dd0d0 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa2f972740 10 client.16098 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa2f972740 1 -- 172.21.15.16:0/3587431742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ef335c5c0 con 0x561ef32dd0d0 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa0dffb640 1 -- 172.21.15.16:0/3587431742 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faa10003a20 con 0x561ef32dd0d0 2024-03-27T06:45:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa0dffb640 1 -- 172.21.15.16:0/3587431742 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7faa20025070 con 0x561ef32dd0d0 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa2f972740 10 client.16098 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa2f972740 10 client.16098 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa2f972740 20 client.16098 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa2f972740 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561ef335cd70 0x561ef335f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa2f972740 4 client.16098 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa2f972740 1 -- 172.21.15.16:0/3587431742 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ef3247fa0 con 0x561ef335cd70 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa25d9b640 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561ef335cd70 0x561ef335f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa0dffb640 10 client.16098 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.321+0000 7faa25d9b640 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561ef335cd70 0x561ef335f190 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cfe2ad4e9638a9eb server_cookie=21064b9f4484de22 in_seq=0 out_seq=0 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa0dffb640 1 -- 172.21.15.16:0/3587431742 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ef3247fa0 con 0x561ef335cd70 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa0dffb640 10 client.16098 handle_command_reply: tid=0 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 1 client.16098 shutdown 2024-03-27T06:45:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 1 -- 172.21.15.16:0/3587431742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dd0d0 msgr2=0x561ef32e9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dd0d0 0x561ef32e9c60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x561ef32dc280 tx=0x7faa2001e7c0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 1 -- 172.21.15.16:0/3587431742 shutdown_connections 2024-03-27T06:45:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x561ef335cd70 0x561ef335f190 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ef32e5cb0 0x561ef32ea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ef32dd0d0 0x561ef32e9c60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa0dffb640 0 client.16098 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:45:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 1 --2- 172.21.15.16:0/3587431742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561ef32dc7b0 0x561ef32e9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 1 -- 172.21.15.16:0/3587431742 >> 172.21.15.16:0/3587431742 conn(0x561ef32d6ee0 msgr2=0x561ef32e34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 1 -- 172.21.15.16:0/3587431742 shutdown_connections 2024-03-27T06:45:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 1 -- 172.21.15.16:0/3587431742 wait complete. 2024-03-27T06:45:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2f972740 20 client.16098 trim_cache size 0 max 16384 2024-03-27T06:45:19.324 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:45:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2cd8a640 1 -- 172.21.15.16:0/1362938580 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7faa140a3780 msgr2=0x7faa140a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2cd8a640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7faa140a3780 0x7faa140a5c40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa180efdc0 tx=0x7faa180b3be0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2cd8a640 1 -- 172.21.15.16:0/1362938580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28107090 msgr2=0x7faa2810da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.322+0000 7faa2cd8a640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28107090 0x7faa2810da70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faa1802c700 tx=0x7faa180093d0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.323+0000 7faa2cd8a640 1 -- 172.21.15.16:0/1362938580 shutdown_connections 2024-03-27T06:45:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.323+0000 7faa2cd8a640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28107090 0x7faa2810da70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.323+0000 7faa2cd8a640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7faa140a3780 0x7faa140a5c40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.323+0000 7faa2cd8a640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281066d0 0x7faa281a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.323+0000 7faa2cd8a640 1 --2- 172.21.15.16:0/1362938580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa28069130 0x7faa281a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.323+0000 7faa2cd8a640 1 -- 172.21.15.16:0/1362938580 >> 172.21.15.16:0/1362938580 conn(0x7faa28077b50 msgr2=0x7faa280fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.323+0000 7faa2cd8a640 1 -- 172.21.15.16:0/1362938580 shutdown_connections 2024-03-27T06:45:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.323+0000 7faa2cd8a640 1 -- 172.21.15.16:0/1362938580 wait complete. 2024-03-27T06:45:19.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-27T06:45:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.439+0000 7f432fc76640 1 -- 172.21.15.16:0/3706206288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328107090 msgr2=0x7f432810e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.439+0000 7f432fc76640 1 --2- 172.21.15.16:0/3706206288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328107090 0x7f432810e950 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4320009ca0 tx=0x7f432002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.440+0000 7f432fc76640 1 -- 172.21.15.16:0/3706206288 shutdown_connections 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.440+0000 7f432fc76640 1 --2- 172.21.15.16:0/3706206288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328107090 0x7f432810e950 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.440+0000 7f432fc76640 1 --2- 172.21.15.16:0/3706206288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43281066d0 0x7f4328106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.440+0000 7f432fc76640 1 --2- 172.21.15.16:0/3706206288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4328069130 0x7f4328069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.440+0000 7f432fc76640 1 -- 172.21.15.16:0/3706206288 >> 172.21.15.16:0/3706206288 conn(0x7f4328077b50 msgr2=0x7f4328077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.440+0000 7f432fc76640 1 -- 172.21.15.16:0/3706206288 shutdown_connections 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.440+0000 7f432fc76640 1 -- 172.21.15.16:0/3706206288 wait complete. 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.440+0000 7f432fc76640 1 Processor -- start 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.440+0000 7f432fc76640 1 -- start start 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.440+0000 7f432fc76640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4328069130 0x7f4328198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.440+0000 7f432fc76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43281066d0 0x7f4328198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432fc76640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4328107090 0x7f432819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432fc76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4328111ff0 con 0x7f43281066d0 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432fc76640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f4328111e70 con 0x7f4328069130 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432fc76640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4328112170 con 0x7f4328107090 2024-03-27T06:45:19.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43281066d0 0x7f4328198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43281066d0 0x7f4328198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51916/0 (socket says 172.21.15.16:51916) 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432d211640 1 -- 172.21.15.16:0/3171684756 learned_addr learned my addr 172.21.15.16:0/3171684756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432da12640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4328069130 0x7f4328198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432e213640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4328107090 0x7f432819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432d211640 1 -- 172.21.15.16:0/3171684756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4328107090 msgr2=0x7f432819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432d211640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4328107090 0x7f432819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432d211640 1 -- 172.21.15.16:0/3171684756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4328069130 msgr2=0x7f4328198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432d211640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4328069130 0x7f4328198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432d211640 1 -- 172.21.15.16:0/3171684756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432819d440 con 0x7f43281066d0 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432d211640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43281066d0 0x7f4328198990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f431800e9f0 tx=0x7f431800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f431effd640 1 -- 172.21.15.16:0/3171684756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431800cd80 con 0x7f43281066d0 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432fc76640 1 -- 172.21.15.16:0/3171684756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432819d730 con 0x7f43281066d0 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f431effd640 1 -- 172.21.15.16:0/3171684756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4318005520 con 0x7f43281066d0 2024-03-27T06:45:19.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f431effd640 1 -- 172.21.15.16:0/3171684756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4318010690 con 0x7f43281066d0 2024-03-27T06:45:19.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.441+0000 7f432fc76640 1 -- 172.21.15.16:0/3171684756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43281a4fc0 con 0x7f43281066d0 2024-03-27T06:45:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.444+0000 7f4332864740 1 Processor -- start 2024-03-27T06:45:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.444+0000 7f4332864740 1 -- start start 2024-03-27T06:45:19.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.444+0000 7f4332864740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 0x5581d20d5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.444+0000 7f4332864740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.444+0000 7f4332864740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.444+0000 7f4332864740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5581d2039710 con 0x5581d20d5d80 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.444+0000 7f4332864740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5581d1fe1af0 con 0x5581d20d5460 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.444+0000 7f4332864740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5581d20356c0 con 0x5581d20e30c0 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f432da12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f432e213640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 0x5581d20d5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f432da12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51926/0 (socket says 172.21.15.16:51926) 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f432da12640 1 -- 172.21.15.16:0/3208636906 learned_addr learned my addr 172.21.15.16:0/3208636906 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f431effd640 1 -- 172.21.15.16:0/3171684756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f43180046c0 con 0x7f43281066d0 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f432da12640 1 -- 172.21.15.16:0/3208636906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 msgr2=0x5581d20e54b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f432da12640 1 --2- 172.21.15.16:0/3208636906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f432da12640 1 -- 172.21.15.16:0/3208636906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 msgr2=0x5581d20d5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f432da12640 1 --2- 172.21.15.16:0/3208636906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 0x5581d20d5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f432da12640 1 -- 172.21.15.16:0/3208636906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d1d4d3f0 con 0x5581d20d5d80 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f432e213640 1 --2- 172.21.15.16:0/3208636906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 0x5581d20d5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f432da12640 1 --2- 172.21.15.16:0/3208636906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e2b80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4324009cf0 tx=0x7f4324011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd5c54443bdd607 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f4313fff640 1 -- 172.21.15.16:0/3208636906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432401e070 con 0x5581d20d5d80 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f4313fff640 1 -- 172.21.15.16:0/3208636906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4324019b30 con 0x5581d20d5d80 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f4313fff640 1 -- 172.21.15.16:0/3208636906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432400eac0 con 0x5581d20d5d80 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.445+0000 7f431effd640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f43140a3780 0x7f43140a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f432e213640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f43140a3780 0x7f43140a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f431effd640 1 -- 172.21.15.16:0/3171684756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f43180d0450 con 0x7f43281066d0 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f432e213640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f43140a3780 0x7f43140a5c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4320004af0 tx=0x7f4320005c50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f4332864740 1 -- 172.21.15.16:0/3208636906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 msgr2=0x5581d20e2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f4332864740 1 --2- 172.21.15.16:0/3208636906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e2b80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4324009cf0 tx=0x7f4324011810 comp rx=0 tx=0).stop 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f4332864740 1 -- 172.21.15.16:0/3208636906 shutdown_connections 2024-03-27T06:45:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f4332864740 1 --2- 172.21.15.16:0/3208636906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20e54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f4332864740 1 --2- 172.21.15.16:0/3208636906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e2b80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f4332864740 1 --2- 172.21.15.16:0/3208636906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 0x5581d20d5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f4332864740 1 -- 172.21.15.16:0/3208636906 >> 172.21.15.16:0/3208636906 conn(0x5581d20cfee0 msgr2=0x5581d20d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f4332864740 1 -- 172.21.15.16:0/3208636906 shutdown_connections 2024-03-27T06:45:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.446+0000 7f4332864740 1 -- 172.21.15.16:0/3208636906 wait complete. 2024-03-27T06:45:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.447+0000 7f4332864740 1 -- start start 2024-03-27T06:45:19.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.447+0000 7f4332864740 1 Processor -- start 2024-03-27T06:45:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.447+0000 7f4332864740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 0x5581d20e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.447+0000 7f4332864740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.447+0000 7f4332864740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.447+0000 7f4332864740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5581d20e5f00 con 0x5581d20d5d80 2024-03-27T06:45:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.447+0000 7f4332864740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5581d1ff4eb0 con 0x5581d20d5460 2024-03-27T06:45:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.447+0000 7f4332864740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5581d20e6080 con 0x5581d20e30c0 2024-03-27T06:45:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f4332864740 10 client.0 _ll_register_callbacks cb 0x5581d20316a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432da12640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 0x5581d20e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432e213640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20e0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51934/0 (socket says 172.21.15.16:51934) 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432d211640 1 -- 172.21.15.16:0/1355862365 learned_addr learned my addr 172.21.15.16:0/1355862365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432da12640 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 0x5581d20e00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:55508/0 (socket says 172.21.15.16:55508) 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432d211640 1 -- 172.21.15.16:0/1355862365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 msgr2=0x5581d20e0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432d211640 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20e0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432d211640 1 -- 172.21.15.16:0/1355862365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 msgr2=0x5581d20e00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432d211640 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 0x5581d20e00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432d211640 1 -- 172.21.15.16:0/1355862365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d1e177d0 con 0x5581d20d5d80 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432da12640 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 0x5581d20e00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f432d211640 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e05e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f431809c310 tx=0x7f4318098970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d4bd3cae56fabb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f43117fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f43117fa640 1 -- 172.21.15.16:0/1355862365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180b5060 con 0x5581d20d5d80 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f43117fa640 1 -- 172.21.15.16:0/1355862365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f43180a4ff0 con 0x5581d20d5d80 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f4332864740 1 -- 172.21.15.16:0/1355862365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4324019b30 con 0x5581d20d5d80 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f43117fa640 1 -- 172.21.15.16:0/1355862365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f431809cd10 con 0x5581d20d5d80 2024-03-27T06:45:19.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f4332864740 10 client.16110 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.448+0000 7f4332864740 1 -- 172.21.15.16:0/1355862365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581d20f21f0 con 0x5581d20d5d80 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.449+0000 7f43117fa640 1 -- 172.21.15.16:0/1355862365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4324019b30 con 0x5581d20d5d80 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.449+0000 7f43117fa640 1 -- 172.21.15.16:0/1355862365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f43180b5ba0 con 0x5581d20d5d80 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.449+0000 7f4332864740 10 client.16110 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.449+0000 7f4332864740 10 client.16110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.449+0000 7f4332864740 20 client.16110 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.449+0000 7f4332864740 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5581d20f3600 0x5581d20f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.449+0000 7f4332864740 4 client.16110 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.449+0000 7f4332864740 1 -- 172.21.15.16:0/1355862365 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581d2046c50 con 0x5581d20f3600 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.449+0000 7f432da12640 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5581d20f3600 0x5581d20f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.449+0000 7f43117fa640 10 client.16110 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.449+0000 7f432da12640 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5581d20f3600 0x5581d20f5a10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=602a60267b74e7de server_cookie=5dce195e149017a3 in_seq=0 out_seq=0 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f43117fa640 1 -- 172.21.15.16:0/1355862365 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5581d2046c50 con 0x5581d20f3600 2024-03-27T06:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f43117fa640 10 client.16110 handle_command_reply: tid=0 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f4332864740 1 client.16110 shutdown 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f4332864740 1 -- 172.21.15.16:0/1355862365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 msgr2=0x5581d20e05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f4332864740 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e05e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f431809c310 tx=0x7f4318098970 comp rx=0 tx=0).stop 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f4332864740 1 -- 172.21.15.16:0/1355862365 shutdown_connections 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f4332864740 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5581d20f3600 0x5581d20f5a10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f4332864740 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20e0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f4332864740 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5d80 0x5581d20e05e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f43117fa640 0 client.16110 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f4332864740 1 --2- 172.21.15.16:0/1355862365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5460 0x5581d20e00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f4332864740 1 -- 172.21.15.16:0/1355862365 >> 172.21.15.16:0/1355862365 conn(0x5581d20cfee0 msgr2=0x5581d20d1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.450+0000 7f4332864740 1 -- 172.21.15.16:0/1355862365 shutdown_connections 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.451+0000 7f4332864740 1 -- 172.21.15.16:0/1355862365 wait complete. 2024-03-27T06:45:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.451+0000 7f4332864740 20 client.16110 trim_cache size 0 max 16384 2024-03-27T06:45:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.466+0000 7f4332864740 1 Processor -- start 2024-03-27T06:45:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.466+0000 7f4332864740 1 -- start start 2024-03-27T06:45:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.466+0000 7f4332864740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20d5460 0x5581d20e27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.466+0000 7f4332864740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5d80 0x5581d20eed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.466+0000 7f4332864740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20e30c0 0x5581d20fb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.466+0000 7f4332864740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43281ab670 con 0x5581d20e30c0 2024-03-27T06:45:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.466+0000 7f4332864740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f43281ad270 con 0x5581d20d5d80 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.466+0000 7f4332864740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f43281abd40 con 0x5581d20d5460 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.466+0000 7f432e213640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20e30c0 0x5581d20fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f432d211640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5d80 0x5581d20eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f432e213640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20e30c0 0x5581d20fb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51950/0 (socket says 172.21.15.16:51950) 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f432e213640 1 -- 172.21.15.16:0/3964869429 learned_addr learned my addr 172.21.15.16:0/3964869429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f432da12640 1 --2- 172.21.15.16:0/3964869429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20d5460 0x5581d20e27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f432e213640 1 -- 172.21.15.16:0/3964869429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20d5460 msgr2=0x5581d20e27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f432e213640 1 --2- 172.21.15.16:0/3964869429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20d5460 0x5581d20e27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f432e213640 1 -- 172.21.15.16:0/3964869429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5d80 msgr2=0x5581d20eed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f432e213640 1 --2- 172.21.15.16:0/3964869429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5d80 0x5581d20eed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f432e213640 1 -- 172.21.15.16:0/3964869429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d20e54b0 con 0x5581d20e30c0 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f432d211640 1 --2- 172.21.15.16:0/3964869429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5d80 0x5581d20eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f432e213640 1 --2- 172.21.15.16:0/3964869429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20e30c0 0x5581d20fb240 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4320009ca0 tx=0x7f4320041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e867c1fa4a1ed server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f43137fe640 1 -- 172.21.15.16:0/3964869429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320039040 con 0x5581d20e30c0 2024-03-27T06:45:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f43137fe640 1 -- 172.21.15.16:0/3964869429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4320004040 con 0x5581d20e30c0 2024-03-27T06:45:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.467+0000 7f43137fe640 1 -- 172.21.15.16:0/3964869429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432004d780 con 0x5581d20e30c0 2024-03-27T06:45:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.468+0000 7f4332864740 1 -- 172.21.15.16:0/3964869429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20e30c0 msgr2=0x5581d20fb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.468+0000 7f4332864740 1 --2- 172.21.15.16:0/3964869429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20e30c0 0x5581d20fb240 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4320009ca0 tx=0x7f4320041000 comp rx=0 tx=0).stop 2024-03-27T06:45:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.468+0000 7f4332864740 1 -- 172.21.15.16:0/3964869429 shutdown_connections 2024-03-27T06:45:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.468+0000 7f4332864740 1 --2- 172.21.15.16:0/3964869429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20e30c0 0x5581d20fb240 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.468+0000 7f4332864740 1 --2- 172.21.15.16:0/3964869429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5d80 0x5581d20eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.468+0000 7f4332864740 1 --2- 172.21.15.16:0/3964869429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20d5460 0x5581d20e27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.468+0000 7f4332864740 1 -- 172.21.15.16:0/3964869429 >> 172.21.15.16:0/3964869429 conn(0x5581d20cfee0 msgr2=0x5581d20e0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.468+0000 7f4332864740 1 -- 172.21.15.16:0/3964869429 shutdown_connections 2024-03-27T06:45:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.468+0000 7f4332864740 1 -- 172.21.15.16:0/3964869429 wait complete. 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.468+0000 7f4332864740 1 -- start start 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.468+0000 7f4332864740 1 Processor -- start 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f4332864740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5460 0x5581d20dcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f4332864740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5d80 0x5581d20dd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f4332864740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20dd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f4332864740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43281acba0 con 0x5581d20d5460 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f4332864740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f43281ac410 con 0x5581d20d5d80 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f4332864740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f43281a9630 con 0x5581d20e30c0 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f4332864740 10 client.0 _ll_register_callbacks cb 0x7f431800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432da12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5460 0x5581d20dcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432da12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5460 0x5581d20dcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51960/0 (socket says 172.21.15.16:51960) 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432da12640 1 -- 172.21.15.16:0/3875626443 learned_addr learned my addr 172.21.15.16:0/3875626443 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432e213640 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20dd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432d211640 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5d80 0x5581d20dd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432da12640 1 -- 172.21.15.16:0/3875626443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 msgr2=0x5581d20dd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432da12640 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20dd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432da12640 1 -- 172.21.15.16:0/3875626443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5d80 msgr2=0x5581d20dd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432da12640 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5d80 0x5581d20dd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432da12640 1 -- 172.21.15.16:0/3875626443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d2158940 con 0x5581d20d5460 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432e213640 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20dd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f432da12640 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5460 0x5581d20dcc90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f432400e770 tx=0x7f4324005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558ef37e61ed7c6d server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.469+0000 7f43117fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f43117fa640 1 -- 172.21.15.16:0/3875626443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432401e040 con 0x5581d20d5460 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f43117fa640 1 -- 172.21.15.16:0/3875626443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f43240052c0 con 0x5581d20d5460 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f4332864740 1 -- 172.21.15.16:0/3875626443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4320004040 con 0x5581d20d5460 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f43117fa640 1 -- 172.21.15.16:0/3875626443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4324038050 con 0x5581d20d5460 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f4332864740 10 client.16116 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f4332864740 1 -- 172.21.15.16:0/3875626443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581d2159c40 con 0x5581d20d5460 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f43117fa640 1 -- 172.21.15.16:0/3875626443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4320004040 con 0x5581d20d5460 2024-03-27T06:45:19.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f43117fa640 1 -- 172.21.15.16:0/3875626443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f432400e810 con 0x5581d20d5460 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f4332864740 10 client.16116 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f4332864740 10 client.16116 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f4332864740 20 client.16116 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f4332864740 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5581d215a250 0x5581d215c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f4332864740 4 client.16116 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f4332864740 1 -- 172.21.15.16:0/3875626443 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581d2040fa0 con 0x5581d215a250 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.470+0000 7f432d211640 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5581d215a250 0x5581d215c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f43117fa640 10 client.16116 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f432d211640 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5581d215a250 0x5581d215c670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c90bc70f2447960c server_cookie=179d0728ae0e66ba in_seq=0 out_seq=0 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f43117fa640 1 -- 172.21.15.16:0/3875626443 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581d2040fa0 con 0x5581d215a250 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f43117fa640 10 client.16116 handle_command_reply: tid=0 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 1 client.16116 shutdown 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 1 -- 172.21.15.16:0/3875626443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5460 msgr2=0x5581d20dcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5460 0x5581d20dcc90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f432400e770 tx=0x7f4324005f10 comp rx=0 tx=0).stop 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 1 -- 172.21.15.16:0/3875626443 shutdown_connections 2024-03-27T06:45:19.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x5581d215a250 0x5581d215c670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581d20e30c0 0x5581d20dd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5581d20d5d80 0x5581d20dd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f43117fa640 0 client.16116 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:45:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 1 --2- 172.21.15.16:0/3875626443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581d20d5460 0x5581d20dcc90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 1 -- 172.21.15.16:0/3875626443 >> 172.21.15.16:0/3875626443 conn(0x5581d20cfee0 msgr2=0x5581d20d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 1 -- 172.21.15.16:0/3875626443 shutdown_connections 2024-03-27T06:45:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 1 -- 172.21.15.16:0/3875626443 wait complete. 2024-03-27T06:45:19.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.471+0000 7f4332864740 20 client.16116 trim_cache size 0 max 16384 2024-03-27T06:45:19.472 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:45:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 -- 172.21.15.16:0/3171684756 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f43140a3780 msgr2=0x7f43140a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f43140a3780 0x7f43140a5c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4320004af0 tx=0x7f4320005c50 comp rx=0 tx=0).stop 2024-03-27T06:45:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 -- 172.21.15.16:0/3171684756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43281066d0 msgr2=0x7f4328198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43281066d0 0x7f4328198990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f431800e9f0 tx=0x7f431800edf0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 -- 172.21.15.16:0/3171684756 shutdown_connections 2024-03-27T06:45:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f43140a3780 0x7f43140a5c40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4328107090 0x7f432819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43281066d0 0x7f4328198990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 --2- 172.21.15.16:0/3171684756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4328069130 0x7f4328198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 -- 172.21.15.16:0/3171684756 >> 172.21.15.16:0/3171684756 conn(0x7f4328077b50 msgr2=0x7f43280fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 -- 172.21.15.16:0/3171684756 shutdown_connections 2024-03-27T06:45:19.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.472+0000 7f432fc76640 1 -- 172.21.15.16:0/3171684756 wait complete. 2024-03-27T06:45:19.478 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-27T06:45:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.589+0000 7fbf4b24e640 1 -- 172.21.15.16:0/8538146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf441069d0 msgr2=0x7fbf44106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.589+0000 7fbf4b24e640 1 --2- 172.21.15.16:0/8538146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf441069d0 0x7fbf44106db0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009930 tx=0x7fbf3802bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.589+0000 7fbf4b24e640 1 -- 172.21.15.16:0/8538146 shutdown_connections 2024-03-27T06:45:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.589+0000 7fbf4b24e640 1 --2- 172.21.15.16:0/8538146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf44101380 0x7fbf4410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.589+0000 7fbf4b24e640 1 --2- 172.21.15.16:0/8538146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf441009c0 0x7fbf44100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.589+0000 7fbf4b24e640 1 --2- 172.21.15.16:0/8538146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf441069d0 0x7fbf44106db0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.589+0000 7fbf4b24e640 1 -- 172.21.15.16:0/8538146 >> 172.21.15.16:0/8538146 conn(0x7fbf44077b50 msgr2=0x7fbf44077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.589+0000 7fbf4b24e640 1 -- 172.21.15.16:0/8538146 shutdown_connections 2024-03-27T06:45:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.589+0000 7fbf4b24e640 1 -- 172.21.15.16:0/8538146 wait complete. 2024-03-27T06:45:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf4b24e640 1 Processor -- start 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf4b24e640 1 -- start start 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf4b24e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf441009c0 0x7fbf44198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf4b24e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf44101380 0x7fbf44198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf4b24e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf441069d0 0x7fbf4419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf4b24e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44110eb0 con 0x7fbf441009c0 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf4b24e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44110d30 con 0x7fbf44101380 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf4b24e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44111030 con 0x7fbf441069d0 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf48fea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf441009c0 0x7fbf44198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf48fea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf441009c0 0x7fbf44198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51988/0 (socket says 172.21.15.16:51988) 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf48fea640 1 -- 172.21.15.16:0/2154409318 learned_addr learned my addr 172.21.15.16:0/2154409318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf497eb640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf441069d0 0x7fbf4419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf43fff640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf44101380 0x7fbf44198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf48fea640 1 -- 172.21.15.16:0/2154409318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf441069d0 msgr2=0x7fbf4419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf48fea640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf441069d0 0x7fbf4419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf48fea640 1 -- 172.21.15.16:0/2154409318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf44101380 msgr2=0x7fbf44198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf48fea640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf44101380 0x7fbf44198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.590+0000 7fbf48fea640 1 -- 172.21.15.16:0/2154409318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4419d550 con 0x7fbf441009c0 2024-03-27T06:45:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.591+0000 7fbf48fea640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf441009c0 0x7fbf44198600 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009230 tx=0x7fbf3802dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.591+0000 7fbf41ffb640 1 -- 172.21.15.16:0/2154409318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38043020 con 0x7fbf441009c0 2024-03-27T06:45:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.591+0000 7fbf41ffb640 1 -- 172.21.15.16:0/2154409318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf38004250 con 0x7fbf441009c0 2024-03-27T06:45:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.591+0000 7fbf4b24e640 1 -- 172.21.15.16:0/2154409318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4419d7e0 con 0x7fbf441009c0 2024-03-27T06:45:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.591+0000 7fbf41ffb640 1 -- 172.21.15.16:0/2154409318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38003970 con 0x7fbf441009c0 2024-03-27T06:45:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.591+0000 7fbf4b24e640 1 -- 172.21.15.16:0/2154409318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf441a50d0 con 0x7fbf441009c0 2024-03-27T06:45:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.593+0000 7fbf41ffb640 1 -- 172.21.15.16:0/2154409318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fbf38046020 con 0x7fbf441009c0 2024-03-27T06:45:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.594+0000 7fbf41ffb640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbf340a3800 0x7fbf340a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.594+0000 7fbf43fff640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbf340a3800 0x7fbf340a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.594+0000 7fbf41ffb640 1 -- 172.21.15.16:0/2154409318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fbf380f1f90 con 0x7fbf441009c0 2024-03-27T06:45:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.594+0000 7fbf4de35740 1 Processor -- start 2024-03-27T06:45:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.594+0000 7fbf43fff640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbf340a3800 0x7fbf340a5cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbf44100690 tx=0x7fbf30005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.594+0000 7fbf4de35740 1 -- start start 2024-03-27T06:45:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47fecb0 0x558af48010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf4de35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558af4759710 con 0x558af47f57b0 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf4de35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x558af4701af0 con 0x558af47fecb0 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf4de35740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558af47556c0 con 0x558af47f60d0 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf48fea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf48fea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47f5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51994/0 (socket says 172.21.15.16:51994) 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf48fea640 1 -- 172.21.15.16:0/1467974397 learned_addr learned my addr 172.21.15.16:0/1467974397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf43fff640 1 --2- 172.21.15.16:0/1467974397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf497eb640 1 --2- 172.21.15.16:0/1467974397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47fecb0 0x558af48010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf43fff640 1 -- 172.21.15.16:0/1467974397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47fecb0 msgr2=0x558af48010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf43fff640 1 --2- 172.21.15.16:0/1467974397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47fecb0 0x558af48010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf43fff640 1 -- 172.21.15.16:0/1467974397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 msgr2=0x558af47f5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf43fff640 1 --2- 172.21.15.16:0/1467974397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47f5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf43fff640 1 -- 172.21.15.16:0/1467974397 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af446d3f0 con 0x558af47f60d0 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf48fea640 1 --2- 172.21.15.16:0/1467974397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.595+0000 7fbf43fff640 1 --2- 172.21.15.16:0/1467974397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fe770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf300118d0 tx=0x7fbf30014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25e94042d14f44e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.596+0000 7fbf2effd640 1 -- 172.21.15.16:0/1467974397 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30004030 con 0x558af47f60d0 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.596+0000 7fbf2effd640 1 -- 172.21.15.16:0/1467974397 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf300041d0 con 0x558af47f60d0 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.596+0000 7fbf2effd640 1 -- 172.21.15.16:0/1467974397 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf300267f0 con 0x558af47f60d0 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.596+0000 7fbf4de35740 1 -- 172.21.15.16:0/1467974397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 msgr2=0x558af47fe770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.597+0000 7fbf4de35740 1 --2- 172.21.15.16:0/1467974397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fe770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf300118d0 tx=0x7fbf30014000 comp rx=0 tx=0).stop 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.597+0000 7fbf4de35740 1 -- 172.21.15.16:0/1467974397 shutdown_connections 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.597+0000 7fbf4de35740 1 --2- 172.21.15.16:0/1467974397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47fecb0 0x558af48010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.597+0000 7fbf4de35740 1 --2- 172.21.15.16:0/1467974397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fe770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.597+0000 7fbf4de35740 1 --2- 172.21.15.16:0/1467974397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.597+0000 7fbf4de35740 1 -- 172.21.15.16:0/1467974397 >> 172.21.15.16:0/1467974397 conn(0x558af47efee0 msgr2=0x558af47f02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.597+0000 7fbf4de35740 1 -- 172.21.15.16:0/1467974397 shutdown_connections 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.597+0000 7fbf4de35740 1 -- 172.21.15.16:0/1467974397 wait complete. 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.597+0000 7fbf4de35740 1 -- start start 2024-03-27T06:45:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.597+0000 7fbf4de35740 1 Processor -- start 2024-03-27T06:45:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47fecb0 0x558af47fc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf4de35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558af4801aa0 con 0x558af47f57b0 2024-03-27T06:45:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf4de35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x558af4714eb0 con 0x558af47fecb0 2024-03-27T06:45:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf4de35740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558af4801c20 con 0x558af47f60d0 2024-03-27T06:45:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf4de35740 10 client.0 _ll_register_callbacks cb 0x558af47516a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf48fea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf48fea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51996/0 (socket says 172.21.15.16:51996) 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf48fea640 1 -- 172.21.15.16:0/1472342035 learned_addr learned my addr 172.21.15.16:0/1472342035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf497eb640 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47fecb0 0x558af47fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf43fff640 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf48fea640 1 -- 172.21.15.16:0/1472342035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 msgr2=0x558af47fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf48fea640 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf48fea640 1 -- 172.21.15.16:0/1472342035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47fecb0 msgr2=0x558af47fc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf48fea640 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47fecb0 0x558af47fc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf48fea640 1 -- 172.21.15.16:0/1472342035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af45377d0 con 0x558af47f57b0 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf48fea640 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47fbce0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009a50 tx=0x7fbf3802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95919c78bf2b8af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf23fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf23fff640 1 -- 172.21.15.16:0/1472342035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf380d7020 con 0x558af47f57b0 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf23fff640 1 -- 172.21.15.16:0/1472342035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fbf380de040 con 0x558af47f57b0 2024-03-27T06:45:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.598+0000 7fbf4de35740 1 -- 172.21.15.16:0/1472342035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf300041d0 con 0x558af47f57b0 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf23fff640 1 -- 172.21.15.16:0/1472342035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf380c25b0 con 0x558af47f57b0 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf4de35740 10 client.16128 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf4de35740 1 -- 172.21.15.16:0/1472342035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558af480dd70 con 0x558af47f57b0 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf23fff640 1 -- 172.21.15.16:0/1472342035 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf300041d0 con 0x558af47f57b0 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf23fff640 1 -- 172.21.15.16:0/1472342035 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fbf380c69e0 con 0x558af47f57b0 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf4de35740 10 client.16128 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf4de35740 10 client.16128 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf4de35740 20 client.16128 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf4de35740 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x558af480f1d0 0x558af48115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf4de35740 4 client.16128 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf4de35740 1 -- 172.21.15.16:0/1472342035 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558af4766c50 con 0x558af480f1d0 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.599+0000 7fbf43fff640 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x558af480f1d0 0x558af48115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf23fff640 10 client.16128 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf43fff640 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x558af480f1d0 0x558af48115c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b56d02b139f2f3e server_cookie=ee9f81d899921fa in_seq=0 out_seq=0 2024-03-27T06:45:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf23fff640 1 -- 172.21.15.16:0/1472342035 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558af4766c50 con 0x558af480f1d0 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf23fff640 10 client.16128 handle_command_reply: tid=0 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf4de35740 1 client.16128 shutdown 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf4de35740 1 -- 172.21.15.16:0/1472342035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 msgr2=0x558af47fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf4de35740 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47fbce0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009a50 tx=0x7fbf3802bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf4de35740 1 -- 172.21.15.16:0/1472342035 shutdown_connections 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf4de35740 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x558af480f1d0 0x558af48115c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf4de35740 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47fecb0 0x558af47fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf4de35740 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf23fff640 0 client.16128 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf4de35740 1 --2- 172.21.15.16:0/1472342035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47f57b0 0x558af47fbce0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.600+0000 7fbf4de35740 1 -- 172.21.15.16:0/1472342035 >> 172.21.15.16:0/1472342035 conn(0x558af47efee0 msgr2=0x558af47f1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.601+0000 7fbf4de35740 1 -- 172.21.15.16:0/1472342035 shutdown_connections 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.601+0000 7fbf4de35740 1 -- 172.21.15.16:0/1472342035 wait complete. 2024-03-27T06:45:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.601+0000 7fbf4de35740 20 client.16128 trim_cache size 0 max 16384 2024-03-27T06:45:19.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:19 smithi016 ceph-mon[31908]: pgmap v56: 97 pgs: 97 active+clean; 88 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 8.7 MiB/s wr, 43 op/s 2024-03-27T06:45:19.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:19 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/613648781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:45:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf4de35740 1 Processor -- start 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf4de35740 1 -- start start 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47f57b0 0x558af480a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4816e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf4de35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441ad8b0 con 0x558af47fecb0 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf4de35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441af4b0 con 0x558af47f57b0 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf4de35740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441adf80 con 0x558af47f60d0 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf497eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4816e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf43fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf497eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4816e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52000/0 (socket says 172.21.15.16:52000) 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf497eb640 1 -- 172.21.15.16:0/2585213740 learned_addr learned my addr 172.21.15.16:0/2585213740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.625+0000 7fbf48fea640 1 --2- 172.21.15.16:0/2585213740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47f57b0 0x558af480a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.626+0000 7fbf497eb640 1 -- 172.21.15.16:0/2585213740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 msgr2=0x558af47fc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.626+0000 7fbf497eb640 1 --2- 172.21.15.16:0/2585213740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.626+0000 7fbf497eb640 1 -- 172.21.15.16:0/2585213740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47f57b0 msgr2=0x558af480a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.626+0000 7fbf497eb640 1 --2- 172.21.15.16:0/2585213740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47f57b0 0x558af480a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.626+0000 7fbf497eb640 1 -- 172.21.15.16:0/2585213740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af47efd10 con 0x558af47fecb0 2024-03-27T06:45:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.626+0000 7fbf43fff640 1 --2- 172.21.15.16:0/2585213740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.626+0000 7fbf497eb640 1 --2- 172.21.15.16:0/2585213740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4816e90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c003cc0 tx=0x7fbf3c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a23976344c209ba server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.626+0000 7fbf2e7fc640 1 -- 172.21.15.16:0/2585213740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c004b20 con 0x558af47fecb0 2024-03-27T06:45:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.626+0000 7fbf2e7fc640 1 -- 172.21.15.16:0/2585213740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf3c004cc0 con 0x558af47fecb0 2024-03-27T06:45:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.626+0000 7fbf2e7fc640 1 -- 172.21.15.16:0/2585213740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c0107b0 con 0x558af47fecb0 2024-03-27T06:45:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.627+0000 7fbf4de35740 1 -- 172.21.15.16:0/2585213740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 msgr2=0x558af4816e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.627+0000 7fbf4de35740 1 --2- 172.21.15.16:0/2585213740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4816e90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c003cc0 tx=0x7fbf3c004410 comp rx=0 tx=0).stop 2024-03-27T06:45:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.627+0000 7fbf4de35740 1 -- 172.21.15.16:0/2585213740 shutdown_connections 2024-03-27T06:45:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.627+0000 7fbf4de35740 1 --2- 172.21.15.16:0/2585213740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4816e90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.627+0000 7fbf4de35740 1 --2- 172.21.15.16:0/2585213740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af47fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.627+0000 7fbf4de35740 1 --2- 172.21.15.16:0/2585213740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47f57b0 0x558af480a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.627+0000 7fbf4de35740 1 -- 172.21.15.16:0/2585213740 >> 172.21.15.16:0/2585213740 conn(0x558af47efee0 msgr2=0x558af47ee770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.627+0000 7fbf4de35740 1 -- 172.21.15.16:0/2585213740 shutdown_connections 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.627+0000 7fbf4de35740 1 -- 172.21.15.16:0/2585213740 wait complete. 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.627+0000 7fbf4de35740 1 -- start start 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.627+0000 7fbf4de35740 1 Processor -- start 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47f57b0 0x558af4807150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af48024c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf4de35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4802a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf4de35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441aede0 con 0x558af47fecb0 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf4de35740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441ae650 con 0x558af47f57b0 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf4de35740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441ab890 con 0x558af47f60d0 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf4de35740 10 client.0 _ll_register_callbacks cb 0x7fbf380ef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf497eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4802a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf48fea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47f57b0 0x558af4807150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf497eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4802a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52002/0 (socket says 172.21.15.16:52002) 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf497eb640 1 -- 172.21.15.16:0/3578759174 learned_addr learned my addr 172.21.15.16:0/3578759174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf43fff640 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af48024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf48fea640 1 -- 172.21.15.16:0/3578759174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 msgr2=0x558af48024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf48fea640 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af48024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf48fea640 1 -- 172.21.15.16:0/3578759174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 msgr2=0x558af4802a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf48fea640 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4802a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf48fea640 1 -- 172.21.15.16:0/3578759174 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af48010a0 con 0x558af47f57b0 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf497eb640 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4802a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf43fff640 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af48024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf48fea640 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47f57b0 0x558af4807150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005940 tx=0x7fbf380ba5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f60b8049032576 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf23fff640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf23fff640 1 -- 172.21.15.16:0/3578759174 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf380a7040 con 0x558af47f57b0 2024-03-27T06:45:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.628+0000 7fbf4de35740 1 -- 172.21.15.16:0/3578759174 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf3c004cc0 con 0x558af47f57b0 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf23fff640 1 -- 172.21.15.16:0/3578759174 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fbf380efe00 con 0x558af47f57b0 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf23fff640 1 -- 172.21.15.16:0/3578759174 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf380d7040 con 0x558af47f57b0 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf4de35740 10 client.25958 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf4de35740 1 -- 172.21.15.16:0/3578759174 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558af4875310 con 0x558af47f57b0 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf23fff640 1 -- 172.21.15.16:0/3578759174 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf3c004cc0 con 0x558af47f57b0 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf23fff640 1 -- 172.21.15.16:0/3578759174 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fbf380cc020 con 0x558af47f57b0 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf4de35740 10 client.25958 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf4de35740 10 client.25958 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf4de35740 20 client.25958 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf4de35740 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x558af4875850 0x558af4877c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf4de35740 4 client.25958 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf4de35740 1 -- 172.21.15.16:0/3578759174 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558af47f78e0 con 0x558af4875850 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.629+0000 7fbf43fff640 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x558af4875850 0x558af4877c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf23fff640 10 client.25958 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:45:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf43fff640 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x558af4875850 0x558af4877c40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5a40945922c99cf server_cookie=7fbe5a2555290a9e in_seq=0 out_seq=0 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf23fff640 1 -- 172.21.15.16:0/3578759174 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558af47f78e0 con 0x558af4875850 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf23fff640 10 client.25958 handle_command_reply: tid=0 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 1 client.25958 shutdown 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 1 -- 172.21.15.16:0/3578759174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47f57b0 msgr2=0x558af4807150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47f57b0 0x558af4807150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005940 tx=0x7fbf380ba5d0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 1 -- 172.21.15.16:0/3578759174 shutdown_connections 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x558af4875850 0x558af4877c40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558af47fecb0 0x558af4802a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558af47f60d0 0x558af48024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf23fff640 0 client.25958 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 1 --2- 172.21.15.16:0/3578759174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af47f57b0 0x558af4807150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 1 -- 172.21.15.16:0/3578759174 >> 172.21.15.16:0/3578759174 conn(0x558af47efee0 msgr2=0x558af47f1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 1 -- 172.21.15.16:0/3578759174 shutdown_connections 2024-03-27T06:45:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 1 -- 172.21.15.16:0/3578759174 wait complete. 2024-03-27T06:45:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.630+0000 7fbf4de35740 20 client.25958 trim_cache size 0 max 16384 2024-03-27T06:45:19.631 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 -- 172.21.15.16:0/2154409318 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbf340a3800 msgr2=0x7fbf340a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbf340a3800 0x7fbf340a5cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbf44100690 tx=0x7fbf30005f90 comp rx=0 tx=0).stop 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 -- 172.21.15.16:0/2154409318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf441009c0 msgr2=0x7fbf44198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf441009c0 0x7fbf44198600 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009230 tx=0x7fbf3802dc90 comp rx=0 tx=0).stop 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 -- 172.21.15.16:0/2154409318 shutdown_connections 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbf340a3800 0x7fbf340a5cc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf441069d0 0x7fbf4419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf44101380 0x7fbf44198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 --2- 172.21.15.16:0/2154409318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf441009c0 0x7fbf44198600 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 -- 172.21.15.16:0/2154409318 >> 172.21.15.16:0/2154409318 conn(0x7fbf44077b50 msgr2=0x7fbf44069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 -- 172.21.15.16:0/2154409318 shutdown_connections 2024-03-27T06:45:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.631+0000 7fbf4b24e640 1 -- 172.21.15.16:0/2154409318 wait complete. 2024-03-27T06:45:19.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-27T06:45:19.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:19 smithi156 ceph-mon[36261]: pgmap v56: 97 pgs: 97 active+clean; 88 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 8.7 MiB/s wr, 43 op/s 2024-03-27T06:45:19.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:19 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/613648781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:45:19.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.748+0000 7f5a439f7640 1 -- 172.21.15.16:0/1294612808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a3c107ad0 msgr2=0x7f5a3c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.748+0000 7f5a439f7640 1 --2- 172.21.15.16:0/1294612808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a3c107ad0 0x7f5a3c107eb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009930 tx=0x7f5a2c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:19.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.748+0000 7f5a439f7640 1 -- 172.21.15.16:0/1294612808 shutdown_connections 2024-03-27T06:45:19.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.748+0000 7f5a439f7640 1 --2- 172.21.15.16:0/1294612808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c0696f0 0x7f5a3c10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.748+0000 7f5a439f7640 1 --2- 172.21.15.16:0/1294612808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a3c068d30 0x7f5a3c0691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.748+0000 7f5a439f7640 1 --2- 172.21.15.16:0/1294612808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a3c107ad0 0x7f5a3c107eb0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.748+0000 7f5a439f7640 1 -- 172.21.15.16:0/1294612808 >> 172.21.15.16:0/1294612808 conn(0x7f5a3c077b50 msgr2=0x7f5a3c077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.748+0000 7f5a439f7640 1 -- 172.21.15.16:0/1294612808 shutdown_connections 2024-03-27T06:45:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.748+0000 7f5a439f7640 1 -- 172.21.15.16:0/1294612808 wait complete. 2024-03-27T06:45:19.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a439f7640 1 Processor -- start 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a439f7640 1 -- start start 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a439f7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c068d30 0x7f5a3c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a439f7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a3c0696f0 0x7f5a3c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a439f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a3c107ad0 0x7f5a3c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a439f7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c111fb0 con 0x7f5a3c107ad0 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a439f7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c111e30 con 0x7f5a3c068d30 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a439f7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c112130 con 0x7f5a3c0696f0 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a41f94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a3c107ad0 0x7f5a3c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a41793640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c068d30 0x7f5a3c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a41f94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a3c107ad0 0x7f5a3c19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52022/0 (socket says 172.21.15.16:52022) 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a41f94640 1 -- 172.21.15.16:0/1511407188 learned_addr learned my addr 172.21.15.16:0/1511407188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.749+0000 7f5a40f92640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a3c0696f0 0x7f5a3c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a41f94640 1 -- 172.21.15.16:0/1511407188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a3c0696f0 msgr2=0x7f5a3c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a41f94640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a3c0696f0 0x7f5a3c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a41f94640 1 -- 172.21.15.16:0/1511407188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c068d30 msgr2=0x7f5a3c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a41f94640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c068d30 0x7f5a3c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a41f94640 1 -- 172.21.15.16:0/1511407188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c19d520 con 0x7f5a3c107ad0 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a41793640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c068d30 0x7f5a3c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a41f94640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a3c107ad0 0x7f5a3c19cdf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400eb10 tx=0x7f5a3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a327fc640 1 -- 172.21.15.16:0/1511407188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34004b20 con 0x7f5a3c107ad0 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a327fc640 1 -- 172.21.15.16:0/1511407188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5a34004cc0 con 0x7f5a3c107ad0 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a439f7640 1 -- 172.21.15.16:0/1511407188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3c19d810 con 0x7f5a3c107ad0 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a327fc640 1 -- 172.21.15.16:0/1511407188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a340107f0 con 0x7f5a3c107ad0 2024-03-27T06:45:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.750+0000 7f5a439f7640 1 -- 172.21.15.16:0/1511407188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3c1a50a0 con 0x7f5a3c107ad0 2024-03-27T06:45:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.752+0000 7f5a327fc640 1 -- 172.21.15.16:0/1511407188 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f5a34005020 con 0x7f5a3c107ad0 2024-03-27T06:45:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a465e4740 1 Processor -- start 2024-03-27T06:45:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a465e4740 1 -- start start 2024-03-27T06:45:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7c7b0 0x562f17b7cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7d0d0 0x562f17b85770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a327fc640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5a280a3780 0x7f5a280a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 0x562f17b880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a465e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f17ae0710 con 0x562f17b85cb0 2024-03-27T06:45:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a465e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x562f17a88af0 con 0x562f17b7c7b0 2024-03-27T06:45:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a465e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562f17adc6c0 con 0x562f17b7d0d0 2024-03-27T06:45:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a40f92640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5a280a3780 0x7f5a280a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a41f94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 0x562f17b880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a41793640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7d0d0 0x562f17b85770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a41793640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7d0d0 0x562f17b85770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:33624/0 (socket says 172.21.15.16:33624) 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a327fc640 1 -- 172.21.15.16:0/1511407188 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f5a340d0660 con 0x7f5a3c107ad0 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a41793640 1 -- 172.21.15.16:0/2894284063 learned_addr learned my addr 172.21.15.16:0/2894284063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.753+0000 7f5a41f94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 0x562f17b880a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52026/0 (socket says 172.21.15.16:52026) 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.754+0000 7f5a40f92640 1 --2- 172.21.15.16:0/2894284063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7c7b0 0x562f17b7cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.754+0000 7f5a41793640 1 -- 172.21.15.16:0/2894284063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7c7b0 msgr2=0x562f17b7cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.754+0000 7f5a41793640 1 --2- 172.21.15.16:0/2894284063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7c7b0 0x562f17b7cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.754+0000 7f5a41793640 1 -- 172.21.15.16:0/2894284063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 msgr2=0x562f17b880a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.754+0000 7f5a41793640 1 --2- 172.21.15.16:0/2894284063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 0x562f17b880a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.754+0000 7f5a41793640 1 -- 172.21.15.16:0/2894284063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f177f43f0 con 0x562f17b7d0d0 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.754+0000 7f5a40f92640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5a280a3780 0x7f5a280a5c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c100510 tx=0x7f5a38009b50 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.754+0000 7f5a41793640 1 --2- 172.21.15.16:0/2894284063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7d0d0 0x562f17b85770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c006010 tx=0x7f5a2c0095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d08c309a05b199 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.754+0000 7f5a277fe640 1 -- 172.21.15.16:0/2894284063 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c02dae0 con 0x562f17b7d0d0 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.754+0000 7f5a277fe640 1 -- 172.21.15.16:0/2894284063 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5a2c02dc80 con 0x562f17b7d0d0 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.754+0000 7f5a277fe640 1 -- 172.21.15.16:0/2894284063 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c004920 con 0x562f17b7d0d0 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.755+0000 7f5a465e4740 1 -- 172.21.15.16:0/2894284063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7d0d0 msgr2=0x562f17b85770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.755+0000 7f5a465e4740 1 --2- 172.21.15.16:0/2894284063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7d0d0 0x562f17b85770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c006010 tx=0x7f5a2c0095f0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.755+0000 7f5a465e4740 1 -- 172.21.15.16:0/2894284063 shutdown_connections 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.755+0000 7f5a465e4740 1 --2- 172.21.15.16:0/2894284063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 0x562f17b880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.755+0000 7f5a465e4740 1 --2- 172.21.15.16:0/2894284063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7d0d0 0x562f17b85770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.755+0000 7f5a465e4740 1 --2- 172.21.15.16:0/2894284063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7c7b0 0x562f17b7cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.755+0000 7f5a465e4740 1 -- 172.21.15.16:0/2894284063 >> 172.21.15.16:0/2894284063 conn(0x562f17b76ee0 msgr2=0x562f17b772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.755+0000 7f5a465e4740 1 -- 172.21.15.16:0/2894284063 shutdown_connections 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.755+0000 7f5a465e4740 1 -- 172.21.15.16:0/2894284063 wait complete. 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.755+0000 7f5a465e4740 1 -- start start 2024-03-27T06:45:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.755+0000 7f5a465e4740 1 Processor -- start 2024-03-27T06:45:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b82c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7d0d0 0x562f17b83190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b85cb0 0x562f17b836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a465e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f17b88a40 con 0x562f17b7d0d0 2024-03-27T06:45:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a465e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x562f17a9beb0 con 0x562f17b85cb0 2024-03-27T06:45:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a465e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562f17b88bc0 con 0x562f17b7c7b0 2024-03-27T06:45:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a465e4740 10 client.0 _ll_register_callbacks cb 0x562f17ad86a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a41793640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b82c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a40f92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7d0d0 0x562f17b83190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a41793640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b82c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:33636/0 (socket says 172.21.15.16:33636) 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a41793640 1 -- 172.21.15.16:0/1889358222 learned_addr learned my addr 172.21.15.16:0/1889358222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a41f94640 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b85cb0 0x562f17b836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a41793640 1 -- 172.21.15.16:0/1889358222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b85cb0 msgr2=0x562f17b836d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a41793640 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b85cb0 0x562f17b836d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a41793640 1 -- 172.21.15.16:0/1889358222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7d0d0 msgr2=0x562f17b83190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a41793640 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7d0d0 0x562f17b83190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a41793640 1 -- 172.21.15.16:0/1889358222 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f178be7d0 con 0x562f17b7c7b0 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a41793640 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b82c50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c000c00 tx=0x7f5a2c0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cac256ae0e9f796 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.756+0000 7f5a24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a24ff9640 1 -- 172.21.15.16:0/1889358222 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c04d070 con 0x562f17b7c7b0 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a24ff9640 1 -- 172.21.15.16:0/1889358222 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f5a2c0344e0 con 0x562f17b7c7b0 2024-03-27T06:45:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a465e4740 1 -- 172.21.15.16:0/1889358222 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a2c02dc80 con 0x562f17b7c7b0 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a24ff9640 1 -- 172.21.15.16:0/1889358222 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5a2c059050 con 0x562f17b7c7b0 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a465e4740 10 client.25975 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a465e4740 1 -- 172.21.15.16:0/1889358222 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f17b88a40 con 0x562f17b7c7b0 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a24ff9640 1 -- 172.21.15.16:0/1889358222 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5a2c02dc80 con 0x562f17b7c7b0 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a24ff9640 1 -- 172.21.15.16:0/1889358222 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f5a2c041480 con 0x562f17b7c7b0 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a465e4740 10 client.25975 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a465e4740 10 client.25975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a465e4740 20 client.25975 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a465e4740 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562f17b95dc0 0x562f17b981f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a465e4740 4 client.25975 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a465e4740 1 -- 172.21.15.16:0/1889358222 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f17aedc50 con 0x562f17b95dc0 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.757+0000 7f5a40f92640 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562f17b95dc0 0x562f17b981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.758+0000 7f5a24ff9640 10 client.25975 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.758+0000 7f5a40f92640 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562f17b95dc0 0x562f17b981f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4048bb61cd1ee93 server_cookie=9f4db557d044e263 in_seq=0 out_seq=0 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.758+0000 7f5a24ff9640 1 -- 172.21.15.16:0/1889358222 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562f17aedc50 con 0x562f17b95dc0 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.758+0000 7f5a24ff9640 10 client.25975 handle_command_reply: tid=0 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.758+0000 7f5a465e4740 1 client.25975 shutdown 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a465e4740 1 -- 172.21.15.16:0/1889358222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 msgr2=0x562f17b82c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a465e4740 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b82c50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c000c00 tx=0x7f5a2c0038a0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a465e4740 1 -- 172.21.15.16:0/1889358222 shutdown_connections 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a465e4740 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562f17b95dc0 0x562f17b981f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a465e4740 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b85cb0 0x562f17b836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a465e4740 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7d0d0 0x562f17b83190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a465e4740 1 --2- 172.21.15.16:0/1889358222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b82c50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a24ff9640 0 client.25975 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a465e4740 1 -- 172.21.15.16:0/1889358222 >> 172.21.15.16:0/1889358222 conn(0x562f17b76ee0 msgr2=0x562f17b78c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a465e4740 1 -- 172.21.15.16:0/1889358222 shutdown_connections 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a465e4740 1 -- 172.21.15.16:0/1889358222 wait complete. 2024-03-27T06:45:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.759+0000 7f5a465e4740 20 client.25975 trim_cache size 0 max 16384 2024-03-27T06:45:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.772+0000 7f5a465e4740 1 Processor -- start 2024-03-27T06:45:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.772+0000 7f5a465e4740 1 -- start start 2024-03-27T06:45:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.772+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7c7b0 0x562f17b917e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.772+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7d0d0 0x562f17b834f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.772+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b85cb0 0x562f17b78a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.772+0000 7f5a465e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1ab6f0 con 0x562f17b7c7b0 2024-03-27T06:45:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.772+0000 7f5a465e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1ad2f0 con 0x562f17b7d0d0 2024-03-27T06:45:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.772+0000 7f5a465e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1abdc0 con 0x562f17b85cb0 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.772+0000 7f5a41793640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7c7b0 0x562f17b917e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a41793640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7c7b0 0x562f17b917e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52040/0 (socket says 172.21.15.16:52040) 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a41793640 1 -- 172.21.15.16:0/1738021121 learned_addr learned my addr 172.21.15.16:0/1738021121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a40f92640 1 --2- 172.21.15.16:0/1738021121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7d0d0 0x562f17b834f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a41f94640 1 --2- 172.21.15.16:0/1738021121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b85cb0 0x562f17b78a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a41793640 1 -- 172.21.15.16:0/1738021121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b85cb0 msgr2=0x562f17b78a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a41793640 1 --2- 172.21.15.16:0/1738021121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b85cb0 0x562f17b78a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a41793640 1 -- 172.21.15.16:0/1738021121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7d0d0 msgr2=0x562f17b834f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a41793640 1 --2- 172.21.15.16:0/1738021121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7d0d0 0x562f17b834f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a41793640 1 -- 172.21.15.16:0/1738021121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f17b76d10 con 0x562f17b7c7b0 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a41793640 1 --2- 172.21.15.16:0/1738021121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7c7b0 0x562f17b917e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c005e00 tx=0x7f5a2c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df34adc8fa2b8c8b server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a26ffd640 1 -- 172.21.15.16:0/1738021121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c04d040 con 0x562f17b7c7b0 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a26ffd640 1 -- 172.21.15.16:0/1738021121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5a2c003ca0 con 0x562f17b7c7b0 2024-03-27T06:45:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.773+0000 7f5a26ffd640 1 -- 172.21.15.16:0/1738021121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c02dd40 con 0x562f17b7c7b0 2024-03-27T06:45:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.774+0000 7f5a465e4740 1 -- 172.21.15.16:0/1738021121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7c7b0 msgr2=0x562f17b917e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.774+0000 7f5a465e4740 1 --2- 172.21.15.16:0/1738021121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7c7b0 0x562f17b917e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c005e00 tx=0x7f5a2c0056d0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.774+0000 7f5a465e4740 1 -- 172.21.15.16:0/1738021121 shutdown_connections 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.774+0000 7f5a465e4740 1 --2- 172.21.15.16:0/1738021121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b85cb0 0x562f17b78a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.774+0000 7f5a465e4740 1 --2- 172.21.15.16:0/1738021121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7d0d0 0x562f17b834f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.774+0000 7f5a465e4740 1 --2- 172.21.15.16:0/1738021121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b7c7b0 0x562f17b917e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.774+0000 7f5a465e4740 1 -- 172.21.15.16:0/1738021121 >> 172.21.15.16:0/1738021121 conn(0x562f17b76ee0 msgr2=0x562f17b78330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.774+0000 7f5a465e4740 1 -- 172.21.15.16:0/1738021121 shutdown_connections 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.774+0000 7f5a465e4740 1 -- 172.21.15.16:0/1738021121 wait complete. 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.774+0000 7f5a465e4740 1 -- start start 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.774+0000 7f5a465e4740 1 Processor -- start 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7d0d0 0x562f17b893d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a465e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 0x562f17b89910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a465e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1acc20 con 0x562f17b85cb0 2024-03-27T06:45:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a465e4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1ac490 con 0x562f17b7d0d0 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a465e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1a96d0 con 0x562f17b7c7b0 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a465e4740 10 client.0 _ll_register_callbacks cb 0x7f5a34004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41f94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 0x562f17b89910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41793640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41f94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 0x562f17b89910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52044/0 (socket says 172.21.15.16:52044) 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41793640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b8dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:33656/0 (socket says 172.21.15.16:33656) 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41f94640 1 -- 172.21.15.16:0/2822931533 learned_addr learned my addr 172.21.15.16:0/2822931533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a40f92640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7d0d0 0x562f17b893d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41f94640 1 -- 172.21.15.16:0/2822931533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 msgr2=0x562f17b8dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41f94640 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b8dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41f94640 1 -- 172.21.15.16:0/2822931533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7d0d0 msgr2=0x562f17b893d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41f94640 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7d0d0 0x562f17b893d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41f94640 1 -- 172.21.15.16:0/2822931533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f17b8a050 con 0x562f17b85cb0 2024-03-27T06:45:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41793640 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a41f94640 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 0x562f17b89910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5a3409c750 tx=0x7f5a3400acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20076f511057bc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.775+0000 7f5a24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a24ff9640 1 -- 172.21.15.16:0/2822931533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3409cec0 con 0x562f17b85cb0 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a24ff9640 1 -- 172.21.15.16:0/2822931533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f5a3400ad40 con 0x562f17b85cb0 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a465e4740 1 -- 172.21.15.16:0/2822931533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a2c003ca0 con 0x562f17b85cb0 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a24ff9640 1 -- 172.21.15.16:0/2822931533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5a340a62f0 con 0x562f17b85cb0 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a465e4740 10 client.16152 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a465e4740 1 -- 172.21.15.16:0/2822931533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f17bfc030 con 0x562f17b85cb0 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a24ff9640 1 -- 172.21.15.16:0/2822931533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5a2c003ca0 con 0x562f17b85cb0 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a24ff9640 1 -- 172.21.15.16:0/2822931533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f5a340ce070 con 0x562f17b85cb0 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a465e4740 10 client.16152 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a465e4740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a465e4740 20 client.16152 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a465e4740 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562f17bfc810 0x562f17bfec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a465e4740 4 client.16152 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a465e4740 1 -- 172.21.15.16:0/2822931533 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f17ae7fa0 con 0x562f17bfc810 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.776+0000 7f5a41793640 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562f17bfc810 0x562f17bfec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a41793640 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562f17bfc810 0x562f17bfec30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=552d7ad465019ce5 server_cookie=b7d2971de639e64 in_seq=0 out_seq=0 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a24ff9640 10 client.16152 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a24ff9640 1 -- 172.21.15.16:0/2822931533 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f17ae7fa0 con 0x562f17bfc810 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a24ff9640 10 client.16152 handle_command_reply: tid=0 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 1 client.16152 shutdown 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 1 -- 172.21.15.16:0/2822931533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 msgr2=0x562f17b89910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 0x562f17b89910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5a3409c750 tx=0x7f5a3400acd0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 1 -- 172.21.15.16:0/2822931533 shutdown_connections 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x562f17bfc810 0x562f17bfec30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f17b85cb0 0x562f17b89910 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f17b7d0d0 0x562f17b893d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a24ff9640 0 client.16152 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 1 --2- 172.21.15.16:0/2822931533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f17b7c7b0 0x562f17b8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 1 -- 172.21.15.16:0/2822931533 >> 172.21.15.16:0/2822931533 conn(0x562f17b76ee0 msgr2=0x562f17b76000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 1 -- 172.21.15.16:0/2822931533 shutdown_connections 2024-03-27T06:45:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 1 -- 172.21.15.16:0/2822931533 wait complete. 2024-03-27T06:45:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.777+0000 7f5a465e4740 20 client.16152 trim_cache size 0 max 16384 2024-03-27T06:45:19.778 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:45:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 -- 172.21.15.16:0/1511407188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5a280a3780 msgr2=0x7f5a280a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5a280a3780 0x7f5a280a5c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c100510 tx=0x7f5a38009b50 comp rx=0 tx=0).stop 2024-03-27T06:45:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 -- 172.21.15.16:0/1511407188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a3c107ad0 msgr2=0x7f5a3c19cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a3c107ad0 0x7f5a3c19cdf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400eb10 tx=0x7f5a3400c2d0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 -- 172.21.15.16:0/1511407188 shutdown_connections 2024-03-27T06:45:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a3c107ad0 0x7f5a3c19cdf0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f5a280a3780 0x7f5a280a5c40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a3c0696f0 0x7f5a3c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 --2- 172.21.15.16:0/1511407188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a3c068d30 0x7f5a3c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 -- 172.21.15.16:0/1511407188 >> 172.21.15.16:0/1511407188 conn(0x7f5a3c077b50 msgr2=0x7f5a3c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 -- 172.21.15.16:0/1511407188 shutdown_connections 2024-03-27T06:45:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:19.778+0000 7f5a439f7640 1 -- 172.21.15.16:0/1511407188 wait complete. 2024-03-27T06:45:19.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:19 smithi170 ceph-mon[36438]: pgmap v56: 97 pgs: 97 active+clean; 88 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 8.7 MiB/s wr, 43 op/s 2024-03-27T06:45:19.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:19 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/613648781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:45:20.784 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-27T06:45:20.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:45:20.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-27T06:45:20.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.899+0000 7f297a977640 1 -- 172.21.15.16:0/923023034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29740696f0 msgr2=0x7f297410e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:20.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.899+0000 7f297a977640 1 --2- 172.21.15.16:0/923023034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29740696f0 0x7f297410e990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f296c009cd0 tx=0x7f296c02c0f0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.899+0000 7f297a977640 1 -- 172.21.15.16:0/923023034 shutdown_connections 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.899+0000 7f297a977640 1 --2- 172.21.15.16:0/923023034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29740696f0 0x7f297410e990 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.899+0000 7f297a977640 1 --2- 172.21.15.16:0/923023034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2974068d30 0x7f29740691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.899+0000 7f297a977640 1 --2- 172.21.15.16:0/923023034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2974107ad0 0x7f2974107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.899+0000 7f297a977640 1 -- 172.21.15.16:0/923023034 >> 172.21.15.16:0/923023034 conn(0x7f2974077b50 msgr2=0x7f2974077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.899+0000 7f297a977640 1 -- 172.21.15.16:0/923023034 shutdown_connections 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.899+0000 7f297a977640 1 -- 172.21.15.16:0/923023034 wait complete. 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f297a977640 1 Processor -- start 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f297a977640 1 -- start start 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f297a977640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974068d30 0x7f2974198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f297a977640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740696f0 0x7f29741989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f297a977640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2974107ad0 0x7f297419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f297a977640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2974112000 con 0x7f2974068d30 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f297a977640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f2974111e80 con 0x7f2974107ad0 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f297a977640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2974112180 con 0x7f29740696f0 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f2973fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974068d30 0x7f2974198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f2973fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974068d30 0x7f2974198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52062/0 (socket says 172.21.15.16:52062) 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f2973fff640 1 -- 172.21.15.16:0/3623553465 learned_addr learned my addr 172.21.15.16:0/3623553465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.900+0000 7f2978f14640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2974107ad0 0x7f297419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f29737fe640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740696f0 0x7f29741989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f2973fff640 1 -- 172.21.15.16:0/3623553465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740696f0 msgr2=0x7f29741989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f2973fff640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740696f0 0x7f29741989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f2973fff640 1 -- 172.21.15.16:0/3623553465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2974107ad0 msgr2=0x7f297419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f2973fff640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2974107ad0 0x7f297419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f2973fff640 1 -- 172.21.15.16:0/3623553465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297419d450 con 0x7f2974068d30 2024-03-27T06:45:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f2973fff640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974068d30 0x7f2974198460 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f296400e9f0 tx=0x7f296400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f29717fa640 1 -- 172.21.15.16:0/3623553465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296400cd80 con 0x7f2974068d30 2024-03-27T06:45:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f297a977640 1 -- 172.21.15.16:0/3623553465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297419d740 con 0x7f2974068d30 2024-03-27T06:45:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f29717fa640 1 -- 172.21.15.16:0/3623553465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2964010930 con 0x7f2974068d30 2024-03-27T06:45:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f29717fa640 1 -- 172.21.15.16:0/3623553465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964018b00 con 0x7f2974068d30 2024-03-27T06:45:20.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.901+0000 7f297a977640 1 -- 172.21.15.16:0/3623553465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741a4fd0 con 0x7f2974068d30 2024-03-27T06:45:20.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.904+0000 7f29717fa640 1 -- 172.21.15.16:0/3623553465 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f2964018ca0 con 0x7f2974068d30 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.904+0000 7f297d636740 1 Processor -- start 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.904+0000 7f297d636740 1 -- start start 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.904+0000 7f297d636740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613948e50 0x55f613949230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.904+0000 7f297d636740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f613951e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.904+0000 7f29717fa640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f295c0a3780 0x7f295c0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.904+0000 7f297d636740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 0x55f613954740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.904+0000 7f297d636740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f613853c20 con 0x55f613952350 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.904+0000 7f297d636740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55f6138ac430 con 0x55f613948e50 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.904+0000 7f297d636740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f613854900 con 0x55f613949770 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f2978f14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 0x55f613954740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f2973fff640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f295c0a3780 0x7f295c0a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f2978f14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 0x55f613954740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52072/0 (socket says 172.21.15.16:52072) 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f2978f14640 1 -- 172.21.15.16:0/3824268858 learned_addr learned my addr 172.21.15.16:0/3824268858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f29737fe640 1 --2- 172.21.15.16:0/3824268858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f613951e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f2978f14640 1 --2- 172.21.15.16:0/3824268858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613948e50 0x55f613949230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f29717fa640 1 -- 172.21.15.16:0/3623553465 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f29640d0400 con 0x7f2974068d30 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f29737fe640 1 -- 172.21.15.16:0/3824268858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613948e50 msgr2=0x55f613949230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f29737fe640 1 --2- 172.21.15.16:0/3824268858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613948e50 0x55f613949230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f29737fe640 1 -- 172.21.15.16:0/3824268858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 msgr2=0x55f613954740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f29737fe640 1 --2- 172.21.15.16:0/3824268858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 0x55f613954740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f29737fe640 1 -- 172.21.15.16:0/3824268858 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6138a7180 con 0x55f613949770 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f2978f14640 1 --2- 172.21.15.16:0/3824268858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 0x55f613954740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f2973fff640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f295c0a3780 0x7f295c0a5c40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f29640cdc30 tx=0x7f2964018f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f29737fe640 1 --2- 172.21.15.16:0/3824268858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f613951e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2968009bd0 tx=0x7f2968011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4080ecde1e7b5aef server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f295a7fc640 1 -- 172.21.15.16:0/3824268858 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968004230 con 0x55f613949770 2024-03-27T06:45:20.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f295a7fc640 1 -- 172.21.15.16:0/3824268858 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2968009e40 con 0x55f613949770 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.905+0000 7f295a7fc640 1 -- 172.21.15.16:0/3824268858 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968004d50 con 0x55f613949770 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.906+0000 7f297d636740 1 -- 172.21.15.16:0/3824268858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 msgr2=0x55f613951e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.906+0000 7f297d636740 1 --2- 172.21.15.16:0/3824268858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f613951e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2968009bd0 tx=0x7f2968011810 comp rx=0 tx=0).stop 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.906+0000 7f297d636740 1 -- 172.21.15.16:0/3824268858 shutdown_connections 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.906+0000 7f297d636740 1 --2- 172.21.15.16:0/3824268858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 0x55f613954740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.906+0000 7f297d636740 1 --2- 172.21.15.16:0/3824268858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f613951e10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.906+0000 7f297d636740 1 --2- 172.21.15.16:0/3824268858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613948e50 0x55f613949230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.906+0000 7f297d636740 1 -- 172.21.15.16:0/3824268858 >> 172.21.15.16:0/3824268858 conn(0x55f613943580 msgr2=0x55f613943960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.906+0000 7f297d636740 1 -- 172.21.15.16:0/3824268858 shutdown_connections 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.906+0000 7f297d636740 1 -- 172.21.15.16:0/3824268858 wait complete. 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f297d636740 1 -- start start 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f297d636740 1 Processor -- start 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f297d636740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613948e50 0x55f613951c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f297d636740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f61394ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f297d636740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 0x55f61394f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f297d636740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f613867cc0 con 0x55f613952350 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f297d636740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55f6138a83e0 con 0x55f613948e50 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f297d636740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f613955140 con 0x55f613949770 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f297d636740 10 client.0 _ll_register_callbacks cb 0x55f6138a2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f2978f14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 0x55f61394f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f2978f14640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613948e50 0x55f613951c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f29737fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f61394ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f2978f14640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613948e50 0x55f613951c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:55670/0 (socket says 172.21.15.16:55670) 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f29737fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f61394ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:33696/0 (socket says 172.21.15.16:33696) 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.907+0000 7f2978f14640 1 -- 172.21.15.16:0/2835289422 learned_addr learned my addr 172.21.15.16:0/2835289422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f29737fe640 1 -- 172.21.15.16:0/2835289422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613948e50 msgr2=0x55f613951c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:20.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f29737fe640 1 --2- 172.21.15.16:0/2835289422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613948e50 0x55f613951c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f29737fe640 1 -- 172.21.15.16:0/2835289422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 msgr2=0x55f61394f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f29737fe640 1 --2- 172.21.15.16:0/2835289422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 0x55f61394f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f29737fe640 1 -- 172.21.15.16:0/2835289422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61386eef0 con 0x55f613949770 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f29737fe640 1 --2- 172.21.15.16:0/2835289422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f61394ef90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f29680096e0 tx=0x7f29680038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=464542f0c147b9b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f29537fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f29537fe640 1 -- 172.21.15.16:0/2835289422 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968027020 con 0x55f613949770 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f29537fe640 1 -- 172.21.15.16:0/2835289422 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f2968003c90 con 0x55f613949770 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f297d636740 1 -- 172.21.15.16:0/2835289422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f6138b9970 con 0x55f613949770 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f29537fe640 1 -- 172.21.15.16:0/2835289422 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2968032050 con 0x55f613949770 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f297d636740 10 client.25993 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f297d636740 1 -- 172.21.15.16:0/2835289422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f6136887e0 con 0x55f613949770 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.908+0000 7f29537fe640 1 -- 172.21.15.16:0/2835289422 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55f6138b9970 con 0x55f613949770 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.909+0000 7f29537fe640 1 -- 172.21.15.16:0/2835289422 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f2968018070 con 0x55f613949770 2024-03-27T06:45:20.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.909+0000 7f297d636740 10 client.25993 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.909+0000 7f297d636740 10 client.25993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.909+0000 7f297d636740 20 client.25993 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.909+0000 7f297d636740 1 --2- 172.21.15.16:0/2835289422 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f613962190 0x55f6139645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.909+0000 7f297d636740 4 client.25993 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.909+0000 7f297d636740 1 -- 172.21.15.16:0/2835289422 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6138b3cc0 con 0x55f613962190 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.909+0000 7f2978f14640 1 --2- 172.21.15.16:0/2835289422 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f613962190 0x55f6139645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.909+0000 7f29537fe640 10 client.25993 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.909+0000 7f2978f14640 1 --2- 172.21.15.16:0/2835289422 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f613962190 0x55f6139645e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447d28aad7fe4da2 server_cookie=2d395a0aeba859de in_seq=0 out_seq=0 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f29537fe640 1 -- 172.21.15.16:0/2835289422 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f6138b3cc0 con 0x55f613962190 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f29537fe640 10 client.25993 handle_command_reply: tid=0 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 1 client.25993 shutdown 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 1 -- 172.21.15.16:0/2835289422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 msgr2=0x55f61394ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:20.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 1 --2- 172.21.15.16:0/2835289422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f61394ef90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f29680096e0 tx=0x7f29680038a0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 1 -- 172.21.15.16:0/2835289422 shutdown_connections 2024-03-27T06:45:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 1 --2- 172.21.15.16:0/2835289422 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f613962190 0x55f6139645e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 1 --2- 172.21.15.16:0/2835289422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613952350 0x55f61394f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 1 --2- 172.21.15.16:0/2835289422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f61394ef90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f29537fe640 0 client.25993 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 1 --2- 172.21.15.16:0/2835289422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613948e50 0x55f613951c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 1 -- 172.21.15.16:0/2835289422 >> 172.21.15.16:0/2835289422 conn(0x55f613943580 msgr2=0x55f6139452d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 1 -- 172.21.15.16:0/2835289422 shutdown_connections 2024-03-27T06:45:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 1 -- 172.21.15.16:0/2835289422 wait complete. 2024-03-27T06:45:20.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.910+0000 7f297d636740 20 client.25993 trim_cache size 0 max 16384 2024-03-27T06:45:20.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.928+0000 7f297d636740 1 Processor -- start 2024-03-27T06:45:20.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.928+0000 7f297d636740 1 -- start start 2024-03-27T06:45:20.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f297d636740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613948e50 0x55f61395ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f297d636740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f613969eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f297d636740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613952350 0x55f6139516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f297d636740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29741ad750 con 0x55f613948e50 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f297d636740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f29741af350 con 0x55f613952350 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f297d636740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f29741ade20 con 0x55f613949770 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2978f14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613948e50 0x55f61395ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2978f14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613948e50 0x55f61395ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52100/0 (socket says 172.21.15.16:52100) 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2978f14640 1 -- 172.21.15.16:0/1964563121 learned_addr learned my addr 172.21.15.16:0/1964563121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f29737fe640 1 --2- 172.21.15.16:0/1964563121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f613969eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2978f14640 1 --2- 172.21.15.16:0/1964563121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613952350 0x55f6139516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2978f14640 1 -- 172.21.15.16:0/1964563121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 msgr2=0x55f613969eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2978f14640 1 --2- 172.21.15.16:0/1964563121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f613969eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2978f14640 1 -- 172.21.15.16:0/1964563121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613952350 msgr2=0x55f6139516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:20.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2978f14640 1 --2- 172.21.15.16:0/1964563121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613952350 0x55f6139516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2978f14640 1 -- 172.21.15.16:0/1964563121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61394df40 con 0x55f613948e50 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2978f14640 1 --2- 172.21.15.16:0/1964563121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613948e50 0x55f61395ddc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f296c004af0 tx=0x7f296c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e69903bcf53ccc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2959ffb640 1 -- 172.21.15.16:0/1964563121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c039040 con 0x55f613948e50 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2959ffb640 1 -- 172.21.15.16:0/1964563121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f296c04e310 con 0x55f613948e50 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.929+0000 7f2959ffb640 1 -- 172.21.15.16:0/1964563121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c004290 con 0x55f613948e50 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.930+0000 7f297d636740 1 -- 172.21.15.16:0/1964563121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613948e50 msgr2=0x55f61395ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.930+0000 7f297d636740 1 --2- 172.21.15.16:0/1964563121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613948e50 0x55f61395ddc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f296c004af0 tx=0x7f296c0095c0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.930+0000 7f297d636740 1 -- 172.21.15.16:0/1964563121 shutdown_connections 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.930+0000 7f297d636740 1 --2- 172.21.15.16:0/1964563121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613952350 0x55f6139516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.930+0000 7f297d636740 1 --2- 172.21.15.16:0/1964563121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613949770 0x55f613969eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.930+0000 7f297d636740 1 --2- 172.21.15.16:0/1964563121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613948e50 0x55f61395ddc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.930+0000 7f297d636740 1 -- 172.21.15.16:0/1964563121 >> 172.21.15.16:0/1964563121 conn(0x55f613943580 msgr2=0x55f613942590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.930+0000 7f297d636740 1 -- 172.21.15.16:0/1964563121 shutdown_connections 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.930+0000 7f297d636740 1 -- 172.21.15.16:0/1964563121 wait complete. 2024-03-27T06:45:20.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f297d636740 1 -- start start 2024-03-27T06:45:20.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f297d636740 1 Processor -- start 2024-03-27T06:45:20.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f297d636740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613948e50 0x55f61394bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f297d636740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613949770 0x55f61394c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f297d636740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613952350 0x55f61394c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f297d636740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29741aec80 con 0x55f613949770 2024-03-27T06:45:20.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f297d636740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f29741ae4f0 con 0x55f613952350 2024-03-27T06:45:20.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f297d636740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f29741ab730 con 0x55f613948e50 2024-03-27T06:45:20.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f297d636740 10 client.0 _ll_register_callbacks cb 0x7f2964005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:20.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f29737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613949770 0x55f61394c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f29737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613949770 0x55f61394c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52112/0 (socket says 172.21.15.16:52112) 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f29737fe640 1 -- 172.21.15.16:0/511654746 learned_addr learned my addr 172.21.15.16:0/511654746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f2978f14640 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613948e50 0x55f61394bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f2978f14640 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613952350 0x55f61394c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f29737fe640 1 -- 172.21.15.16:0/511654746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613948e50 msgr2=0x55f61394bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f29737fe640 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613948e50 0x55f61394bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f29737fe640 1 -- 172.21.15.16:0/511654746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613952350 msgr2=0x55f61394c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f29737fe640 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613952350 0x55f61394c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f29737fe640 1 -- 172.21.15.16:0/511654746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6136e8f70 con 0x55f613949770 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.931+0000 7f2978f14640 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613952350 0x55f61394c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f29737fe640 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613949770 0x55f61394c300 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55f613950560 tx=0x7f2968011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce56955b73786317 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f29537fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f29537fe640 1 -- 172.21.15.16:0/511654746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968027040 con 0x55f613949770 2024-03-27T06:45:20.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f297d636740 1 -- 172.21.15.16:0/511654746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f296c04e310 con 0x55f613949770 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f29537fe640 1 -- 172.21.15.16:0/511654746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f2968003910 con 0x55f613949770 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f29537fe640 1 -- 172.21.15.16:0/511654746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f296801e040 con 0x55f613949770 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f297d636740 10 client.16170 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f297d636740 1 -- 172.21.15.16:0/511654746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f613580170 con 0x55f613949770 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f29537fe640 1 -- 172.21.15.16:0/511654746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f296c04e310 con 0x55f613949770 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f29537fe640 1 -- 172.21.15.16:0/511654746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f2968018030 con 0x55f613949770 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f297d636740 10 client.16170 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f297d636740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f297d636740 20 client.16170 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.932+0000 7f297d636740 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f6139c8e30 0x55f6139cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.933+0000 7f297d636740 4 client.16170 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.933+0000 7f297d636740 1 -- 172.21.15.16:0/511654746 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f613831be0 con 0x55f6139c8e30 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.933+0000 7f2978f14640 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f6139c8e30 0x55f6139cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.933+0000 7f29537fe640 10 client.16170 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.933+0000 7f2978f14640 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f6139c8e30 0x55f6139cb250 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56da2486bbfbfee server_cookie=9edd4cc0aa41614a in_seq=0 out_seq=0 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.934+0000 7f29537fe640 1 -- 172.21.15.16:0/511654746 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f613831be0 con 0x55f6139c8e30 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.934+0000 7f29537fe640 10 client.16170 handle_command_reply: tid=0 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.934+0000 7f297d636740 1 client.16170 shutdown 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297d636740 1 -- 172.21.15.16:0/511654746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613949770 msgr2=0x55f61394c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297d636740 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613949770 0x55f61394c300 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55f613950560 tx=0x7f2968011d40 comp rx=0 tx=0).stop 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297d636740 1 -- 172.21.15.16:0/511654746 shutdown_connections 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297d636740 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f6139c8e30 0x55f6139cb250 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297d636740 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f613952350 0x55f61394c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297d636740 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f613949770 0x55f61394c300 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297d636740 1 --2- 172.21.15.16:0/511654746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f613948e50 0x55f61394bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f29537fe640 0 client.16170 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297d636740 1 -- 172.21.15.16:0/511654746 >> 172.21.15.16:0/511654746 conn(0x55f613943580 msgr2=0x55f613949c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:20.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297d636740 1 -- 172.21.15.16:0/511654746 shutdown_connections 2024-03-27T06:45:20.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297d636740 1 -- 172.21.15.16:0/511654746 wait complete. 2024-03-27T06:45:20.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297d636740 20 client.16170 trim_cache size 0 max 16384 2024-03-27T06:45:20.935 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:45:20.935 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-03-27T06:45:20.935 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "5c855730-f5df-4eae-a7e3-637f53387896", 2024-03-27T06:45:20.935 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-03-27T06:45:20.935 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:45:20.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297a977640 1 -- 172.21.15.16:0/3623553465 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f295c0a3780 msgr2=0x7f295c0a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:20.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297a977640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f295c0a3780 0x7f295c0a5c40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f29640cdc30 tx=0x7f2964018f90 comp rx=0 tx=0).stop 2024-03-27T06:45:20.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297a977640 1 -- 172.21.15.16:0/3623553465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974068d30 msgr2=0x7f2974198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:20.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.935+0000 7f297a977640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974068d30 0x7f2974198460 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f296400e9f0 tx=0x7f296400eeb0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.936+0000 7f297a977640 1 -- 172.21.15.16:0/3623553465 shutdown_connections 2024-03-27T06:45:20.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.936+0000 7f297a977640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2974107ad0 0x7f297419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.936+0000 7f297a977640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f295c0a3780 0x7f295c0a5c40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.936+0000 7f297a977640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740696f0 0x7f29741989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.936+0000 7f297a977640 1 --2- 172.21.15.16:0/3623553465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2974068d30 0x7f2974198460 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:20.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.936+0000 7f297a977640 1 -- 172.21.15.16:0/3623553465 >> 172.21.15.16:0/3623553465 conn(0x7f2974077b50 msgr2=0x7f29740fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:20.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.936+0000 7f297a977640 1 -- 172.21.15.16:0/3623553465 shutdown_connections 2024-03-27T06:45:20.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:20.936+0000 7f297a977640 1 -- 172.21.15.16:0/3623553465 wait complete. 2024-03-27T06:45:20.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T06:45:20.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T06:45:21.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.060+0000 7fb33f1a0640 1 -- 172.21.15.16:0/612403194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3381024b0 msgr2=0x7fb33810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.060+0000 7fb33f1a0640 1 --2- 172.21.15.16:0/612403194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3381024b0 0x7fb33810e9c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb330009cd0 tx=0x7fb33002c0f0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 -- 172.21.15.16:0/612403194 shutdown_connections 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 --2- 172.21.15.16:0/612403194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3381024b0 0x7fb33810e9c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 --2- 172.21.15.16:0/612403194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb338101af0 0x7fb338101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 --2- 172.21.15.16:0/612403194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb338107b90 0x7fb338107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 -- 172.21.15.16:0/612403194 >> 172.21.15.16:0/612403194 conn(0x7fb3380fd580 msgr2=0x7fb3380ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 -- 172.21.15.16:0/612403194 shutdown_connections 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 -- 172.21.15.16:0/612403194 wait complete. 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 Processor -- start 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 -- start start 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb338101af0 0x7fb3381984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3381024b0 0x7fb338198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb338107b90 0x7fb33819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb338112000 con 0x7fb338107b90 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fb338111e80 con 0x7fb3381024b0 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33f1a0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb338112180 con 0x7fb338101af0 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.061+0000 7fb33d73d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb338107b90 0x7fb33819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb33cf3c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb338101af0 0x7fb3381984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb337fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3381024b0 0x7fb338198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb33d73d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb338107b90 0x7fb33819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52136/0 (socket says 172.21.15.16:52136) 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb33d73d640 1 -- 172.21.15.16:0/294555156 learned_addr learned my addr 172.21.15.16:0/294555156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb33cf3c640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb338101af0 0x7fb3381984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:33736/0 (socket says 172.21.15.16:33736) 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb33d73d640 1 -- 172.21.15.16:0/294555156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb338101af0 msgr2=0x7fb3381984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb33d73d640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb338101af0 0x7fb3381984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb33d73d640 1 -- 172.21.15.16:0/294555156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3381024b0 msgr2=0x7fb338198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:21.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb33d73d640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3381024b0 0x7fb338198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb33d73d640 1 -- 172.21.15.16:0/294555156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33819d4b0 con 0x7fb338107b90 2024-03-27T06:45:21.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb33d73d640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb338107b90 0x7fb33819cd80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb330009ca0 tx=0x7fb330034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:21.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb335ffb640 1 -- 172.21.15.16:0/294555156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330039040 con 0x7fb338107b90 2024-03-27T06:45:21.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb33f1a0640 1 -- 172.21.15.16:0/294555156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33819d740 con 0x7fb338107b90 2024-03-27T06:45:21.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.062+0000 7fb335ffb640 1 -- 172.21.15.16:0/294555156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb330044070 con 0x7fb338107b90 2024-03-27T06:45:21.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.063+0000 7fb33f1a0640 1 -- 172.21.15.16:0/294555156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3381a5030 con 0x7fb338107b90 2024-03-27T06:45:21.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.063+0000 7fb335ffb640 1 -- 172.21.15.16:0/294555156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330004460 con 0x7fb338107b90 2024-03-27T06:45:21.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.065+0000 7fb335ffb640 1 -- 172.21.15.16:0/294555156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fb330032870 con 0x7fb338107b90 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.065+0000 7fb341d81740 1 Processor -- start 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.065+0000 7fb341d81740 1 -- start start 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb341d81740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f64244d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb341d81740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642456210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb341d81740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642458b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb341d81740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f6423b1860 con 0x55f64244d250 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb341d81740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55f6423598d0 con 0x55f64244db70 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb341d81740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f6423ad810 con 0x55f642456750 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb33cf3c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642456210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb337fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f64244d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb33cf3c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642456210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:55704/0 (socket says 172.21.15.16:55704) 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb33cf3c640 1 -- 172.21.15.16:0/31310466 learned_addr learned my addr 172.21.15.16:0/31310466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb335ffb640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb3280a3780 0x7fb3280a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb337fff640 1 --2- 172.21.15.16:0/31310466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642458b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb33d73d640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb3280a3780 0x7fb3280a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb335ffb640 1 -- 172.21.15.16:0/294555156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fb3300f2590 con 0x7fb338107b90 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb33cf3c640 1 -- 172.21.15.16:0/31310466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 msgr2=0x55f642458b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb33cf3c640 1 --2- 172.21.15.16:0/31310466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642458b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb33cf3c640 1 -- 172.21.15.16:0/31310466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 msgr2=0x55f64244d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb33cf3c640 1 --2- 172.21.15.16:0/31310466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f64244d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb33cf3c640 1 -- 172.21.15.16:0/31310466 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f642364120 con 0x55f64244db70 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.066+0000 7fb337fff640 1 --2- 172.21.15.16:0/31310466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f64244d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.067+0000 7fb33cf3c640 1 --2- 172.21.15.16:0/31310466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642456210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00bc90 tx=0x7fb32c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d90e3961084382 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.067+0000 7fb33d73d640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb3280a3780 0x7fb3280a5c40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb3300efed0 tx=0x7fb33004e000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.067+0000 7fb322ffd640 1 -- 172.21.15.16:0/31310466 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c01c480 con 0x55f64244db70 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.067+0000 7fb322ffd640 1 -- 172.21.15.16:0/31310466 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb32c01cb20 con 0x55f64244db70 2024-03-27T06:45:21.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.067+0000 7fb322ffd640 1 -- 172.21.15.16:0/31310466 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c011bf0 con 0x55f64244db70 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.068+0000 7fb341d81740 1 -- 172.21.15.16:0/31310466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 msgr2=0x55f642456210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.068+0000 7fb341d81740 1 --2- 172.21.15.16:0/31310466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642456210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00bc90 tx=0x7fb32c0107a0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.068+0000 7fb341d81740 1 -- 172.21.15.16:0/31310466 shutdown_connections 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.068+0000 7fb341d81740 1 --2- 172.21.15.16:0/31310466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642458b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.068+0000 7fb341d81740 1 --2- 172.21.15.16:0/31310466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642456210 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.068+0000 7fb341d81740 1 --2- 172.21.15.16:0/31310466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f64244d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.068+0000 7fb341d81740 1 -- 172.21.15.16:0/31310466 >> 172.21.15.16:0/31310466 conn(0x55f642447980 msgr2=0x55f642447d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.068+0000 7fb341d81740 1 -- 172.21.15.16:0/31310466 shutdown_connections 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.068+0000 7fb341d81740 1 -- 172.21.15.16:0/31310466 wait complete. 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.068+0000 7fb341d81740 1 -- start start 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.068+0000 7fb341d81740 1 Processor -- start 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb341d81740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244d250 0x55f6424609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb341d81740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244db70 0x55f642460ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb341d81740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f64245adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb341d81740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f642459540 con 0x55f64244db70 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb341d81740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55f64236ccd0 con 0x55f64244d250 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb341d81740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f6424596c0 con 0x55f642456750 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb341d81740 10 client.0 _ll_register_callbacks cb 0x55f64235eee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb337fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f64245adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb33cf3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244db70 0x55f642460ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb33cf3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244db70 0x55f642460ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52150/0 (socket says 172.21.15.16:52150) 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb33cf3c640 1 -- 172.21.15.16:0/4234794865 learned_addr learned my addr 172.21.15.16:0/4234794865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb337fff640 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244d250 0x55f6424609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb33cf3c640 1 -- 172.21.15.16:0/4234794865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 msgr2=0x55f64245adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb33cf3c640 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f64245adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb33cf3c640 1 -- 172.21.15.16:0/4234794865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244d250 msgr2=0x55f6424609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb33cf3c640 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244d250 0x55f6424609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb33cf3c640 1 -- 172.21.15.16:0/4234794865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f64218f7d0 con 0x55f64244db70 2024-03-27T06:45:21.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb337fff640 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f64245adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:45:21.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb33cf3c640 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244db70 0x55f642460ef0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb32c010ca0 tx=0x7fb32c01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817fc194ceff42cd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:21.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb317fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:21.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.069+0000 7fb317fff640 1 -- 172.21.15.16:0/4234794865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c011b00 con 0x55f64244db70 2024-03-27T06:45:21.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.070+0000 7fb341d81740 1 -- 172.21.15.16:0/4234794865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb32c01cb20 con 0x55f64244db70 2024-03-27T06:45:21.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.070+0000 7fb317fff640 1 -- 172.21.15.16:0/4234794865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fb32c01c920 con 0x55f64244db70 2024-03-27T06:45:21.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.070+0000 7fb317fff640 1 -- 172.21.15.16:0/4234794865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb32c011ca0 con 0x55f64244db70 2024-03-27T06:45:21.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.070+0000 7fb341d81740 10 client.16182 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:21.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.070+0000 7fb341d81740 1 -- 172.21.15.16:0/4234794865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f6420c53f0 con 0x55f64244db70 2024-03-27T06:45:21.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.070+0000 7fb317fff640 1 -- 172.21.15.16:0/4234794865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb32c01cb20 con 0x55f64244db70 2024-03-27T06:45:21.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.071+0000 7fb317fff640 1 -- 172.21.15.16:0/4234794865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb32c01dce0 con 0x55f64244db70 2024-03-27T06:45:21.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.071+0000 7fb341d81740 10 client.16182 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:21.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.071+0000 7fb341d81740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:21.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.071+0000 7fb341d81740 20 client.16182 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:21.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.071+0000 7fb341d81740 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f64246ae60 0x55f64246d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.071+0000 7fb341d81740 4 client.16182 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.071+0000 7fb341d81740 1 -- 172.21.15.16:0/4234794865 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6423beda0 con 0x55f64246ae60 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.071+0000 7fb337fff640 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f64246ae60 0x55f64246d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.071+0000 7fb337fff640 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f64246ae60 0x55f64246d250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f075429d82ffa2d0 server_cookie=1681388aa1499045 in_seq=0 out_seq=0 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.071+0000 7fb317fff640 10 client.16182 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.072+0000 7fb317fff640 1 -- 172.21.15.16:0/4234794865 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f6423beda0 con 0x55f64246ae60 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb317fff640 10 client.16182 handle_command_reply: tid=0 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 1 client.16182 shutdown 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 1 -- 172.21.15.16:0/4234794865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244db70 msgr2=0x55f642460ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244db70 0x55f642460ef0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb32c010ca0 tx=0x7fb32c01c730 comp rx=0 tx=0).stop 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 1 -- 172.21.15.16:0/4234794865 shutdown_connections 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f64246ae60 0x55f64246d250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f64245adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244db70 0x55f642460ef0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 1 --2- 172.21.15.16:0/4234794865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244d250 0x55f6424609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 1 -- 172.21.15.16:0/4234794865 >> 172.21.15.16:0/4234794865 conn(0x55f642447980 msgr2=0x55f6424496d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:21.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb317fff640 0 client.16182 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:21.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 1 -- 172.21.15.16:0/4234794865 shutdown_connections 2024-03-27T06:45:21.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 1 -- 172.21.15.16:0/4234794865 wait complete. 2024-03-27T06:45:21.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.073+0000 7fb341d81740 20 client.16182 trim_cache size 0 max 16384 2024-03-27T06:45:21.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.094+0000 7fb341d81740 1 Processor -- start 2024-03-27T06:45:21.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.094+0000 7fb341d81740 1 -- start start 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb341d81740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642460c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb341d81740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642472ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb341d81740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642454d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb341d81740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381ab6b0 con 0x55f64244d250 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb341d81740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381ad2b0 con 0x55f64244db70 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb341d81740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381abd80 con 0x55f642456750 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb337fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642460c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb337fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642460c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52158/0 (socket says 172.21.15.16:52158) 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb337fff640 1 -- 172.21.15.16:0/137691094 learned_addr learned my addr 172.21.15.16:0/137691094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb33cf3c640 1 --2- 172.21.15.16:0/137691094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642472ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb337fff640 1 --2- 172.21.15.16:0/137691094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642454d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb337fff640 1 -- 172.21.15.16:0/137691094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 msgr2=0x55f642454d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb337fff640 1 --2- 172.21.15.16:0/137691094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642454d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb337fff640 1 -- 172.21.15.16:0/137691094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 msgr2=0x55f642472ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:21.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb337fff640 1 --2- 172.21.15.16:0/137691094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642472ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb337fff640 1 -- 172.21.15.16:0/137691094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6420850f0 con 0x55f64244d250 2024-03-27T06:45:21.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb337fff640 1 --2- 172.21.15.16:0/137691094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642460c10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb3381070b0 tx=0x7fb3240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bbe60be50a6374 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:21.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.095+0000 7fb3227fc640 1 -- 172.21.15.16:0/137691094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324004220 con 0x55f64244d250 2024-03-27T06:45:21.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.096+0000 7fb3227fc640 1 -- 172.21.15.16:0/137691094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb3240043c0 con 0x55f64244d250 2024-03-27T06:45:21.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.096+0000 7fb3227fc640 1 -- 172.21.15.16:0/137691094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3240035f0 con 0x55f64244d250 2024-03-27T06:45:21.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.096+0000 7fb341d81740 1 -- 172.21.15.16:0/137691094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 msgr2=0x55f642460c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:21.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.096+0000 7fb341d81740 1 --2- 172.21.15.16:0/137691094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642460c10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb3381070b0 tx=0x7fb3240046c0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.096+0000 7fb341d81740 1 -- 172.21.15.16:0/137691094 shutdown_connections 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.096+0000 7fb341d81740 1 --2- 172.21.15.16:0/137691094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642454d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.096+0000 7fb341d81740 1 --2- 172.21.15.16:0/137691094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642472ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.096+0000 7fb341d81740 1 --2- 172.21.15.16:0/137691094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642460c10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.096+0000 7fb341d81740 1 -- 172.21.15.16:0/137691094 >> 172.21.15.16:0/137691094 conn(0x55f642447980 msgr2=0x55f642460500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.096+0000 7fb341d81740 1 -- 172.21.15.16:0/137691094 shutdown_connections 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.096+0000 7fb341d81740 1 -- 172.21.15.16:0/137691094 wait complete. 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.097+0000 7fb341d81740 1 -- start start 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.097+0000 7fb341d81740 1 Processor -- start 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.097+0000 7fb341d81740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642466a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.097+0000 7fb341d81740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642466f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.097+0000 7fb341d81740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642463ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.097+0000 7fb341d81740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381acbe0 con 0x55f64244d250 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.097+0000 7fb341d81740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381ac450 con 0x55f64244db70 2024-03-27T06:45:21.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.097+0000 7fb341d81740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381a96b0 con 0x55f642456750 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.097+0000 7fb341d81740 10 client.0 _ll_register_callbacks cb 0x7fb32400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb337fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642466a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb337fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642466a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52162/0 (socket says 172.21.15.16:52162) 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb337fff640 1 -- 172.21.15.16:0/1336566809 learned_addr learned my addr 172.21.15.16:0/1336566809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb33cf3c640 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642466f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb337fff640 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642463ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb337fff640 1 -- 172.21.15.16:0/1336566809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 msgr2=0x55f642463ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb337fff640 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642463ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb337fff640 1 -- 172.21.15.16:0/1336566809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 msgr2=0x55f642466f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb337fff640 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642466f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb337fff640 1 -- 172.21.15.16:0/1336566809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6424646a0 con 0x55f64244d250 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb337fff640 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642466a30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb324005860 tx=0x7fb3240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58acbe20873af873 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb317fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:21.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb317fff640 1 -- 172.21.15.16:0/1336566809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324034070 con 0x55f64244d250 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb317fff640 1 -- 172.21.15.16:0/1336566809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fb3240035f0 con 0x55f64244d250 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb341d81740 1 -- 172.21.15.16:0/1336566809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3240043c0 con 0x55f64244d250 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb317fff640 1 -- 172.21.15.16:0/1336566809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb3240037f0 con 0x55f64244d250 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb341d81740 10 client.16188 fetch_fsmap learned FSMap version 27 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.098+0000 7fb341d81740 1 -- 172.21.15.16:0/1336566809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f6424d0bb0 con 0x55f64244d250 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.099+0000 7fb317fff640 1 -- 172.21.15.16:0/1336566809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb3240043c0 con 0x55f64244d250 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.099+0000 7fb317fff640 1 -- 172.21.15.16:0/1336566809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb324005250 con 0x55f64244d250 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.099+0000 7fb341d81740 10 client.16188 fetch_fsmap finished waiting for FSMap version 27 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.099+0000 7fb341d81740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.099+0000 7fb341d81740 20 client.16188 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.099+0000 7fb341d81740 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f6424d12b0 0x55f6424d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.099+0000 7fb341d81740 4 client.16188 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.099+0000 7fb341d81740 1 -- 172.21.15.16:0/1336566809 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f6423b90f0 con 0x55f6424d12b0 2024-03-27T06:45:21.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.099+0000 7fb33cf3c640 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f6424d12b0 0x55f6424d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb317fff640 10 client.16188 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb33cf3c640 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f6424d12b0 0x55f6424d36d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=443a7132a0d2dcae server_cookie=c1e487577d543d2b in_seq=0 out_seq=0 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb317fff640 1 -- 172.21.15.16:0/1336566809 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55f6423b90f0 con 0x55f6424d12b0 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb317fff640 10 client.16188 handle_command_reply: tid=0 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb341d81740 1 client.16188 shutdown 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb341d81740 1 -- 172.21.15.16:0/1336566809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 msgr2=0x55f642466a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb341d81740 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642466a30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb324005860 tx=0x7fb3240057f0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb341d81740 1 -- 172.21.15.16:0/1336566809 shutdown_connections 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb341d81740 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55f6424d12b0 0x55f6424d36d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb341d81740 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f642456750 0x55f642463ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb341d81740 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f64244db70 0x55f642466f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb341d81740 1 --2- 172.21.15.16:0/1336566809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f64244d250 0x55f642466a30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb341d81740 1 -- 172.21.15.16:0/1336566809 >> 172.21.15.16:0/1336566809 conn(0x55f642447980 msgr2=0x55f642446090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb317fff640 0 client.16188 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb341d81740 1 -- 172.21.15.16:0/1336566809 shutdown_connections 2024-03-27T06:45:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.100+0000 7fb341d81740 1 -- 172.21.15.16:0/1336566809 wait complete. 2024-03-27T06:45:21.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb341d81740 20 client.16188 trim_cache size 0 max 16384 2024-03-27T06:45:21.101 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:45:21.101 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "scrub active (920 inodes in the stack)", 2024-03-27T06:45:21.101 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-03-27T06:45:21.101 INFO:teuthology.orchestra.run.smithi016.stdout: "5c855730-f5df-4eae-a7e3-637f53387896": { 2024-03-27T06:45:21.101 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-03-27T06:45:21.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "5c855730-f5df-4eae-a7e3-637f53387896", 2024-03-27T06:45:21.101 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-03-27T06:45:21.101 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:45:21.101 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:45:21.102 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:45:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 -- 172.21.15.16:0/294555156 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb3280a3780 msgr2=0x7fb3280a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb3280a3780 0x7fb3280a5c40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb3300efed0 tx=0x7fb33004e000 comp rx=0 tx=0).stop 2024-03-27T06:45:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 -- 172.21.15.16:0/294555156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb338107b90 msgr2=0x7fb33819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb338107b90 0x7fb33819cd80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb330009ca0 tx=0x7fb330034670 comp rx=0 tx=0).stop 2024-03-27T06:45:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 -- 172.21.15.16:0/294555156 shutdown_connections 2024-03-27T06:45:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb338107b90 0x7fb33819cd80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3381024b0 0x7fb338198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb3280a3780 0x7fb3280a5c40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 --2- 172.21.15.16:0/294555156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb338101af0 0x7fb3381984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 -- 172.21.15.16:0/294555156 >> 172.21.15.16:0/294555156 conn(0x7fb3380fd580 msgr2=0x7fb3380fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 -- 172.21.15.16:0/294555156 shutdown_connections 2024-03-27T06:45:21.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:21.101+0000 7fb33f1a0640 1 -- 172.21.15.16:0/294555156 wait complete. 2024-03-27T06:45:21.108 INFO:tasks.cephfs.filesystem:scrub status for tag:5c855730-f5df-4eae-a7e3-637f53387896 - {'path': '/', 'tag': '5c855730-f5df-4eae-a7e3-637f53387896', 'options': 'recursive,force'} 2024-03-27T06:45:21.142 INFO:tasks.workunit.client.0.smithi016.stderr:+ tar cf doc.tar linux-4.0.5 2024-03-27T06:45:21.222 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.972+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000df2 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/sentelic.txt auth v54 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b6b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000df2:[<0x10000000dd7/sentelic.txt v54>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000df2:[<0x10000000dd7/sentelic.txt v54>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.223 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.973+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000de2 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/event-codes.txt auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b6b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000de2:[<0x10000000dd7/event-codes.txt v22>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000de2:[<0x10000000dd7/event-codes.txt v22>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.223 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e[66157]: 2024-03-27T06:45:20.947+0000 7f87a0e28700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f52 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/phy/samsung-usb2.txt auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561e3c11ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000f52:[<0x10000000f51/samsung-usb2.txt v2>,<0x10000000203/phy v1842>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000f52:[<0x10000000f51/samsung-usb2.txt v2>,<0x10000000203/phy v1842>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.223 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e[66157]: 2024-03-27T06:45:21.021+0000 7f87a0e28700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001126 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/00-INDEX auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561e3c0e7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001126:[<0x10000001125/00-INDEX v2>,<0x10000000203/virtual v2098>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000001126:[<0x10000001125/00-INDEX v2>,<0x10000000203/virtual v2116>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.223 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e[66157]: 2024-03-27T06:45:21.027+0000 7f87a7635700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001127 [...4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/ auth{0=1,1=1} v41 ap=2 RANDEPHEMERALPIN f(v0 m2024-03-27T06:45:15.739699+0000 13=12+1) n(v0 rc2024-03-27T06:45:15.741699+0000 21=19+2) (inest sync->mix g=0,1) caps={25223=pAsLsXsFs/pAsLsXsFsxcral@2} | lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x561e3c0e6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000001127:[<0x10000001125/kvm v30>,<0x10000000203/virtual v2098>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x10000001127:[<0x10000001125/kvm v41>,<0x10000000203/virtual v2116>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-27T06:45:15.739699+0000 13=12+1)","ondisk_value.rstat":"n(v0 rc2024-03-27T06:45:15.739699+0000 21=19+2)","memory_value.dirstat":"f(v0 m2024-03-27T06:45:15.739699+0000 13=12+1)","memory_value.rstat":"n(v0 rc2024-03-27T06:45:15.741699+0000 21=19+2)","error_str":""},"return_code":0} 2024-03-27T06:45:21.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.973+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000de5 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gameport-programming.txt auth v28 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b6bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000de5:[<0x10000000dd7/gameport-programming.txt v28>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000de5:[<0x10000000dd7/gameport-programming.txt v28>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.973+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000dec [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-parport.txt auth v42 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b6c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000dec:[<0x10000000dd7/joystick-parport.txt v42>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000dec:[<0x10000000dd7/joystick-parport.txt v42>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.973+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000de4 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gamepad.txt auth v26 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b6c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000de4:[<0x10000000dd7/gamepad.txt v26>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000de4:[<0x10000000dd7/gamepad.txt v26>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.226 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.973+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000de1 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/elantech.txt auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b6d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000de1:[<0x10000000dd7/elantech.txt v20>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000de1:[<0x10000000dd7/elantech.txt v20>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.974+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000df1 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/rotary-encoder.txt auth v52 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b7e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000df1:[<0x10000000dd7/rotary-encoder.txt v52>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000df1:[<0x10000000dd7/rotary-encoder.txt v52>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.974+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000deb [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-api.txt auth v40 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b6cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000deb:[<0x10000000dd7/joystick-api.txt v40>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000deb:[<0x10000000dd7/joystick-api.txt v40>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.974+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000de6 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gpio-tilt.txt auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b7e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000de6:[<0x10000000dd7/gpio-tilt.txt v30>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000de6:[<0x10000000dd7/gpio-tilt.txt v30>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.974+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000df3 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/shape.fig auth v56 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b6d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000df3:[<0x10000000dd7/shape.fig v56>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000df3:[<0x10000000dd7/shape.fig v56>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.975+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ddc [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/bcm5974.txt auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b7eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ddc:[<0x10000000dd7/bcm5974.txt v10>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000ddc:[<0x10000000dd7/bcm5974.txt v10>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.975+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000de7 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/iforce-protocol.txt auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b7f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000de7:[<0x10000000dd7/iforce-protocol.txt v32>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000de7:[<0x10000000dd7/iforce-protocol.txt v32>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.975+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000de0 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/edt-ft5x06.txt auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b7f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000de0:[<0x10000000dd7/edt-ft5x06.txt v18>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000de0:[<0x10000000dd7/edt-ft5x06.txt v18>,<0x10000000203/input v1562>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.975+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000df4 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/walkera0701.txt auth v58 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b80100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000df4:[<0x10000000dd7/walkera0701.txt v58>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000df4:[<0x10000000dd7/walkera0701.txt v58>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.975+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000dee [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/multi-touch-protocol.txt auth v46 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b7fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000dee:[<0x10000000dd7/multi-touch-protocol.txt v46>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000dee:[<0x10000000dd7/multi-touch-protocol.txt v46>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.227 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.976+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ddf [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cs461x.txt auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b80680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ddf:[<0x10000000dd7/cs461x.txt v16>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000ddf:[<0x10000000dd7/cs461x.txt v16>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.228 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.976+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000dda [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/appletouch.txt auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b80c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000dda:[<0x10000000dd7/appletouch.txt v6>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000dda:[<0x10000000dd7/appletouch.txt v6>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.228 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.976+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000dd8 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/alps.txt auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b81700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000dd8:[<0x10000000dd7/alps.txt v2>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000dd8:[<0x10000000dd7/alps.txt v2>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.228 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.977+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000dde [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cma3000_d0x.txt auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b86000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000dde:[<0x10000000dd7/cma3000_d0x.txt v14>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000dde:[<0x10000000dd7/cma3000_d0x.txt v14>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.228 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.977+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000de3 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ff.txt auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b86580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000de3:[<0x10000000dd7/ff.txt v24>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000de3:[<0x10000000dd7/ff.txt v24>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.228 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.977+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000def [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/notifier.txt auth v48 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b86b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000def:[<0x10000000dd7/notifier.txt v48>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000def:[<0x10000000dd7/notifier.txt v48>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.228 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.977+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000de8 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input-programming.txt auth v34 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b87080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000de8:[<0x10000000dd7/input-programming.txt v34>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000de8:[<0x10000000dd7/input-programming.txt v34>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.228 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.977+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000df5 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/xpad.txt auth v60 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b87600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000df5:[<0x10000000dd7/xpad.txt v60>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000df5:[<0x10000000dd7/xpad.txt v60>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.228 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.977+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000df6 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/yealink.txt auth v62 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b87b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000df6:[<0x10000000dd7/yealink.txt v62>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000df6:[<0x10000000dd7/yealink.txt v62>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.228 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.978+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000dd9 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/amijoy.txt auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b88100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000dd9:[<0x10000000dd7/amijoy.txt v4>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000dd9:[<0x10000000dd7/amijoy.txt v4>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.228 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.978+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000de9 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input.txt auth v36 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b88680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000de9:[<0x10000000dd7/input.txt v36>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000de9:[<0x10000000dd7/input.txt v36>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.229 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.978+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ded [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick.txt auth v44 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b89180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ded:[<0x10000000dd7/joystick.txt v44>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000ded:[<0x10000000dd7/joystick.txt v44>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.229 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.978+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000dea [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/interactive.fig auth v38 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b89700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000dea:[<0x10000000dd7/interactive.fig v38>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000dea:[<0x10000000dd7/interactive.fig v38>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.229 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.978+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ddb [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/atarikbd.txt auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b90000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ddb:[<0x10000000dd7/atarikbd.txt v8>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000ddb:[<0x10000000dd7/atarikbd.txt v8>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.229 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.978+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ddd [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cd32.txt auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b88c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ddd:[<0x10000000dd7/cd32.txt v12>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000ddd:[<0x10000000dd7/cd32.txt v12>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.229 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.979+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000df0 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ntrig.txt auth v50 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b90580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000df0:[<0x10000000dd7/ntrig.txt v50>,<0x10000000203/input v1562>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000df0:[<0x10000000dd7/ntrig.txt v50>,<0x10000000203/input v1968>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.229 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.982+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f9e [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/rapidio.txt auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5bfc100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000f9e:[<0x10000000f9d/rapidio.txt v2>,<0x10000000203/rapidio v1858>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000f9e:[<0x10000000f9d/rapidio.txt v2>,<0x10000000203/rapidio v1954>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.229 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.982+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fa0 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/tsi721.txt auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5bfc680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000fa0:[<0x10000000f9d/tsi721.txt v6>,<0x10000000203/rapidio v1858>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000fa0:[<0x10000000f9d/tsi721.txt v6>,<0x10000000203/rapidio v1954>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.229 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.982+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f9f [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/sysfs.txt auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5bfcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000f9f:[<0x10000000f9d/sysfs.txt v4>,<0x10000000203/rapidio v1858>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000f9f:[<0x10000000f9d/sysfs.txt v4>,<0x10000000203/rapidio v1954>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.229 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.999+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e12 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.diversion auth v28 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c2a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e12:[<0x10000000e04/README.diversion v28>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e12:[<0x10000000e04/README.diversion v28>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.229 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.999+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e09 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.CAPI auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c2ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e09:[<0x10000000e04/INTERFACE.CAPI v10>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e09:[<0x10000000e04/INTERFACE.CAPI v10>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.229 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:20.999+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e17 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.icn auth v38 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c2b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e17:[<0x10000000e04/README.icn v38>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e17:[<0x10000000e04/README.icn v38>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.230 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.000+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e14 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.gigaset auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c2bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e14:[<0x10000000e04/README.gigaset v32>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e14:[<0x10000000e04/README.gigaset v32>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.230 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.000+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e1c [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.x25 auth v48 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c2b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e1c:[<0x10000000e04/README.x25 v48>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e1c:[<0x10000000e04/README.x25 v48>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.230 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.000+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e06 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/CREDITS auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c2c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e06:[<0x10000000e04/CREDITS v4>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e06:[<0x10000000e04/CREDITS v4>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.230 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.000+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e15 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hfc-pci auth v34 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c2c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e15:[<0x10000000e04/README.hfc-pci v34>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e15:[<0x10000000e04/README.hfc-pci v34>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.230 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.001+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e1d [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/syncPPP.FAQ auth v50 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c2cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e1d:[<0x10000000e04/syncPPP.FAQ v50>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e1d:[<0x10000000e04/syncPPP.FAQ v50>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.230 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.001+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e05 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/00-INDEX auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c2d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e05:[<0x10000000e04/00-INDEX v2>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e05:[<0x10000000e04/00-INDEX v2>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.230 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.001+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e10 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.avmb1 auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c2d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e10:[<0x10000000e04/README.avmb1 v24>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e10:[<0x10000000e04/README.avmb1 v24>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.230 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.001+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e0d [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.HiSax auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c3a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e0d:[<0x10000000e04/README.HiSax v18>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e0d:[<0x10000000e04/README.HiSax v18>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.230 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.001+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e1a [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.sc auth v44 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c3a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e1a:[<0x10000000e04/README.sc v44>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e1a:[<0x10000000e04/README.sc v44>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.230 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.002+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e1b [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.syncppp auth v46 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c3ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e1b:[<0x10000000e04/README.syncppp v46>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e1b:[<0x10000000e04/README.syncppp v46>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.231 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.002+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e0f [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.audio auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c3b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e0f:[<0x10000000e04/README.audio v22>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e0f:[<0x10000000e04/README.audio v22>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.231 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.002+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e16 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hysdn auth v36 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c3b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e16:[<0x10000000e04/README.hysdn v36>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e16:[<0x10000000e04/README.hysdn v36>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.231 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.002+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e07 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/HiSax.cert auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c3bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e07:[<0x10000000e04/HiSax.cert v6>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e07:[<0x10000000e04/HiSax.cert v6>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.231 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.002+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e0c [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.FAQ auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c3c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e0c:[<0x10000000e04/README.FAQ v16>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e0c:[<0x10000000e04/README.FAQ v16>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.231 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.002+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e13 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.fax auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c3c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e13:[<0x10000000e04/README.fax v30>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e13:[<0x10000000e04/README.fax v30>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.231 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.003+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e18 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.mISDN auth v40 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c3cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e18:[<0x10000000e04/README.mISDN v40>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e18:[<0x10000000e04/README.mISDN v40>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.231 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.003+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e0e [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.act2000 auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c3d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e0e:[<0x10000000e04/README.act2000 v20>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e0e:[<0x10000000e04/README.act2000 v20>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.231 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.003+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e0a [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.fax auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c3d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e0a:[<0x10000000e04/INTERFACE.fax v12>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e0a:[<0x10000000e04/INTERFACE.fax v12>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.232 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.003+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e0b [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c88000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e0b:[<0x10000000e04/README v14>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e0b:[<0x10000000e04/README v14>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.232 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.003+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e08 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c88580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e08:[<0x10000000e04/INTERFACE v8>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e08:[<0x10000000e04/INTERFACE v8>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.232 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.003+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e11 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.concap auth v26 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c89080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e11:[<0x10000000e04/README.concap v26>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e11:[<0x10000000e04/README.concap v26>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.232 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.003+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e19 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.pcbit auth v42 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c88b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e19:[<0x10000000e04/README.pcbit v42>,<0x10000000203/isdn v1644>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e19:[<0x10000000e04/README.pcbit v42>,<0x10000000203/isdn v1862>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.232 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.020+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f97 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/Makefile auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5bfd180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000f97:[<0x10000000f95/Makefile v4>,<0x10000000203/ptp v1864>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000f97:[<0x10000000f95/Makefile v4>,<0x10000000203/ptp v2108>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.020+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f9a [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.mk auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5bfd700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000f9a:[<0x10000000f95/testptp.mk v10>,<0x10000000203/ptp v1864>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000f9a:[<0x10000000f95/testptp.mk v10>,<0x10000000203/ptp v2108>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.020+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f96 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/.gitignore auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c04b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000f96:[<0x10000000f95/.gitignore v2>,<0x10000000203/ptp v1864>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000f96:[<0x10000000f95/.gitignore v2>,<0x10000000203/ptp v2108>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.020+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f99 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.c auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c04000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000f99:[<0x10000000f95/testptp.c v8>,<0x10000000203/ptp v1864>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000f99:[<0x10000000f95/testptp.c v8>,<0x10000000203/ptp v2108>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.021+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f98 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/ptp.txt auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c04580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000f98:[<0x10000000f95/ptp.txt v6>,<0x10000000203/ptp v1864>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000f98:[<0x10000000f95/ptp.txt v6>,<0x10000000203/ptp v2108>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.022+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001128 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/00-INDEX auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c06680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001128:[<0x10000001127/00-INDEX v2>,<0x10000001125/kvm v30>,<0x10000000203/virtual v2098>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000001128:[<0x10000001127/00-INDEX v2>,<0x10000001125/kvm v41>,<0x10000000203/virtual v2116>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.023+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001129 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/api.txt auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c07180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001129:[<0x10000001127/api.txt v4>,<0x10000001125/kvm v30>,<0x10000000203/virtual v2098>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000001129:[<0x10000001127/api.txt v4>,<0x10000001125/kvm v41>,<0x10000000203/virtual v2116>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.023+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000112a [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/cpuid.txt auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c07700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000112a:[<0x10000001127/cpuid.txt v6>,<0x10000001125/kvm v30>,<0x10000000203/virtual v2098>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x1000000112a:[<0x10000001127/cpuid.txt v6>,<0x10000001125/kvm v41>,<0x10000000203/virtual v2116>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.024+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000112b [...4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/ auth v22 ap=1 f(v0 m2024-03-27T06:45:15.725699+0000 7=7+0) n(v0 rc2024-03-27T06:45:15.725699+0000 8=7+1) (inest lock) (ifile excl) caps={25223=pAsLsXsFsx/pAsLsXsFsxcral@1},l=25223 | importingcaps=0 dirfrag=1 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c1a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000112b:[<0x10000001127/devices v22>,<0x10000001125/kvm v30>,<0x10000000203/virtual v2098>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x1000000112b:[<0x10000001127/devices v22>,<0x10000001125/kvm v41>,<0x10000000203/virtual v2116>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-27T06:45:15.725699+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-03-27T06:45:15.725699+0000 8=7+1)","memory_value.dirstat":"f(v0 m2024-03-27T06:45:15.725699+0000 7=7+0)","memory_value.rstat":"n(v0 rc2024-03-27T06:45:15.725699+0000 8=7+1)","error_str":""},"return_code":0} 2024-03-27T06:45:21.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.025+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000112d [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c1c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000112d:[<0x1000000112b/arm-vgic.txt v4>,<0x10000001127/devices v22>,<0x10000001125/kvm v30>,<0x10000000203/virtual v2098>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x1000000112d:[<0x1000000112b/arm-vgic.txt v4>,<0x10000001127/devices v22>,<0x10000001125/kvm v41>,<0x10000000203/virtual v2116>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.233 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.025+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000112c [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/README auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5c1cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000112c:[<0x1000000112b/README v2>,<0x10000001127/devices v22>,<0x10000001125/kvm v30>,<0x10000000203/virtual v2098>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x1000000112c:[<0x1000000112b/README v2>,<0x10000001127/devices v22>,<0x10000001125/kvm v41>,<0x10000000203/virtual v2116>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.344 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:45:20 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c[67568]: 2024-03-27T06:45:20.971+0000 7f314dd06700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e83 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/00-INDEX auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559a26c15080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e83:[<0x10000000e82/00-INDEX v2>,<0x10000000203/mips v1814>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e83:[<0x10000000e82/00-INDEX v2>,<0x10000000203/mips v1970>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.344 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:45:20 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c[67568]: 2024-03-27T06:45:20.971+0000 7f314dd06700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e84 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/AU1xxx_IDE.README auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559a26c15600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e84:[<0x10000000e82/AU1xxx_IDE.README v4>,<0x10000000203/mips v1814>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e84:[<0x10000000e82/AU1xxx_IDE.README v4>,<0x10000000203/mips v1970>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.345 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: 2024-03-27T06:45:21.012+0000 7fdf595b0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e2d [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/modules.txt auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55f98519db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e2d:[<0x10000000e26/modules.txt v14>,<0x10000000203/kbuild v1676>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e2d:[<0x10000000e26/modules.txt v14>,<0x10000000203/kbuild v2114>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.345 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: 2024-03-27T06:45:21.013+0000 7fdf595b0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e27 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/00-INDEX auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55f98519cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e27:[<0x10000000e26/00-INDEX v2>,<0x10000000203/kbuild v1676>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e27:[<0x10000000e26/00-INDEX v2>,<0x10000000203/kbuild v2114>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.345 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: 2024-03-27T06:45:21.013+0000 7fdf595b0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e2a [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig-language.txt auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55f98519d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e2a:[<0x10000000e26/kconfig-language.txt v8>,<0x10000000203/kbuild v1676>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e2a:[<0x10000000e26/kconfig-language.txt v8>,<0x10000000203/kbuild v2114>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.345 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: 2024-03-27T06:45:21.013+0000 7fdf595b0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e2b [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig.txt auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55f98519d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e2b:[<0x10000000e26/kconfig.txt v10>,<0x10000000203/kbuild v1676>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e2b:[<0x10000000e26/kconfig.txt v10>,<0x10000000203/kbuild v2114>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.345 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: 2024-03-27T06:45:21.013+0000 7fdf595b0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e29 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kbuild.txt auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55f98519e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e29:[<0x10000000e26/kbuild.txt v6>,<0x10000000203/kbuild v1676>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e29:[<0x10000000e26/kbuild.txt v6>,<0x10000000203/kbuild v2114>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.345 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: 2024-03-27T06:45:21.013+0000 7fdf595b0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e28 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/headers_install.txt auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55f98519e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e28:[<0x10000000e26/headers_install.txt v4>,<0x10000000203/kbuild v1676>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e28:[<0x10000000e26/headers_install.txt v4>,<0x10000000203/kbuild v2114>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.345 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: 2024-03-27T06:45:21.013+0000 7fdf595b0700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e2c [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/makefiles.txt auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55f98519ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000e2c:[<0x10000000e26/makefiles.txt v12>,<0x10000000203/kbuild v1676>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000000e2c:[<0x10000000e26/makefiles.txt v12>,<0x10000000203/kbuild v2114>,<0x10000000202/Documentation v325>,<0x10000000201/linux-4.0.5 v23>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.723 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.341+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001039 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b92680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001039:[<0x10000001037/DPCM.txt v4>,<0x1000000101d/soc v72>,<0x1000000101c/alsa v54>,<0x10000000203/sound v1856>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000001039:[<0x10000001037/DPCM.txt v4>,<0x1000000101d/soc v75>,<0x1000000101c/alsa v109>,<0x10000000203/sound v1966>,<0x10000000202/Documentation v587>,<0x10000000201/linux-4.0.5 v34>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.723 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.342+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000103d [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/jack.txt auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b92c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000103d:[<0x10000001037/jack.txt v12>,<0x1000000101d/soc v72>,<0x1000000101c/alsa v54>,<0x10000000203/sound v1856>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x1000000103d:[<0x10000001037/jack.txt v12>,<0x1000000101d/soc v75>,<0x1000000101c/alsa v109>,<0x10000000203/sound v1966>,<0x10000000202/Documentation v587>,<0x10000000201/linux-4.0.5 v34>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.723 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.342+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000103c [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b93180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000103c:[<0x10000001037/dapm.txt v10>,<0x1000000101d/soc v72>,<0x1000000101c/alsa v54>,<0x10000000203/sound v1856>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x1000000103c:[<0x10000001037/dapm.txt v10>,<0x1000000101d/soc v75>,<0x1000000101c/alsa v109>,<0x10000000203/sound v1966>,<0x10000000202/Documentation v587>,<0x10000000201/linux-4.0.5 v34>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.723 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.342+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000103a [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5b93700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000103a:[<0x10000001037/clocking.txt v6>,<0x1000000101d/soc v72>,<0x1000000101c/alsa v54>,<0x10000000203/sound v1856>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x1000000103a:[<0x10000001037/clocking.txt v6>,<0x1000000101d/soc v75>,<0x1000000101c/alsa v109>,<0x10000000203/sound v1966>,<0x10000000202/Documentation v587>,<0x10000000201/linux-4.0.5 v34>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.723 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.342+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001041 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5bfa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001041:[<0x10000001037/pops_clicks.txt v20>,<0x1000000101d/soc v72>,<0x1000000101c/alsa v54>,<0x10000000203/sound v1856>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000001041:[<0x10000001037/pops_clicks.txt v20>,<0x1000000101d/soc v75>,<0x1000000101c/alsa v109>,<0x10000000203/sound v1966>,<0x10000000202/Documentation v587>,<0x10000000201/linux-4.0.5 v34>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.723 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.342+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000103f [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/overview.txt auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5bfab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000103f:[<0x10000001037/overview.txt v16>,<0x1000000101d/soc v72>,<0x1000000101c/alsa v54>,<0x10000000203/sound v1856>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x1000000103f:[<0x10000001037/overview.txt v16>,<0x1000000101d/soc v75>,<0x1000000101c/alsa v109>,<0x10000000203/sound v1966>,<0x10000000202/Documentation v587>,<0x10000000201/linux-4.0.5 v34>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: pgmap v57: 97 pgs: 97 active+clean; 82 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 865 op/s 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: scrub queued for path: / 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e83 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/00-INDEX) see mds.c log and `damage ls` output for details 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e84 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/AU1xxx_IDE.README) see mds.c log and `damage ls` output for details 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000df2 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/sentelic.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000de2 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/event-codes.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000de5 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gameport-programming.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000dec (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-parport.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000de4 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gamepad.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000de1 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/elantech.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000df1 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/rotary-encoder.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000deb (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-api.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000de6 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gpio-tilt.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000df3 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/shape.fig) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000ddc (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/bcm5974.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.725 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.343+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001040 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/platform.txt auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5bfa000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001040:[<0x10000001037/platform.txt v18>,<0x1000000101d/soc v72>,<0x1000000101c/alsa v54>,<0x10000000203/sound v1856>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000001040:[<0x10000001037/platform.txt v18>,<0x1000000101d/soc v75>,<0x1000000101c/alsa v109>,<0x10000000203/sound v1966>,<0x10000000202/Documentation v587>,<0x10000000201/linux-4.0.5 v34>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.725 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.343+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001038 [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5bfb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001038:[<0x10000001037/DAI.txt v2>,<0x1000000101d/soc v72>,<0x1000000101c/alsa v54>,<0x10000000203/sound v1856>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x10000001038:[<0x10000001037/DAI.txt v2>,<0x1000000101d/soc v75>,<0x1000000101c/alsa v109>,<0x10000000203/sound v1966>,<0x10000000202/Documentation v587>,<0x10000000201/linux-4.0.5 v34>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.725 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.343+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000103e [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/machine.txt auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5bfbb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000103e:[<0x10000001037/machine.txt v14>,<0x1000000101d/soc v72>,<0x1000000101c/alsa v54>,<0x10000000203/sound v1856>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x1000000103e:[<0x10000001037/machine.txt v14>,<0x1000000101d/soc v75>,<0x1000000101c/alsa v109>,<0x10000000203/sound v1966>,<0x10000000202/Documentation v587>,<0x10000000201/linux-4.0.5 v34>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.725 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:45:21.343+0000 7f5894d15700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000103b [4,head] /volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/codec.txt auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25223=0-4194304@3} caps={25223=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=25223 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55b3e5bfb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000103b:[<0x10000001037/codec.txt v8>,<0x1000000101d/soc v72>,<0x1000000101c/alsa v54>,<0x10000000203/sound v1856>,<0x10000000202/Documentation v523>,<0x10000000201/linux-4.0.5 v29>,<0x10000000006/untar_tar v45>,<0x10000000005/tmp v43>,<0x100000001fb/client.0 v34>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v44>,<0x10000000001/sv_1 v68>,<0x10000000000/qa v58>,<0x1/volumes v56>]//[]","memoryvalue":"(3)0x1000000103b:[<0x10000001037/codec.txt v8>,<0x1000000101d/soc v75>,<0x1000000101c/alsa v109>,<0x10000000203/sound v1966>,<0x10000000202/Documentation v587>,<0x10000000201/linux-4.0.5 v34>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v45>,<0x100000001fb/client.0 v37>,<0x100000001fa/1070fb6a-f0e5-4cc6-a30e-e73dd004606a v46>,<0x10000000001/sv_1 v37>,<0x10000000000/qa v32>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-27T06:45:21.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000de7 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/iforce-protocol.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000de0 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/edt-ft5x06.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000df4 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/walkera0701.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000dee (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/multi-touch-protocol.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000ddf (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cs461x.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000dda (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/appletouch.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000dd8 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/alps.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000dde (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cma3000_d0x.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000de3 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ff.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000def (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/notifier.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000de8 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input-programming.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000df5 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/xpad.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000df6 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/yealink.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000dd9 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/amijoy.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000de9 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000ded (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000dea (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/interactive.fig) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000ddb (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/atarikbd.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000ddd (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cd32.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000df0 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ntrig.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000f9e (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/rapidio.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000fa0 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/tsi721.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000f9f (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/sysfs.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e12 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.diversion) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e09 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.CAPI) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e17 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.icn) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e14 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.gigaset) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e1c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.x25) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e06 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/CREDITS) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e15 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hfc-pci) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e1d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/syncPPP.FAQ) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e05 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/00-INDEX) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e10 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.avmb1) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e0d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.HiSax) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e1a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.sc) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e1b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.syncppp) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e0f (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.audio) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e16 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hysdn) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e07 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/HiSax.cert) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e0c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.FAQ) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e13 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.fax) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e18 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.mISDN) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e0e (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.act2000) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.729 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e0a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.fax) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e0b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e08 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e11 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.concap) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e19 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.pcbit) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.730 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:21 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: pgmap v57: 97 pgs: 97 active+clean; 82 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 865 op/s 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: scrub queued for path: / 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e83 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/00-INDEX) see mds.c log and `damage ls` output for details 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e84 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/AU1xxx_IDE.README) see mds.c log and `damage ls` output for details 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000df2 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/sentelic.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000de2 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/event-codes.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000de5 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gameport-programming.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000dec (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-parport.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000de4 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gamepad.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000de1 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/elantech.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000df1 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/rotary-encoder.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000deb (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-api.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000de6 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gpio-tilt.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000df3 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/shape.fig) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000ddc (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/bcm5974.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000de7 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/iforce-protocol.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000de0 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/edt-ft5x06.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000df4 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/walkera0701.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000dee (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/multi-touch-protocol.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000ddf (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cs461x.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000dda (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/appletouch.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000dd8 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/alps.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000dde (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cma3000_d0x.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000de3 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ff.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000def (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/notifier.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000de8 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input-programming.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000df5 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/xpad.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000df6 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/yealink.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000dd9 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/amijoy.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000de9 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000ded (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000dea (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/interactive.fig) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000ddb (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/atarikbd.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000ddd (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cd32.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000df0 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ntrig.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000f9e (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/rapidio.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000fa0 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/tsi721.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000f9f (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/sysfs.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e12 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.diversion) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e09 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.CAPI) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e17 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.icn) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e14 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.gigaset) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e1c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.x25) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e06 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/CREDITS) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e15 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hfc-pci) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e1d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/syncPPP.FAQ) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e05 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/00-INDEX) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e10 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.avmb1) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e0d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.HiSax) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e1a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.sc) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e1b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.syncppp) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e0f (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.audio) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e16 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hysdn) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e07 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/HiSax.cert) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e0c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.FAQ) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e13 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.fax) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e18 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.mISDN) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e0e (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.act2000) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e0a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.fax) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e0b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e08 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.851 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e11 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.concap) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.851 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e19 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.pcbit) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.851 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:21 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: pgmap v57: 97 pgs: 97 active+clean; 82 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 865 op/s 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: scrub queued for path: / 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e83 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/00-INDEX) see mds.c log and `damage ls` output for details 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e84 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/AU1xxx_IDE.README) see mds.c log and `damage ls` output for details 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000df2 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/sentelic.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000de2 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/event-codes.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000de5 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gameport-programming.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000dec (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-parport.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000de4 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gamepad.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000de1 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/elantech.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000df1 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/rotary-encoder.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000deb (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-api.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000de6 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gpio-tilt.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000df3 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/shape.fig) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000ddc (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/bcm5974.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000de7 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/iforce-protocol.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000de0 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/edt-ft5x06.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000df4 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/walkera0701.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000dee (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/multi-touch-protocol.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000ddf (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cs461x.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000dda (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/appletouch.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000dd8 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/alps.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000dde (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cma3000_d0x.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000de3 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ff.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000def (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/notifier.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000de8 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input-programming.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000df5 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/xpad.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000df6 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/yealink.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000dd9 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/amijoy.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000de9 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000ded (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000dea (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/interactive.fig) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000ddb (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/atarikbd.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000ddd (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cd32.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000df0 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ntrig.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000f9e (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/rapidio.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000fa0 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/tsi721.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000f9f (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/sysfs.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:21.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:21.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e12 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.diversion) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e09 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.CAPI) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e17 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.icn) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e14 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.gigaset) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e1c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.x25) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e06 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/CREDITS) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e15 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hfc-pci) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e1d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/syncPPP.FAQ) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e05 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/00-INDEX) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e10 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.avmb1) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e0d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.HiSax) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e1a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.sc) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e1b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.syncppp) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e0f (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.audio) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e16 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hysdn) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e07 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/HiSax.cert) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e0c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.FAQ) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e13 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.fax) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e18 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.mISDN) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e0e (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.act2000) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e0a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.fax) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e0b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e08 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e11 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.concap) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e19 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.pcbit) see mds.h log and `damage ls` output for details 2024-03-27T06:45:21.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:21 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000f52 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/phy/samsung-usb2.txt) see mds.e log and `damage ls` output for details 2024-03-27T06:45:22.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e2d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/modules.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e27 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/00-INDEX) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e2a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig-language.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e2b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e29 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kbuild.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e28 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/headers_install.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000e2c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/makefiles.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000f97 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/Makefile) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000f9a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.mk) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.724 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000f96 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/.gitignore) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000f99 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.c) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000000f98 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/ptp.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000001126 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/00-INDEX) see mds.e log and `damage ls` output for details 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000001128 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/00-INDEX) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000001129 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/api.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x1000000112a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/cpuid.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x1000000112b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x1000000112d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x1000000112c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/README) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000001127 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm) see mds.e log and `damage ls` output for details 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000001039 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.726 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x1000000103d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/jack.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x1000000103c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x1000000103a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000001041 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x1000000103f (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/overview.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000001040 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/platform.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x10000001038 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x1000000103e (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/machine.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: Scrub error on inode 0x1000000103b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/codec.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.727 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:22.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.728 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:22 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000f52 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/phy/samsung-usb2.txt) see mds.e log and `damage ls` output for details 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e2d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/modules.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e27 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/00-INDEX) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e2a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig-language.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e2b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e29 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kbuild.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e28 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/headers_install.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000e2c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/makefiles.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000f97 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/Makefile) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000f9a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.mk) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000f96 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/.gitignore) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000f99 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.c) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000000f98 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/ptp.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000001126 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/00-INDEX) see mds.e log and `damage ls` output for details 2024-03-27T06:45:22.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000001128 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/00-INDEX) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000001129 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/api.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x1000000112a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/cpuid.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x1000000112b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x1000000112d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x1000000112c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/README) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000001127 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm) see mds.e log and `damage ls` output for details 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000001039 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x1000000103d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/jack.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x1000000103c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x1000000103a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000001041 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x1000000103f (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/overview.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000001040 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/platform.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x10000001038 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x1000000103e (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/machine.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: Scrub error on inode 0x1000000103b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/codec.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:22 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000f52 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/phy/samsung-usb2.txt) see mds.e log and `damage ls` output for details 2024-03-27T06:45:22.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e2d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/modules.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e27 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/00-INDEX) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e2a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig-language.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e2b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e29 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kbuild.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e28 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/headers_install.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000e2c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/makefiles.txt) see mds.i log and `damage ls` output for details 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000f97 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/Makefile) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000f9a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.mk) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000f96 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/.gitignore) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000f99 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.c) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000000f98 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/ptp.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000001126 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/00-INDEX) see mds.e log and `damage ls` output for details 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000001128 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/00-INDEX) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000001129 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/api.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x1000000112a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/cpuid.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x1000000112b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x1000000112d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x1000000112c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/README) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000001127 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm) see mds.e log and `damage ls` output for details 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000001039 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x1000000103d (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/jack.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x1000000103c (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x1000000103a (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000001041 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x1000000103f (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/overview.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000001040 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/platform.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x10000001038 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x1000000103e (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/machine.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: Scrub error on inode 0x1000000103b (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/codec.txt) see mds.h log and `damage ls` output for details 2024-03-27T06:45:22.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:22.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:22 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:22.867 INFO:tasks.workunit.client.0.smithi016.stderr:+ tar xf doc.tar 2024-03-27T06:45:23.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:23 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:23.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:23 smithi156 ceph-mon[36261]: scrub summary: active paths [/] 2024-03-27T06:45:23.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:23 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:23.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:23 smithi156 ceph-mon[36261]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:23.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:23 smithi156 ceph-mon[36261]: pgmap v58: 97 pgs: 97 active+clean; 82 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 865 op/s 2024-03-27T06:45:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:23 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:23 smithi170 ceph-mon[36438]: scrub summary: active paths [/] 2024-03-27T06:45:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:23 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:23 smithi170 ceph-mon[36438]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:23 smithi170 ceph-mon[36438]: pgmap v58: 97 pgs: 97 active+clean; 82 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 865 op/s 2024-03-27T06:45:23.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:23 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:23.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:23 smithi016 ceph-mon[31908]: scrub summary: active paths [/] 2024-03-27T06:45:23.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:23 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:23.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:23 smithi016 ceph-mon[31908]: scrub summary: idle+waiting paths [/] 2024-03-27T06:45:23.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:23 smithi016 ceph-mon[31908]: pgmap v58: 97 pgs: 97 active+clean; 82 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 865 op/s 2024-03-27T06:45:24.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:24 smithi170 ceph-mon[36438]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T06:45:24.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:24 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] up:active 2024-03-27T06:45:24.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:24 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:24.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:24 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31904]: 2024-03-27T06:45:24.468+0000 7f6c4124d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T06:45:24.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:24 smithi016 ceph-mon[31908]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T06:45:24.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:24 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] up:active 2024-03-27T06:45:24.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:24 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:24.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:24 smithi156 ceph-mon[36261]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T06:45:24.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:24 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] up:active 2024-03-27T06:45:24.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:24 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:25 smithi170 ceph-mon[36438]: pgmap v59: 97 pgs: 97 active+clean; 82 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 865 op/s 2024-03-27T06:45:25.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:25 smithi016 ceph-mon[31908]: pgmap v59: 97 pgs: 97 active+clean; 82 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 865 op/s 2024-03-27T06:45:25.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:25 smithi156 ceph-mon[36261]: pgmap v59: 97 pgs: 97 active+clean; 82 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 865 op/s 2024-03-27T06:45:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:26 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] up:active 2024-03-27T06:45:26.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:26 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:26.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:26 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] up:active 2024-03-27T06:45:26.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:26 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:26.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:26 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] up:active 2024-03-27T06:45:26.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:26 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:27 smithi170 ceph-mon[36438]: pgmap v60: 97 pgs: 97 active+clean; 85 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 1.14k op/s 2024-03-27T06:45:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:27 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] up:active 2024-03-27T06:45:27.845 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:27 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:27.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:27 smithi016 ceph-mon[31908]: pgmap v60: 97 pgs: 97 active+clean; 85 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 1.14k op/s 2024-03-27T06:45:27.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:27 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] up:active 2024-03-27T06:45:27.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:27 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:27.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:27 smithi156 ceph-mon[36261]: pgmap v60: 97 pgs: 97 active+clean; 85 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 1.14k op/s 2024-03-27T06:45:27.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:27 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] up:active 2024-03-27T06:45:27.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:27 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:29 smithi170 ceph-mon[36438]: pgmap v61: 97 pgs: 97 active+clean; 85 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 679 KiB/s wr, 1.07k op/s 2024-03-27T06:45:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:29 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] up:active 2024-03-27T06:45:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:29 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:29 smithi016 ceph-mon[31908]: pgmap v61: 97 pgs: 97 active+clean; 85 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 679 KiB/s wr, 1.07k op/s 2024-03-27T06:45:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:29 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] up:active 2024-03-27T06:45:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:29 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:29.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:29 smithi156 ceph-mon[36261]: pgmap v61: 97 pgs: 97 active+clean; 85 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 679 KiB/s wr, 1.07k op/s 2024-03-27T06:45:29.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:29 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] up:active 2024-03-27T06:45:29.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:29 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:30 smithi170 ceph-mon[36438]: Health check update: 4 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T06:45:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31904]: 2024-03-27T06:45:30.513+0000 7f6c4124d700 -1 log_channel(cluster) log [ERR] : Health check update: 4 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T06:45:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:30 smithi016 ceph-mon[31908]: Health check update: 4 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T06:45:30.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:30 smithi156 ceph-mon[36261]: Health check update: 4 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-27T06:45:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.744+0000 7f890624e640 1 -- 172.21.15.16:0/1380851312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8900107ba0 msgr2=0x7f8900107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.744+0000 7f890624e640 1 --2- 172.21.15.16:0/1380851312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8900107ba0 0x7f8900107f80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009930 tx=0x7f88f002bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.744+0000 7f890624e640 1 -- 172.21.15.16:0/1380851312 shutdown_connections 2024-03-27T06:45:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.744+0000 7f890624e640 1 --2- 172.21.15.16:0/1380851312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89001024c0 0x7f890010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.744+0000 7f890624e640 1 --2- 172.21.15.16:0/1380851312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8900101b00 0x7f8900101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.744+0000 7f890624e640 1 --2- 172.21.15.16:0/1380851312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8900107ba0 0x7f8900107f80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.744+0000 7f890624e640 1 -- 172.21.15.16:0/1380851312 >> 172.21.15.16:0/1380851312 conn(0x7f89000fd610 msgr2=0x7f89000ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.744+0000 7f890624e640 1 -- 172.21.15.16:0/1380851312 shutdown_connections 2024-03-27T06:45:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.744+0000 7f890624e640 1 -- 172.21.15.16:0/1380851312 wait complete. 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.744+0000 7f890624e640 1 Processor -- start 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f890624e640 1 -- start start 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f890624e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8900101b00 0x7f8900198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f890624e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89001024c0 0x7f8900198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f890624e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900107ba0 0x7f890019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f890624e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8900111ff0 con 0x7f89001024c0 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f890624e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f8900111e70 con 0x7f8900107ba0 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f890624e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8900112170 con 0x7f8900101b00 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f88feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89001024c0 0x7f8900198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f88feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89001024c0 0x7f8900198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56642/0 (socket says 172.21.15.16:56642) 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f88feffd640 1 -- 172.21.15.16:0/1515425271 learned_addr learned my addr 172.21.15.16:0/1515425271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f88ff7fe640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8900101b00 0x7f8900198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f88fffff640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900107ba0 0x7f890019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f88feffd640 1 -- 172.21.15.16:0/1515425271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8900101b00 msgr2=0x7f8900198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f88feffd640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8900101b00 0x7f8900198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f88feffd640 1 -- 172.21.15.16:0/1515425271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900107ba0 msgr2=0x7f890019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f88feffd640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900107ba0 0x7f890019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f88feffd640 1 -- 172.21.15.16:0/1515425271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890019d440 con 0x7f89001024c0 2024-03-27T06:45:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.745+0000 7f88feffd640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89001024c0 0x7f8900198990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f88f400e9f0 tx=0x7f88f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.746+0000 7f88fcff9640 1 -- 172.21.15.16:0/1515425271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f400cd80 con 0x7f89001024c0 2024-03-27T06:45:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.746+0000 7f88fcff9640 1 -- 172.21.15.16:0/1515425271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f88f4005520 con 0x7f89001024c0 2024-03-27T06:45:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.746+0000 7f890624e640 1 -- 172.21.15.16:0/1515425271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890019d730 con 0x7f89001024c0 2024-03-27T06:45:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.746+0000 7f88fcff9640 1 -- 172.21.15.16:0/1515425271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4010690 con 0x7f89001024c0 2024-03-27T06:45:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.746+0000 7f890624e640 1 -- 172.21.15.16:0/1515425271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89001a4fc0 con 0x7f89001024c0 2024-03-27T06:45:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.748+0000 7f88fcff9640 1 -- 172.21.15.16:0/1515425271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f88f40046c0 con 0x7f89001024c0 2024-03-27T06:45:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.748+0000 7f8908e3c740 1 Processor -- start 2024-03-27T06:45:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.748+0000 7f8908e3c740 1 -- start start 2024-03-27T06:45:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.748+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 0x55e1421114d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.748+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 0x55e14211a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.748+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14211a7e0 0x55e14211cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.748+0000 7f8908e3c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e1420d7cd0 con 0x55e1421110f0 2024-03-27T06:45:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.748+0000 7f8908e3c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e14201c950 con 0x55e14211a7e0 2024-03-27T06:45:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.748+0000 7f8908e3c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e142074e50 con 0x55e142111a10 2024-03-27T06:45:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88fcff9640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f88e80a3820 0x7f88e80a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88feffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14211a7e0 0x55e14211cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 0x55e14211a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 0x55e1421114d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 0x55e14211a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:36918/0 (socket says 172.21.15.16:36918) 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88ff7fe640 1 -- 172.21.15.16:0/128413682 learned_addr learned my addr 172.21.15.16:0/128413682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88fffff640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f88e80a3820 0x7f88e80a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88fcff9640 1 -- 172.21.15.16:0/1515425271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f88f40d0570 con 0x7f89001024c0 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88ff7fe640 1 -- 172.21.15.16:0/128413682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14211a7e0 msgr2=0x55e14211cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88ff7fe640 1 --2- 172.21.15.16:0/128413682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14211a7e0 0x55e14211cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88ff7fe640 1 -- 172.21.15.16:0/128413682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 msgr2=0x55e1421114d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88ff7fe640 1 --2- 172.21.15.16:0/128413682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 0x55e1421114d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88ff7fe640 1 -- 172.21.15.16:0/128413682 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e141d89210 con 0x55e142111a10 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88ff7fe640 1 --2- 172.21.15.16:0/128413682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 0x55e14211a2a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88f0036040 tx=0x7f88f0009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=884397f9fa513395 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88fffff640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f88e80a3820 0x7f88e80a5ce0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f88f8005fd0 tx=0x7f88f8009500 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.749+0000 7f88e5ffb640 1 -- 172.21.15.16:0/128413682 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f002c710 con 0x55e142111a10 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.750+0000 7f88e5ffb640 1 -- 172.21.15.16:0/128413682 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f88f0004040 con 0x55e142111a10 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.750+0000 7f88e5ffb640 1 -- 172.21.15.16:0/128413682 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f00046c0 con 0x55e142111a10 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.750+0000 7f8908e3c740 1 -- 172.21.15.16:0/128413682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 msgr2=0x55e14211a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.750+0000 7f8908e3c740 1 --2- 172.21.15.16:0/128413682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 0x55e14211a2a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88f0036040 tx=0x7f88f0009560 comp rx=0 tx=0).stop 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.750+0000 7f8908e3c740 1 -- 172.21.15.16:0/128413682 shutdown_connections 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.750+0000 7f8908e3c740 1 --2- 172.21.15.16:0/128413682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14211a7e0 0x55e14211cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.750+0000 7f8908e3c740 1 --2- 172.21.15.16:0/128413682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 0x55e14211a2a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.750+0000 7f8908e3c740 1 --2- 172.21.15.16:0/128413682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 0x55e1421114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.750+0000 7f8908e3c740 1 -- 172.21.15.16:0/128413682 >> 172.21.15.16:0/128413682 conn(0x55e14210ba10 msgr2=0x55e14210bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.750+0000 7f8908e3c740 1 -- 172.21.15.16:0/128413682 shutdown_connections 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.750+0000 7f8908e3c740 1 -- 172.21.15.16:0/128413682 wait complete. 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f8908e3c740 1 -- start start 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f8908e3c740 1 Processor -- start 2024-03-27T06:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 0x55e142117810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 0x55e142117d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14211a7e0 0x55e142118290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f8908e3c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e142070e00 con 0x55e1421110f0 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f8908e3c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e14201d520 con 0x55e14211a7e0 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f8908e3c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e1420308e0 con 0x55e142111a10 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f8908e3c740 10 client.0 _ll_register_callbacks cb 0x55e14206ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 0x55e142117810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 0x55e142117810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56672/0 (socket says 172.21.15.16:56672) 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f88ff7fe640 1 -- 172.21.15.16:0/3734455490 learned_addr learned my addr 172.21.15.16:0/3734455490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f88feffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 0x55e142117d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.751+0000 7f88fffff640 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14211a7e0 0x55e142118290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f88ff7fe640 1 -- 172.21.15.16:0/3734455490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 msgr2=0x55e142117d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f88ff7fe640 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 0x55e142117d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f88ff7fe640 1 -- 172.21.15.16:0/3734455490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14211a7e0 msgr2=0x55e142118290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f88ff7fe640 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14211a7e0 0x55e142118290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f88ff7fe640 1 -- 172.21.15.16:0/3734455490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e141e55620 con 0x55e1421110f0 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f88ff7fe640 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 0x55e142117810 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f88f0005700 tx=0x7f88f002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cead74ae80dc6a6e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f88deffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f88deffd640 1 -- 172.21.15.16:0/3734455490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f002c9d0 con 0x55e1421110f0 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f88deffd640 1 -- 172.21.15.16:0/3734455490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f88f002cb70 con 0x55e1421110f0 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f8908e3c740 1 -- 172.21.15.16:0/3734455490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88f0004040 con 0x55e1421110f0 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f88deffd640 1 -- 172.21.15.16:0/3734455490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f88f00035f0 con 0x55e1421110f0 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f8908e3c740 10 client.16200 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f8908e3c740 1 -- 172.21.15.16:0/3734455490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e142129310 con 0x55e1421110f0 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.752+0000 7f88deffd640 1 -- 172.21.15.16:0/3734455490 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f88f0004040 con 0x55e1421110f0 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.753+0000 7f88deffd640 1 -- 172.21.15.16:0/3734455490 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f88f0004ba0 con 0x55e1421110f0 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.753+0000 7f8908e3c740 10 client.16200 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.753+0000 7f8908e3c740 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.753+0000 7f8908e3c740 20 client.16200 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.753+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e14212a660 0x55e14212ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.753+0000 7f8908e3c740 4 client.16200 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.753+0000 7f8908e3c740 1 -- 172.21.15.16:0/3734455490 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e142082390 con 0x55e14212a660 2024-03-27T06:45:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.753+0000 7f88feffd640 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e14212a660 0x55e14212ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.753+0000 7f88feffd640 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e14212a660 0x55e14212ca50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920f3cac0475ef15 server_cookie=6b4a0378e32da246 in_seq=0 out_seq=0 2024-03-27T06:45:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.753+0000 7f88deffd640 10 client.16200 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f88deffd640 1 -- 172.21.15.16:0/3734455490 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e142082390 con 0x55e14212a660 2024-03-27T06:45:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f88deffd640 10 client.16200 handle_command_reply: tid=0 2024-03-27T06:45:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 1 client.16200 shutdown 2024-03-27T06:45:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 1 -- 172.21.15.16:0/3734455490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 msgr2=0x55e142117810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 0x55e142117810 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f88f0005700 tx=0x7f88f002bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 1 -- 172.21.15.16:0/3734455490 shutdown_connections 2024-03-27T06:45:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e14212a660 0x55e14212ca50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e14211a7e0 0x55e142118290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e142111a10 0x55e142117d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f88deffd640 0 client.16200 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3734455490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1421110f0 0x55e142117810 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 1 -- 172.21.15.16:0/3734455490 >> 172.21.15.16:0/3734455490 conn(0x55e14210ba10 msgr2=0x55e14210d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 1 -- 172.21.15.16:0/3734455490 shutdown_connections 2024-03-27T06:45:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 1 -- 172.21.15.16:0/3734455490 wait complete. 2024-03-27T06:45:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.757+0000 7f8908e3c740 20 client.16200 trim_cache size 0 max 16384 2024-03-27T06:45:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.772+0000 7f8908e3c740 1 Processor -- start 2024-03-27T06:45:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.772+0000 7f8908e3c740 1 -- start start 2024-03-27T06:45:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 0x55e142119f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e142111a10 0x55e142125e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14210a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f8908e3c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89001ad780 con 0x55e14211a7e0 2024-03-27T06:45:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f8908e3c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f89001af380 con 0x55e142111a10 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f8908e3c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89001ade50 con 0x55e1421110f0 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14210a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14210a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56678/0 (socket says 172.21.15.16:56678) 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88fffff640 1 -- 172.21.15.16:0/3173217873 learned_addr learned my addr 172.21.15.16:0/3173217873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88feffd640 1 --2- 172.21.15.16:0/3173217873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e142111a10 0x55e142125e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88ff7fe640 1 --2- 172.21.15.16:0/3173217873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 0x55e142119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88fffff640 1 -- 172.21.15.16:0/3173217873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 msgr2=0x55e142119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88fffff640 1 --2- 172.21.15.16:0/3173217873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 0x55e142119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88fffff640 1 -- 172.21.15.16:0/3173217873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e142111a10 msgr2=0x55e142125e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88fffff640 1 --2- 172.21.15.16:0/3173217873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e142111a10 0x55e142125e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88fffff640 1 -- 172.21.15.16:0/3173217873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14211cbd0 con 0x55e14211a7e0 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88fffff640 1 --2- 172.21.15.16:0/3173217873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14210a8f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f88f800fde0 tx=0x7f88f8008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da036924c3cdc3d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88e57fa640 1 -- 172.21.15.16:0/3173217873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8011d30 con 0x55e14211a7e0 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88e57fa640 1 -- 172.21.15.16:0/3173217873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f88f8010930 con 0x55e14211a7e0 2024-03-27T06:45:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.773+0000 7f88e57fa640 1 -- 172.21.15.16:0/3173217873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8020a20 con 0x55e14211a7e0 2024-03-27T06:45:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.774+0000 7f8908e3c740 1 -- 172.21.15.16:0/3173217873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 msgr2=0x55e14210a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.774+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3173217873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14210a8f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f88f800fde0 tx=0x7f88f8008000 comp rx=0 tx=0).stop 2024-03-27T06:45:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.774+0000 7f8908e3c740 1 -- 172.21.15.16:0/3173217873 shutdown_connections 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.774+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3173217873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14210a8f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.774+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3173217873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e142111a10 0x55e142125e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.774+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3173217873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 0x55e142119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.774+0000 7f8908e3c740 1 -- 172.21.15.16:0/3173217873 >> 172.21.15.16:0/3173217873 conn(0x55e14210ba10 msgr2=0x55e14210a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.774+0000 7f8908e3c740 1 -- 172.21.15.16:0/3173217873 shutdown_connections 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.774+0000 7f8908e3c740 1 -- 172.21.15.16:0/3173217873 wait complete. 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.774+0000 7f8908e3c740 1 -- start start 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f8908e3c740 1 Processor -- start 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 0x55e142122670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e142111a10 0x55e14211d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f8908e3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14211df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f8908e3c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89001aecb0 con 0x55e14211a7e0 2024-03-27T06:45:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f8908e3c740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f89001ae520 con 0x55e142111a10 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f8908e3c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89001ab760 con 0x55e1421110f0 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f8908e3c740 10 client.0 _ll_register_callbacks cb 0x7f88f8009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14211df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 0x55e142122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 0x55e142122670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:36938/0 (socket says 172.21.15.16:36938) 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14211df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56692/0 (socket says 172.21.15.16:56692) 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88feffd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e142111a10 0x55e14211d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88ff7fe640 1 -- 172.21.15.16:0/3891224029 learned_addr learned my addr 172.21.15.16:0/3891224029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88fffff640 1 -- 172.21.15.16:0/3891224029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 msgr2=0x55e142122670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88fffff640 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 0x55e142122670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88fffff640 1 -- 172.21.15.16:0/3891224029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e142111a10 msgr2=0x55e14211d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88fffff640 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e142111a10 0x55e14211d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88fffff640 1 -- 172.21.15.16:0/3891224029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e14211e720 con 0x55e14211a7e0 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88feffd640 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e142111a10 0x55e14211d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.775+0000 7f88ff7fe640 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 0x55e142122670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f88fffff640 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14211df20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f88f800fc20 tx=0x7f88f8019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f0589772d1ed2f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f88deffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f88deffd640 1 -- 172.21.15.16:0/3891224029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8010ae0 con 0x55e14211a7e0 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f88deffd640 1 -- 172.21.15.16:0/3891224029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f88f8010c80 con 0x55e14211a7e0 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f8908e3c740 1 -- 172.21.15.16:0/3891224029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88f8010930 con 0x55e14211a7e0 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f88deffd640 1 -- 172.21.15.16:0/3891224029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f88f8010e80 con 0x55e14211a7e0 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f8908e3c740 10 client.16206 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f8908e3c740 1 -- 172.21.15.16:0/3891224029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e142190690 con 0x55e14211a7e0 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f88deffd640 1 -- 172.21.15.16:0/3891224029 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f88f8010930 con 0x55e14211a7e0 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f88deffd640 1 -- 172.21.15.16:0/3891224029 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f88f801d070 con 0x55e14211a7e0 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f8908e3c740 10 client.16206 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f8908e3c740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f8908e3c740 20 client.16206 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e142190e10 0x55e142193230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f8908e3c740 4 client.16206 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:45:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.776+0000 7f8908e3c740 1 -- 172.21.15.16:0/3891224029 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e14207c6e0 con 0x55e142190e10 2024-03-27T06:45:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.777+0000 7f88ff7fe640 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e142190e10 0x55e142193230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.777+0000 7f88deffd640 10 client.16206 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:31.777+0000 7f88ff7fe640 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e142190e10 0x55e142193230 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6677c171669cb0a9 server_cookie=fd711fca25fba63 in_seq=0 out_seq=0 2024-03-27T06:45:31.843 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:31 smithi170 ceph-mon[36438]: pgmap v62: 97 pgs: 97 active+clean; 128 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 1.41k op/s 2024-03-27T06:45:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:31 smithi016 ceph-mon[31908]: pgmap v62: 97 pgs: 97 active+clean; 128 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 1.41k op/s 2024-03-27T06:45:31.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:31 smithi156 ceph-mon[36261]: pgmap v62: 97 pgs: 97 active+clean; 128 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 1.41k op/s 2024-03-27T06:45:32.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:32 smithi170 ceph-mon[36438]: pgmap v63: 97 pgs: 97 active+clean; 129 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 714 op/s 2024-03-27T06:45:32.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:32 smithi016 ceph-mon[31908]: pgmap v63: 97 pgs: 97 active+clean; 129 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 714 op/s 2024-03-27T06:45:32.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:32 smithi156 ceph-mon[36261]: pgmap v63: 97 pgs: 97 active+clean; 129 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 714 op/s 2024-03-27T06:45:35.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:34 smithi156 ceph-mon[36261]: pgmap v64: 97 pgs: 97 active+clean; 129 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 714 op/s 2024-03-27T06:45:35.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:34 smithi156 ceph-mon[36261]: scrub summary: idle 2024-03-27T06:45:35.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:34 smithi156 ceph-mon[36261]: scrub summary: idle 2024-03-27T06:45:35.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:34 smithi156 ceph-mon[36261]: scrub summary: idle 2024-03-27T06:45:35.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:34 smithi170 ceph-mon[36438]: pgmap v64: 97 pgs: 97 active+clean; 129 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 714 op/s 2024-03-27T06:45:35.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:34 smithi170 ceph-mon[36438]: scrub summary: idle 2024-03-27T06:45:35.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:34 smithi170 ceph-mon[36438]: scrub summary: idle 2024-03-27T06:45:35.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:34 smithi170 ceph-mon[36438]: scrub summary: idle 2024-03-27T06:45:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:34 smithi016 ceph-mon[31908]: pgmap v64: 97 pgs: 97 active+clean; 129 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 714 op/s 2024-03-27T06:45:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:34 smithi016 ceph-mon[31908]: scrub summary: idle 2024-03-27T06:45:35.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:34 smithi016 ceph-mon[31908]: scrub summary: idle 2024-03-27T06:45:35.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:34 smithi016 ceph-mon[31908]: scrub summary: idle 2024-03-27T06:45:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.873+0000 7f88deffd640 1 -- 172.21.15.16:0/3891224029 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e14207c6e0 con 0x55e142190e10 2024-03-27T06:45:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f88deffd640 10 client.16206 handle_command_reply: tid=0 2024-03-27T06:45:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 1 client.16206 shutdown 2024-03-27T06:45:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 1 -- 172.21.15.16:0/3891224029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 msgr2=0x55e14211df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14211df20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f88f800fc20 tx=0x7f88f8019040 comp rx=0 tx=0).stop 2024-03-27T06:45:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 1 -- 172.21.15.16:0/3891224029 shutdown_connections 2024-03-27T06:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e142190e10 0x55e142193230 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e14211a7e0 0x55e14211df20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e142111a10 0x55e14211d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 1 --2- 172.21.15.16:0/3891224029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1421110f0 0x55e142122670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 1 -- 172.21.15.16:0/3891224029 >> 172.21.15.16:0/3891224029 conn(0x55e14210ba10 msgr2=0x55e142118040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f88deffd640 0 client.16206 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 1 -- 172.21.15.16:0/3891224029 shutdown_connections 2024-03-27T06:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 1 -- 172.21.15.16:0/3891224029 wait complete. 2024-03-27T06:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.874+0000 7f8908e3c740 20 client.16206 trim_cache size 0 max 16384 2024-03-27T06:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:45:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:45:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:45:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 -- 172.21.15.16:0/1515425271 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f88e80a3820 msgr2=0x7f88e80a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f88e80a3820 0x7f88e80a5ce0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f88f8005fd0 tx=0x7f88f8009500 comp rx=0 tx=0).stop 2024-03-27T06:45:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 -- 172.21.15.16:0/1515425271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89001024c0 msgr2=0x7f8900198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89001024c0 0x7f8900198990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f88f400e9f0 tx=0x7f88f400edf0 comp rx=0 tx=0).stop 2024-03-27T06:45:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 -- 172.21.15.16:0/1515425271 shutdown_connections 2024-03-27T06:45:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900107ba0 0x7f890019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89001024c0 0x7f8900198990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f88e80a3820 0x7f88e80a5ce0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 --2- 172.21.15.16:0/1515425271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8900101b00 0x7f8900198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 -- 172.21.15.16:0/1515425271 >> 172.21.15.16:0/1515425271 conn(0x7f89000fd610 msgr2=0x7f89000fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 -- 172.21.15.16:0/1515425271 shutdown_connections 2024-03-27T06:45:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:35.875+0000 7f890624e640 1 -- 172.21.15.16:0/1515425271 wait complete. 2024-03-27T06:45:36.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:35 smithi170 ceph-mon[36438]: scrub summary: idle 2024-03-27T06:45:36.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:35 smithi170 ceph-mon[36438]: scrub summary: idle 2024-03-27T06:45:36.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:35 smithi156 ceph-mon[36261]: scrub summary: idle 2024-03-27T06:45:36.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:35 smithi156 ceph-mon[36261]: scrub summary: idle 2024-03-27T06:45:36.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:35 smithi016 ceph-mon[31908]: scrub summary: idle 2024-03-27T06:45:36.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:35 smithi016 ceph-mon[31908]: scrub summary: idle 2024-03-27T06:45:36.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:36 smithi156 ceph-mon[36261]: pgmap v65: 97 pgs: 97 active+clean; 144 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 831 op/s 2024-03-27T06:45:37.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:36 smithi170 ceph-mon[36438]: pgmap v65: 97 pgs: 97 active+clean; 144 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 831 op/s 2024-03-27T06:45:37.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:36 smithi016 ceph-mon[31908]: pgmap v65: 97 pgs: 97 active+clean; 144 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 831 op/s 2024-03-27T06:45:39.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:38 smithi156 ceph-mon[36261]: pgmap v66: 97 pgs: 97 active+clean; 144 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 475 op/s 2024-03-27T06:45:39.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:38 smithi170 ceph-mon[36438]: pgmap v66: 97 pgs: 97 active+clean; 144 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 475 op/s 2024-03-27T06:45:39.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:38 smithi016 ceph-mon[31908]: pgmap v66: 97 pgs: 97 active+clean; 144 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 475 op/s 2024-03-27T06:45:41.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:40 smithi156 ceph-mon[36261]: pgmap v67: 97 pgs: 97 active+clean; 131 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 799 op/s 2024-03-27T06:45:41.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:40 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:45:41.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:40 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:41.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:40 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:40 smithi170 ceph-mon[36438]: pgmap v67: 97 pgs: 97 active+clean; 131 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 799 op/s 2024-03-27T06:45:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:40 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:45:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:40 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:41.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:40 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:40 smithi016 ceph-mon[31908]: pgmap v67: 97 pgs: 97 active+clean; 131 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 799 op/s 2024-03-27T06:45:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:40 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:45:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:40 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:41.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:40 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:42.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:42 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:42.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:42 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:42.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:42 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:45:42.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:42 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:45:42.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:42 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:42.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:42 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:42.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:42 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:42.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:42 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:45:42.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:42 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:45:42.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:42 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:42.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:42 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:42.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:42 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:42.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:42 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:45:42.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:42 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:45:42.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:42 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:45:43.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:43 smithi170 ceph-mon[36438]: pgmap v68: 97 pgs: 97 active+clean; 131 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 518 op/s 2024-03-27T06:45:43.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:43 smithi016 ceph-mon[31908]: pgmap v68: 97 pgs: 97 active+clean; 131 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 518 op/s 2024-03-27T06:45:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:43 smithi156 ceph-mon[36261]: pgmap v68: 97 pgs: 97 active+clean; 131 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 518 op/s 2024-03-27T06:45:45.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:45 smithi170 ceph-mon[36438]: pgmap v69: 97 pgs: 97 active+clean; 131 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 499 op/s 2024-03-27T06:45:45.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:45 smithi016 ceph-mon[31908]: pgmap v69: 97 pgs: 97 active+clean; 131 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 499 op/s 2024-03-27T06:45:45.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:45 smithi156 ceph-mon[36261]: pgmap v69: 97 pgs: 97 active+clean; 131 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 499 op/s 2024-03-27T06:45:47.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:47 smithi170 ceph-mon[36438]: pgmap v70: 97 pgs: 97 active+clean; 132 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 569 op/s 2024-03-27T06:45:47.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:47 smithi016 ceph-mon[31908]: pgmap v70: 97 pgs: 97 active+clean; 132 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 569 op/s 2024-03-27T06:45:47.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:47 smithi156 ceph-mon[36261]: pgmap v70: 97 pgs: 97 active+clean; 132 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 569 op/s 2024-03-27T06:45:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.968+0000 7ff75924e640 1 -- 172.21.15.16:0/2719286219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff754101660 msgr2=0x7ff75410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.968+0000 7ff75924e640 1 --2- 172.21.15.16:0/2719286219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff754101660 0x7ff75410d880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff744009ca0 tx=0x7ff74402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.968+0000 7ff75924e640 1 -- 172.21.15.16:0/2719286219 shutdown_connections 2024-03-27T06:45:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.968+0000 7ff75924e640 1 --2- 172.21.15.16:0/2719286219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff754101660 0x7ff75410d880 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.968+0000 7ff75924e640 1 --2- 172.21.15.16:0/2719286219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff754100ca0 0x7ff754101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.968+0000 7ff75924e640 1 --2- 172.21.15.16:0/2719286219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7541059f0 0x7ff754105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.968+0000 7ff75924e640 1 -- 172.21.15.16:0/2719286219 >> 172.21.15.16:0/2719286219 conn(0x7ff7540778a0 msgr2=0x7ff754077cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.969+0000 7ff75924e640 1 -- 172.21.15.16:0/2719286219 shutdown_connections 2024-03-27T06:45:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.969+0000 7ff75924e640 1 -- 172.21.15.16:0/2719286219 wait complete. 2024-03-27T06:45:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.969+0000 7ff75924e640 1 Processor -- start 2024-03-27T06:45:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.969+0000 7ff75924e640 1 -- start start 2024-03-27T06:45:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff75924e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff754100ca0 0x7ff754198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff75924e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff754101660 0x7ff7541989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff75924e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7541059f0 0x7ff75419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff75924e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff754111030 con 0x7ff754100ca0 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff75924e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ff754110eb0 con 0x7ff7541059f0 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff75924e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541111b0 con 0x7ff754101660 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff752d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff754100ca0 0x7ff754198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff752d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff754100ca0 0x7ff754198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60792/0 (socket says 172.21.15.16:60792) 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff752d9d640 1 -- 172.21.15.16:0/2441255212 learned_addr learned my addr 172.21.15.16:0/2441255212 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff75259c640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff754101660 0x7ff7541989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff75359e640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7541059f0 0x7ff75419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff752d9d640 1 -- 172.21.15.16:0/2441255212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff754101660 msgr2=0x7ff7541989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff752d9d640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff754101660 0x7ff7541989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff752d9d640 1 -- 172.21.15.16:0/2441255212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7541059f0 msgr2=0x7ff75419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff752d9d640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7541059f0 0x7ff75419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff752d9d640 1 -- 172.21.15.16:0/2441255212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75419d450 con 0x7ff754100ca0 2024-03-27T06:45:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff752d9d640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff754100ca0 0x7ff754198460 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e9f0 tx=0x7ff74c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff74bfff640 1 -- 172.21.15.16:0/2441255212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c00cd80 con 0x7ff754100ca0 2024-03-27T06:45:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff75924e640 1 -- 172.21.15.16:0/2441255212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75419d740 con 0x7ff754100ca0 2024-03-27T06:45:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff74bfff640 1 -- 172.21.15.16:0/2441255212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff74c005520 con 0x7ff754100ca0 2024-03-27T06:45:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff74bfff640 1 -- 172.21.15.16:0/2441255212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c010690 con 0x7ff754100ca0 2024-03-27T06:45:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.970+0000 7ff75924e640 1 -- 172.21.15.16:0/2441255212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7541a4fd0 con 0x7ff754100ca0 2024-03-27T06:45:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.972+0000 7ff74bfff640 1 -- 172.21.15.16:0/2441255212 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7ff74c0046c0 con 0x7ff754100ca0 2024-03-27T06:45:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75be33740 1 Processor -- start 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75be33740 1 -- start start 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75be33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec3bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75be33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec496b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff74bfff640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff7400a3820 0x7ff7400a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75be33740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec4bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75be33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55685ec02cd0 con 0x55685ec3bc10 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75be33740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55685eb47950 con 0x55685ec49bf0 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75be33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55685eb9fe50 con 0x55685ec3c530 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec3bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75259c640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff7400a3820 0x7ff7400a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec3bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60798/0 (socket says 172.21.15.16:60798) 2024-03-27T06:45:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75359e640 1 -- 172.21.15.16:0/1533808208 learned_addr learned my addr 172.21.15.16:0/1533808208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff752d9d640 1 --2- 172.21.15.16:0/1533808208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff75359e640 1 --2- 172.21.15.16:0/1533808208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec4bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.973+0000 7ff74bfff640 1 -- 172.21.15.16:0/2441255212 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7ff74c0d0510 con 0x7ff754100ca0 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.974+0000 7ff752d9d640 1 -- 172.21.15.16:0/1533808208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 msgr2=0x55685ec4bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.974+0000 7ff752d9d640 1 --2- 172.21.15.16:0/1533808208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec4bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.974+0000 7ff752d9d640 1 -- 172.21.15.16:0/1533808208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 msgr2=0x55685ec3bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.974+0000 7ff752d9d640 1 --2- 172.21.15.16:0/1533808208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec3bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.974+0000 7ff752d9d640 1 -- 172.21.15.16:0/1533808208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55685e8b4210 con 0x55685ec3c530 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.974+0000 7ff75359e640 1 --2- 172.21.15.16:0/1533808208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec3bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.974+0000 7ff75259c640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff7400a3820 0x7ff7400a5ce0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff754077e30 tx=0x7ff73c005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.974+0000 7ff752d9d640 1 --2- 172.21.15.16:0/1533808208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec496b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0cdc90 tx=0x7ff74c098970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc2609b519fa2146 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.974+0000 7ff7497fa640 1 -- 172.21.15.16:0/1533808208 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c09c210 con 0x55685ec3c530 2024-03-27T06:45:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.974+0000 7ff7497fa640 1 -- 172.21.15.16:0/1533808208 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff74c09cdd0 con 0x55685ec3c530 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.974+0000 7ff7497fa640 1 -- 172.21.15.16:0/1533808208 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c09c030 con 0x55685ec3c530 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.975+0000 7ff75be33740 1 -- 172.21.15.16:0/1533808208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 msgr2=0x55685ec496b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.975+0000 7ff75be33740 1 --2- 172.21.15.16:0/1533808208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec496b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0cdc90 tx=0x7ff74c098970 comp rx=0 tx=0).stop 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.975+0000 7ff75be33740 1 -- 172.21.15.16:0/1533808208 shutdown_connections 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.975+0000 7ff75be33740 1 --2- 172.21.15.16:0/1533808208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec4bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.975+0000 7ff75be33740 1 --2- 172.21.15.16:0/1533808208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec496b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.975+0000 7ff75be33740 1 --2- 172.21.15.16:0/1533808208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec3bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.975+0000 7ff75be33740 1 -- 172.21.15.16:0/1533808208 >> 172.21.15.16:0/1533808208 conn(0x55685ec36a10 msgr2=0x55685ec36df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.975+0000 7ff75be33740 1 -- 172.21.15.16:0/1533808208 shutdown_connections 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.975+0000 7ff75be33740 1 -- 172.21.15.16:0/1533808208 wait complete. 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.975+0000 7ff75be33740 1 -- start start 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.975+0000 7ff75be33740 1 Processor -- start 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75be33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3bc10 0x55685ec46c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75be33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3c530 0x55685ec47190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75be33740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75be33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55685eb9be00 con 0x55685ec3c530 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75be33740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55685eb48520 con 0x55685ec49bf0 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75be33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55685eb5b8e0 con 0x55685ec3bc10 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75be33740 10 client.0 _ll_register_callbacks cb 0x55685eb96a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3c530 0x55685ec47190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75359e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff752d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3bc10 0x55685ec46c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3c530 0x55685ec47190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60812/0 (socket says 172.21.15.16:60812) 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75259c640 1 -- 172.21.15.16:0/1364024321 learned_addr learned my addr 172.21.15.16:0/1364024321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75259c640 1 -- 172.21.15.16:0/1364024321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3bc10 msgr2=0x55685ec46c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75259c640 1 --2- 172.21.15.16:0/1364024321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3bc10 0x55685ec46c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75259c640 1 -- 172.21.15.16:0/1364024321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 msgr2=0x55685ec476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75259c640 1 --2- 172.21.15.16:0/1364024321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75259c640 1 -- 172.21.15.16:0/1364024321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55685e980620 con 0x55685ec3c530 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff75259c640 1 --2- 172.21.15.16:0/1364024321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3c530 0x55685ec47190 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00ec00 tx=0x7ff73c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6b50d47f659096 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff73a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff73a7fc640 1 -- 172.21.15.16:0/1364024321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c004220 con 0x55685ec3c530 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.976+0000 7ff73a7fc640 1 -- 172.21.15.16:0/1364024321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7ff73c0043c0 con 0x55685ec3c530 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff75be33740 1 -- 172.21.15.16:0/1364024321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff74c09cdd0 con 0x55685ec3c530 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff73a7fc640 1 -- 172.21.15.16:0/1364024321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff73c01c8a0 con 0x55685ec3c530 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff75be33740 10 client.16218 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff75be33740 1 -- 172.21.15.16:0/1364024321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55685ec587c0 con 0x55685ec3c530 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff73a7fc640 1 -- 172.21.15.16:0/1364024321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff74c09cdd0 con 0x55685ec3c530 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff73a7fc640 1 -- 172.21.15.16:0/1364024321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7ff73c0304e0 con 0x55685ec3c530 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff75be33740 10 client.16218 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff75be33740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff75be33740 20 client.16218 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff75be33740 1 --2- 172.21.15.16:0/1364024321 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55685ec59d40 0x55685ec5c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff75be33740 4 client.16218 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff75be33740 1 -- 172.21.15.16:0/1364024321 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55685ebad390 con 0x55685ec59d40 2024-03-27T06:45:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.977+0000 7ff752d9d640 1 --2- 172.21.15.16:0/1364024321 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55685ec59d40 0x55685ec5c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.978+0000 7ff752d9d640 1 --2- 172.21.15.16:0/1364024321 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55685ec59d40 0x55685ec5c160 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729386e31cace463 server_cookie=b221485e85aa0ab9 in_seq=0 out_seq=0 2024-03-27T06:45:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.978+0000 7ff73a7fc640 10 client.16218 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.979+0000 7ff73a7fc640 1 -- 172.21.15.16:0/1364024321 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55685ebad390 con 0x55685ec59d40 2024-03-27T06:45:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff73a7fc640 10 client.16218 handle_command_reply: tid=0 2024-03-27T06:45:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 1 client.16218 shutdown 2024-03-27T06:45:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 1 -- 172.21.15.16:0/1364024321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3c530 msgr2=0x55685ec47190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 1 --2- 172.21.15.16:0/1364024321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3c530 0x55685ec47190 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00ec00 tx=0x7ff73c004730 comp rx=0 tx=0).stop 2024-03-27T06:45:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 1 -- 172.21.15.16:0/1364024321 shutdown_connections 2024-03-27T06:45:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 1 --2- 172.21.15.16:0/1364024321 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55685ec59d40 0x55685ec5c160 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 1 --2- 172.21.15.16:0/1364024321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 1 --2- 172.21.15.16:0/1364024321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3c530 0x55685ec47190 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 1 --2- 172.21.15.16:0/1364024321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3bc10 0x55685ec46c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff73a7fc640 0 client.16218 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 1 -- 172.21.15.16:0/1364024321 >> 172.21.15.16:0/1364024321 conn(0x55685ec36a10 msgr2=0x55685ec38760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 1 -- 172.21.15.16:0/1364024321 shutdown_connections 2024-03-27T06:45:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 1 -- 172.21.15.16:0/1364024321 wait complete. 2024-03-27T06:45:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.980+0000 7ff75be33740 20 client.16218 trim_cache size 0 max 16384 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff75be33740 1 Processor -- start 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff75be33740 1 -- start start 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff75be33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff75be33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec55230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff75be33740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff75be33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541ab5f0 con 0x55685ec3bc10 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff75be33740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541ad1f0 con 0x55685ec49bf0 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff75be33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541abcc0 con 0x55685ec3c530 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff752d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff752d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec315f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60828/0 (socket says 172.21.15.16:60828) 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff752d9d640 1 -- 172.21.15.16:0/37440300 learned_addr learned my addr 172.21.15.16:0/37440300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff75259c640 1 --2- 172.21.15.16:0/37440300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec55230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff75359e640 1 --2- 172.21.15.16:0/37440300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff752d9d640 1 -- 172.21.15.16:0/37440300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 msgr2=0x55685ec55230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff752d9d640 1 --2- 172.21.15.16:0/37440300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec55230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff752d9d640 1 -- 172.21.15.16:0/37440300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 msgr2=0x55685ec474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff752d9d640 1 --2- 172.21.15.16:0/37440300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.995+0000 7ff752d9d640 1 -- 172.21.15.16:0/37440300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55685ec45c00 con 0x55685ec3bc10 2024-03-27T06:45:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.996+0000 7ff752d9d640 1 --2- 172.21.15.16:0/37440300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec315f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0cdd40 tx=0x7ff74c004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30318cb749554847 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.996+0000 7ff748ff9640 1 -- 172.21.15.16:0/37440300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c09c6d0 con 0x55685ec3bc10 2024-03-27T06:45:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.996+0000 7ff748ff9640 1 -- 172.21.15.16:0/37440300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff74c09cdd0 con 0x55685ec3bc10 2024-03-27T06:45:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.996+0000 7ff748ff9640 1 -- 172.21.15.16:0/37440300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c09cb90 con 0x55685ec3bc10 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.996+0000 7ff75be33740 1 -- 172.21.15.16:0/37440300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 msgr2=0x55685ec315f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.996+0000 7ff75be33740 1 --2- 172.21.15.16:0/37440300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec315f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0cdd40 tx=0x7ff74c004c10 comp rx=0 tx=0).stop 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 -- 172.21.15.16:0/37440300 shutdown_connections 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 --2- 172.21.15.16:0/37440300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 --2- 172.21.15.16:0/37440300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec55230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 --2- 172.21.15.16:0/37440300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec315f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 -- 172.21.15.16:0/37440300 >> 172.21.15.16:0/37440300 conn(0x55685ec36a10 msgr2=0x55685ec35a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 -- 172.21.15.16:0/37440300 shutdown_connections 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 -- 172.21.15.16:0/37440300 wait complete. 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 -- start start 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 Processor -- start 2024-03-27T06:45:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec51780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec51cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec4dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541acb20 con 0x55685ec3bc10 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541ac390 con 0x55685ec49bf0 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541a95d0 con 0x55685ec3c530 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75be33740 10 client.0 _ll_register_callbacks cb 0x7ff73c005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff752d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec51780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff752d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec51780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60830/0 (socket says 172.21.15.16:60830) 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff752d9d640 1 -- 172.21.15.16:0/309518551 learned_addr learned my addr 172.21.15.16:0/309518551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75259c640 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec51cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff75359e640 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec4dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.997+0000 7ff752d9d640 1 -- 172.21.15.16:0/309518551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 msgr2=0x55685ec51cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff752d9d640 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec51cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff752d9d640 1 -- 172.21.15.16:0/309518551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 msgr2=0x55685ec4dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff752d9d640 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec4dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff752d9d640 1 -- 172.21.15.16:0/309518551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55685ec4e500 con 0x55685ec3bc10 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff752d9d640 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec51780 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0cdc90 tx=0x7ff74c098aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6b7897ffca178c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff73a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff73a7fc640 1 -- 172.21.15.16:0/309518551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c09c6d0 con 0x55685ec3bc10 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff73a7fc640 1 -- 172.21.15.16:0/309518551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7ff74c0bb300 con 0x55685ec3bc10 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff75be33740 1 -- 172.21.15.16:0/309518551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff74c09cdd0 con 0x55685ec3bc10 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff73a7fc640 1 -- 172.21.15.16:0/309518551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff74c0a0a80 con 0x55685ec3bc10 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff75be33740 10 client.16224 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff75be33740 1 -- 172.21.15.16:0/309518551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55685ecc0cf0 con 0x55685ec3bc10 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff73a7fc640 1 -- 172.21.15.16:0/309518551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff74c09cdd0 con 0x55685ec3bc10 2024-03-27T06:45:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff73a7fc640 1 -- 172.21.15.16:0/309518551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7ff74c0b5270 con 0x55685ec3bc10 2024-03-27T06:45:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff75be33740 10 client.16224 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff75be33740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff75be33740 20 client.16224 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff75be33740 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55685ecc0f70 0x55685ecc3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff75be33740 4 client.16224 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:45:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.998+0000 7ff75be33740 1 -- 172.21.15.16:0/309518551 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55685eba76e0 con 0x55685ecc0f70 2024-03-27T06:45:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.999+0000 7ff75259c640 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55685ecc0f70 0x55685ecc3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.999+0000 7ff73a7fc640 10 client.16224 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:48.999+0000 7ff75259c640 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55685ecc0f70 0x55685ecc3360 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d42729f2ddcce359 server_cookie=63668a15344e5746 in_seq=0 out_seq=0 2024-03-27T06:45:49.333 INFO:tasks.workunit.client.0.smithi016.stderr:+ sync 2024-03-27T06:45:49.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:49 smithi170 ceph-mon[36438]: pgmap v71: 97 pgs: 97 active+clean; 132 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-03-27T06:45:49.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:49 smithi016 ceph-mon[31908]: pgmap v71: 97 pgs: 97 active+clean; 132 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-03-27T06:45:49.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:49 smithi156 ceph-mon[36261]: pgmap v71: 97 pgs: 97 active+clean; 132 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-03-27T06:45:51.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-27T06:45:51.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-27T06:45:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.233+0000 7f0f5741c640 1 -- 172.21.15.16:0/2515052804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50102480 msgr2=0x7f0f5010e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.233+0000 7f0f5741c640 1 --2- 172.21.15.16:0/2515052804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50102480 0x7f0f5010e990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009cd0 tx=0x7f0f4802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.233+0000 7f0f5741c640 1 -- 172.21.15.16:0/2515052804 shutdown_connections 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.233+0000 7f0f5741c640 1 --2- 172.21.15.16:0/2515052804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50102480 0x7f0f5010e990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.233+0000 7f0f5741c640 1 --2- 172.21.15.16:0/2515052804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f50101ac0 0x7f0f50101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.233+0000 7f0f5741c640 1 --2- 172.21.15.16:0/2515052804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f50107ad0 0x7f0f50107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.233+0000 7f0f5741c640 1 -- 172.21.15.16:0/2515052804 >> 172.21.15.16:0/2515052804 conn(0x7f0f500fd5b0 msgr2=0x7f0f500ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.233+0000 7f0f5741c640 1 -- 172.21.15.16:0/2515052804 shutdown_connections 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.233+0000 7f0f5741c640 1 -- 172.21.15.16:0/2515052804 wait complete. 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f5741c640 1 Processor -- start 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f5741c640 1 -- start start 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f5741c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f50101ac0 0x7f0f501984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f5741c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50102480 0x7f0f50198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f5741c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f50107ad0 0x7f0f5019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f5741c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50112000 con 0x7f0f50102480 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f5741c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50111e80 con 0x7f0f50107ad0 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f5741c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50112180 con 0x7f0f50101ac0 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f549b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50102480 0x7f0f50198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f551b8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f50101ac0 0x7f0f501984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f549b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50102480 0x7f0f50198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60846/0 (socket says 172.21.15.16:60846) 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f549b7640 1 -- 172.21.15.16:0/3705412693 learned_addr learned my addr 172.21.15.16:0/3705412693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f559b9640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f50107ad0 0x7f0f5019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f549b7640 1 -- 172.21.15.16:0/3705412693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f50101ac0 msgr2=0x7f0f501984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f549b7640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f50101ac0 0x7f0f501984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f549b7640 1 -- 172.21.15.16:0/3705412693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f50107ad0 msgr2=0x7f0f5019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f549b7640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f50107ad0 0x7f0f5019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.234+0000 7f0f549b7640 1 -- 172.21.15.16:0/3705412693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5019d4b0 con 0x7f0f50102480 2024-03-27T06:45:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.235+0000 7f0f549b7640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50102480 0x7f0f50198a00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000e9f0 tx=0x7f0f4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.235+0000 7f0f467fc640 1 -- 172.21.15.16:0/3705412693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4000cd80 con 0x7f0f50102480 2024-03-27T06:45:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.235+0000 7f0f467fc640 1 -- 172.21.15.16:0/3705412693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0f40010930 con 0x7f0f50102480 2024-03-27T06:45:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.235+0000 7f0f5741c640 1 -- 172.21.15.16:0/3705412693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5019d7a0 con 0x7f0f50102480 2024-03-27T06:45:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.235+0000 7f0f467fc640 1 -- 172.21.15.16:0/3705412693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40018b00 con 0x7f0f50102480 2024-03-27T06:45:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.235+0000 7f0f5741c640 1 -- 172.21.15.16:0/3705412693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f501a5030 con 0x7f0f50102480 2024-03-27T06:45:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.237+0000 7f0f467fc640 1 -- 172.21.15.16:0/3705412693 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f0f40018ca0 con 0x7f0f50102480 2024-03-27T06:45:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.237+0000 7f0f5a009740 1 Processor -- start 2024-03-27T06:45:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.237+0000 7f0f5a009740 1 -- start start 2024-03-27T06:45:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f467fc640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0f3c0a3800 0x7f0f3c0a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7e250 0x55ae5db7e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db7eb70 0x55ae5db87210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db87750 0x55ae5db89b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f5a009740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae5dae2860 con 0x55ae5db7eb70 2024-03-27T06:45:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f5a009740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55ae5da8a8d0 con 0x55ae5db87750 2024-03-27T06:45:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f5a009740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ae5dade810 con 0x55ae5db7e250 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f467fc640 1 -- 172.21.15.16:0/3705412693 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f0f400d03f0 con 0x7f0f50102480 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f549b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db7eb70 0x55ae5db87210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f559b9640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0f3c0a3800 0x7f0f3c0a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f551b8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7e250 0x55ae5db7e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f549b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db7eb70 0x55ae5db87210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60856/0 (socket says 172.21.15.16:60856) 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f549b7640 1 -- 172.21.15.16:0/3955617997 learned_addr learned my addr 172.21.15.16:0/3955617997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f559b9640 1 --2- 172.21.15.16:0/3955617997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db87750 0x55ae5db89b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f549b7640 1 -- 172.21.15.16:0/3955617997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7e250 msgr2=0x55ae5db7e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f549b7640 1 --2- 172.21.15.16:0/3955617997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7e250 0x55ae5db7e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f549b7640 1 -- 172.21.15.16:0/3955617997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db87750 msgr2=0x55ae5db89b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f549b7640 1 --2- 172.21.15.16:0/3955617997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db87750 0x55ae5db89b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f549b7640 1 -- 172.21.15.16:0/3955617997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae5da95120 con 0x55ae5db7eb70 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f551b8640 1 --2- 172.21.15.16:0/3955617997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7e250 0x55ae5db7e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f549b7640 1 --2- 172.21.15.16:0/3955617997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db7eb70 0x55ae5db87210 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0f400a4cf0 tx=0x7f0f40011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676b30d0bef88018 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.238+0000 7f0f559b9640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0f3c0a3800 0x7f0f3c0a5cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f48005cc0 tx=0x7f0f48005f80 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f3b7fe640 1 -- 172.21.15.16:0/3955617997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f400a0960 con 0x55ae5db7eb70 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f3b7fe640 1 -- 172.21.15.16:0/3955617997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0f400a5220 con 0x55ae5db7eb70 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f3b7fe640 1 -- 172.21.15.16:0/3955617997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40010ae0 con 0x55ae5db7eb70 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f5a009740 1 -- 172.21.15.16:0/3955617997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db7eb70 msgr2=0x55ae5db87210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f5a009740 1 --2- 172.21.15.16:0/3955617997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db7eb70 0x55ae5db87210 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0f400a4cf0 tx=0x7f0f40011f50 comp rx=0 tx=0).stop 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f5a009740 1 -- 172.21.15.16:0/3955617997 shutdown_connections 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f5a009740 1 --2- 172.21.15.16:0/3955617997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db87750 0x55ae5db89b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f5a009740 1 --2- 172.21.15.16:0/3955617997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db7eb70 0x55ae5db87210 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f5a009740 1 --2- 172.21.15.16:0/3955617997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7e250 0x55ae5db7e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f5a009740 1 -- 172.21.15.16:0/3955617997 >> 172.21.15.16:0/3955617997 conn(0x55ae5db78980 msgr2=0x55ae5db78d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f5a009740 1 -- 172.21.15.16:0/3955617997 shutdown_connections 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.239+0000 7f0f5a009740 1 -- 172.21.15.16:0/3955617997 wait complete. 2024-03-27T06:45:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f5a009740 1 -- start start 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f5a009740 1 Processor -- start 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7e250 0x55ae5db84780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7eb70 0x55ae5db84cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5db85200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f5a009740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae5db8a540 con 0x55ae5db87750 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f5a009740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55ae5da9dcd0 con 0x55ae5db7eb70 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f5a009740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ae5db8a6c0 con 0x55ae5db7e250 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f5a009740 10 client.0 _ll_register_callbacks cb 0x55ae5da8fee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f559b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5db85200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f559b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5db85200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60860/0 (socket says 172.21.15.16:60860) 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f551b8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7e250 0x55ae5db84780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f559b9640 1 -- 172.21.15.16:0/126302968 learned_addr learned my addr 172.21.15.16:0/126302968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.240+0000 7f0f549b7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7eb70 0x55ae5db84cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f559b9640 1 -- 172.21.15.16:0/126302968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7e250 msgr2=0x55ae5db84780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f559b9640 1 --2- 172.21.15.16:0/126302968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7e250 0x55ae5db84780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f559b9640 1 -- 172.21.15.16:0/126302968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7eb70 msgr2=0x55ae5db84cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f559b9640 1 --2- 172.21.15.16:0/126302968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7eb70 0x55ae5db84cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f559b9640 1 -- 172.21.15.16:0/126302968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae5d8c07d0 con 0x55ae5db87750 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f559b9640 1 --2- 172.21.15.16:0/126302968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5db85200 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009cd0 tx=0x7f0f4802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4149ad559336c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f38ff9640 1 -- 172.21.15.16:0/126302968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48051060 con 0x55ae5db87750 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f38ff9640 1 -- 172.21.15.16:0/126302968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f0f48004450 con 0x55ae5db87750 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f5a009740 1 -- 172.21.15.16:0/126302968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f400a5220 con 0x55ae5db87750 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f38ff9640 1 -- 172.21.15.16:0/126302968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0f48039040 con 0x55ae5db87750 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f5a009740 10 client.16236 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.241+0000 7f0f5a009740 1 -- 172.21.15.16:0/126302968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ae5d7f63f0 con 0x55ae5db87750 2024-03-27T06:45:51.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.242+0000 7f0f38ff9640 1 -- 172.21.15.16:0/126302968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0f400a5220 con 0x55ae5db87750 2024-03-27T06:45:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.242+0000 7f0f38ff9640 1 -- 172.21.15.16:0/126302968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f0f4804c410 con 0x55ae5db87750 2024-03-27T06:45:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.242+0000 7f0f5a009740 10 client.16236 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.242+0000 7f0f5a009740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.242+0000 7f0f5a009740 20 client.16236 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.242+0000 7f0f5a009740 1 --2- 172.21.15.16:0/126302968 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae5db97bf0 0x55ae5db9a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.242+0000 7f0f5a009740 4 client.16236 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.242+0000 7f0f5a009740 1 -- 172.21.15.16:0/126302968 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae5daefda0 con 0x55ae5db97bf0 2024-03-27T06:45:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.242+0000 7f0f551b8640 1 --2- 172.21.15.16:0/126302968 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae5db97bf0 0x55ae5db9a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.242+0000 7f0f551b8640 1 --2- 172.21.15.16:0/126302968 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae5db97bf0 0x55ae5db9a010 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6f3d2272708c55e server_cookie=ee4f8feb19a8a183 in_seq=0 out_seq=0 2024-03-27T06:45:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.242+0000 7f0f38ff9640 10 client.16236 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:51.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:51 smithi170 ceph-mon[36438]: pgmap v72: 97 pgs: 97 active+clean; 136 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 579 op/s 2024-03-27T06:45:51.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:51 smithi016 ceph-mon[31908]: pgmap v72: 97 pgs: 97 active+clean; 136 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 579 op/s 2024-03-27T06:45:51.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:51 smithi156 ceph-mon[36261]: pgmap v72: 97 pgs: 97 active+clean; 136 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 579 op/s 2024-03-27T06:45:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff73a7fc640 1 -- 172.21.15.16:0/309518551 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55685eba76e0 con 0x55685ecc0f70 2024-03-27T06:45:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff73a7fc640 10 client.16224 handle_command_reply: tid=0 2024-03-27T06:45:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 1 client.16224 shutdown 2024-03-27T06:45:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f38ff9640 1 -- 172.21.15.16:0/126302968 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ae5daefda0 con 0x55ae5db97bf0 2024-03-27T06:45:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f38ff9640 10 client.16236 handle_command_reply: tid=0 2024-03-27T06:45:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f5a009740 1 client.16236 shutdown 2024-03-27T06:45:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 1 -- 172.21.15.16:0/309518551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 msgr2=0x55685ec51780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec51780 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0cdc90 tx=0x7ff74c098aa0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 1 -- 172.21.15.16:0/309518551 shutdown_connections 2024-03-27T06:45:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55685ecc0f70 0x55685ecc3360 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685ec49bf0 0x55685ec4dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55685ec3c530 0x55685ec51cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff73a7fc640 0 client.16224 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 1 --2- 172.21.15.16:0/309518551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55685ec3bc10 0x55685ec51780 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 1 -- 172.21.15.16:0/309518551 >> 172.21.15.16:0/309518551 conn(0x55685ec36a10 msgr2=0x55685ec37e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:51.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f5a009740 1 -- 172.21.15.16:0/126302968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 msgr2=0x55ae5db85200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f5a009740 1 --2- 172.21.15.16:0/126302968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5db85200 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009cd0 tx=0x7f0f4802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 1 -- 172.21.15.16:0/309518551 shutdown_connections 2024-03-27T06:45:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 1 -- 172.21.15.16:0/309518551 wait complete. 2024-03-27T06:45:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f5a009740 1 -- 172.21.15.16:0/126302968 shutdown_connections 2024-03-27T06:45:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f5a009740 1 --2- 172.21.15.16:0/126302968 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae5db97bf0 0x55ae5db9a010 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f5a009740 1 --2- 172.21.15.16:0/126302968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5db85200 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f5a009740 1 --2- 172.21.15.16:0/126302968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7eb70 0x55ae5db84cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f38ff9640 0 client.16236 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f5a009740 1 --2- 172.21.15.16:0/126302968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7e250 0x55ae5db84780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f5a009740 1 -- 172.21.15.16:0/126302968 >> 172.21.15.16:0/126302968 conn(0x55ae5db78980 msgr2=0x55ae5db7a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.525+0000 7ff75be33740 20 client.16224 trim_cache size 0 max 16384 2024-03-27T06:45:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f5a009740 1 -- 172.21.15.16:0/126302968 shutdown_connections 2024-03-27T06:45:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.525+0000 7f0f5a009740 1 -- 172.21.15.16:0/126302968 wait complete. 2024-03-27T06:45:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:45:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:45:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:45:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:45:51.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.526+0000 7f0f5a009740 20 client.16236 trim_cache size 0 max 16384 2024-03-27T06:45:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 -- 172.21.15.16:0/2441255212 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff7400a3820 msgr2=0x7ff7400a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff7400a3820 0x7ff7400a5ce0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff754077e30 tx=0x7ff73c005f90 comp rx=0 tx=0).stop 2024-03-27T06:45:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 -- 172.21.15.16:0/2441255212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff754100ca0 msgr2=0x7ff754198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff754100ca0 0x7ff754198460 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e9f0 tx=0x7ff74c00edf0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 -- 172.21.15.16:0/2441255212 shutdown_connections 2024-03-27T06:45:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7541059f0 0x7ff75419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff7400a3820 0x7ff7400a5ce0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff754101660 0x7ff7541989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 --2- 172.21.15.16:0/2441255212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff754100ca0 0x7ff754198460 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 -- 172.21.15.16:0/2441255212 >> 172.21.15.16:0/2441255212 conn(0x7ff7540778a0 msgr2=0x7ff7540692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 -- 172.21.15.16:0/2441255212 shutdown_connections 2024-03-27T06:45:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:45:51.526+0000 7ff75924e640 1 -- 172.21.15.16:0/2441255212 wait complete. 2024-03-27T06:45:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.544+0000 7f0f5a009740 1 Processor -- start 2024-03-27T06:45:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.544+0000 7f0f5a009740 1 -- start start 2024-03-27T06:45:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.544+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 0x55ae5db86eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.544+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 0x55ae5db932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.544+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5db9f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.544+0000 7f0f5a009740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501ad8a0 con 0x55ae5db87750 2024-03-27T06:45:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.544+0000 7f0f5a009740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501af4a0 con 0x55ae5db7e250 2024-03-27T06:45:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.544+0000 7f0f5a009740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501adf70 con 0x55ae5db7eb70 2024-03-27T06:45:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f551b8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 0x55ae5db86eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f549b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 0x55ae5db932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f549b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 0x55ae5db932e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:48272/0 (socket says 172.21.15.16:48272) 2024-03-27T06:45:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f559b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5db9f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f549b7640 1 -- 172.21.15.16:0/636078298 learned_addr learned my addr 172.21.15.16:0/636078298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f559b9640 1 -- 172.21.15.16:0/636078298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 msgr2=0x55ae5db932e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f559b9640 1 --2- 172.21.15.16:0/636078298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 0x55ae5db932e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f559b9640 1 -- 172.21.15.16:0/636078298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 msgr2=0x55ae5db86eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f559b9640 1 --2- 172.21.15.16:0/636078298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 0x55ae5db86eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f559b9640 1 -- 172.21.15.16:0/636078298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae5d7b60f0 con 0x55ae5db87750 2024-03-27T06:45:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f549b7640 1 --2- 172.21.15.16:0/636078298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 0x55ae5db932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.545+0000 7f0f559b9640 1 --2- 172.21.15.16:0/636078298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5db9f900 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802c700 tx=0x7f0f48032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3185e90856f56c0e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.546+0000 7f0f3affd640 1 -- 172.21.15.16:0/636078298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48051040 con 0x55ae5db87750 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.546+0000 7f0f3affd640 1 -- 172.21.15.16:0/636078298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0f48004750 con 0x55ae5db87750 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.546+0000 7f0f3affd640 1 -- 172.21.15.16:0/636078298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4805e810 con 0x55ae5db87750 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.546+0000 7f0f5a009740 1 -- 172.21.15.16:0/636078298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 msgr2=0x55ae5db9f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.546+0000 7f0f5a009740 1 --2- 172.21.15.16:0/636078298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5db9f900 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802c700 tx=0x7f0f48032bc0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.546+0000 7f0f5a009740 1 -- 172.21.15.16:0/636078298 shutdown_connections 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.546+0000 7f0f5a009740 1 --2- 172.21.15.16:0/636078298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5db9f900 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.546+0000 7f0f5a009740 1 --2- 172.21.15.16:0/636078298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 0x55ae5db932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.546+0000 7f0f5a009740 1 --2- 172.21.15.16:0/636078298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 0x55ae5db86eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.546+0000 7f0f5a009740 1 -- 172.21.15.16:0/636078298 >> 172.21.15.16:0/636078298 conn(0x55ae5db78980 msgr2=0x55ae5db77920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.547+0000 7f0f5a009740 1 -- 172.21.15.16:0/636078298 shutdown_connections 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.547+0000 7f0f5a009740 1 -- 172.21.15.16:0/636078298 wait complete. 2024-03-27T06:45:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.547+0000 7f0f5a009740 1 -- start start 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.547+0000 7f0f5a009740 1 Processor -- start 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.547+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 0x55ae5db81b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.547+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 0x55ae5db82040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.547+0000 7f0f5a009740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5dbfcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.547+0000 7f0f5a009740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501aedd0 con 0x55ae5db87750 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.547+0000 7f0f5a009740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501ae640 con 0x55ae5db7e250 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.547+0000 7f0f5a009740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501ab880 con 0x55ae5db7eb70 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.547+0000 7f0f5a009740 10 client.0 _ll_register_callbacks cb 0x7f0f40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f551b8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 0x55ae5db81b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f551b8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 0x55ae5db81b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:45176/0 (socket says 172.21.15.16:45176) 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f559b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5dbfcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f551b8640 1 -- 172.21.15.16:0/3844742455 learned_addr learned my addr 172.21.15.16:0/3844742455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f549b7640 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 0x55ae5db82040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f559b9640 1 -- 172.21.15.16:0/3844742455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 msgr2=0x55ae5db82040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f559b9640 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 0x55ae5db82040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f559b9640 1 -- 172.21.15.16:0/3844742455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 msgr2=0x55ae5db81b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f559b9640 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 0x55ae5db81b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f559b9640 1 -- 172.21.15.16:0/3844742455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae5dbfd4f0 con 0x55ae5db87750 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f551b8640 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 0x55ae5db81b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f559b9640 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5dbfcf30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0f4804c830 tx=0x7f0f48004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8421184e2103439f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f38ff9640 1 -- 172.21.15.16:0/3844742455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48063060 con 0x55ae5db87750 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f38ff9640 1 -- 172.21.15.16:0/3844742455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f0f4805e5b0 con 0x55ae5db87750 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f5a009740 1 -- 172.21.15.16:0/3844742455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f48004750 con 0x55ae5db87750 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f38ff9640 1 -- 172.21.15.16:0/3844742455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0f4804c570 con 0x55ae5db87750 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.548+0000 7f0f5a009740 10 client.16242 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f5a009740 1 -- 172.21.15.16:0/3844742455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ae5dbfe800 con 0x55ae5db87750 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f38ff9640 1 -- 172.21.15.16:0/3844742455 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0f48004750 con 0x55ae5db87750 2024-03-27T06:45:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f38ff9640 1 -- 172.21.15.16:0/3844742455 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f0f48068580 con 0x55ae5db87750 2024-03-27T06:45:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f5a009740 10 client.16242 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f5a009740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f5a009740 20 client.16242 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f5a009740 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae5dbfeef0 0x55ae5dc01310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f5a009740 4 client.16242 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-27T06:45:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f5a009740 1 -- 172.21.15.16:0/3844742455 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae5daea0f0 con 0x55ae5dbfeef0 2024-03-27T06:45:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f551b8640 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae5dbfeef0 0x55ae5dc01310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f551b8640 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae5dbfeef0 0x55ae5dc01310 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abde4408eb130316 server_cookie=389ae24f9e95187d in_seq=0 out_seq=0 2024-03-27T06:45:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.549+0000 7f0f38ff9640 10 client.16242 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f38ff9640 1 -- 172.21.15.16:0/3844742455 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ae5daea0f0 con 0x55ae5dbfeef0 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f38ff9640 10 client.16242 handle_command_reply: tid=0 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 1 client.16242 shutdown 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 1 -- 172.21.15.16:0/3844742455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 msgr2=0x55ae5dbfcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5dbfcf30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0f4804c830 tx=0x7f0f48004690 comp rx=0 tx=0).stop 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 1 -- 172.21.15.16:0/3844742455 shutdown_connections 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55ae5dbfeef0 0x55ae5dc01310 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae5db87750 0x55ae5dbfcf30 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae5db7eb70 0x55ae5db82040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f38ff9640 0 client.16242 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 1 --2- 172.21.15.16:0/3844742455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ae5db7e250 0x55ae5db81b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 1 -- 172.21.15.16:0/3844742455 >> 172.21.15.16:0/3844742455 conn(0x55ae5db78980 msgr2=0x55ae5db79d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 1 -- 172.21.15.16:0/3844742455 shutdown_connections 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 1 -- 172.21.15.16:0/3844742455 wait complete. 2024-03-27T06:45:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.552+0000 7f0f5a009740 20 client.16242 trim_cache size 0 max 16384 2024-03-27T06:45:51.551 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:45:51.551 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-03-27T06:45:51.551 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-03-27T06:45:51.551 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:45:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 -- 172.21.15.16:0/3705412693 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0f3c0a3800 msgr2=0x7f0f3c0a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0f3c0a3800 0x7f0f3c0a5cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f48005cc0 tx=0x7f0f48005f80 comp rx=0 tx=0).stop 2024-03-27T06:45:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 -- 172.21.15.16:0/3705412693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50102480 msgr2=0x7f0f50198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50102480 0x7f0f50198a00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000e9f0 tx=0x7f0f4000eeb0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 -- 172.21.15.16:0/3705412693 shutdown_connections 2024-03-27T06:45:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f50107ad0 0x7f0f5019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50102480 0x7f0f50198a00 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0f3c0a3800 0x7f0f3c0a5cc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 --2- 172.21.15.16:0/3705412693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f50101ac0 0x7f0f501984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 -- 172.21.15.16:0/3705412693 >> 172.21.15.16:0/3705412693 conn(0x7f0f500fd5b0 msgr2=0x7f0f500fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 -- 172.21.15.16:0/3705412693 shutdown_connections 2024-03-27T06:45:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.553+0000 7f0f5741c640 1 -- 172.21.15.16:0/3705412693 wait complete. 2024-03-27T06:45:51.559 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-27T06:45:51.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:45:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 -- 172.21.15.16:0/3984568469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4628100050 msgr2=0x7f46281004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 --2- 172.21.15.16:0/3984568469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4628100050 0x7f46281004d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f461c009900 tx=0x7f461c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 -- 172.21.15.16:0/3984568469 shutdown_connections 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 --2- 172.21.15.16:0/3984568469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4628100a10 0x7f4628104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 --2- 172.21.15.16:0/3984568469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4628100050 0x7f46281004d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 --2- 172.21.15.16:0/3984568469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4628106a50 0x7f4628106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 -- 172.21.15.16:0/3984568469 >> 172.21.15.16:0/3984568469 conn(0x7f4628076fe0 msgr2=0x7f46280fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 -- 172.21.15.16:0/3984568469 shutdown_connections 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 -- 172.21.15.16:0/3984568469 wait complete. 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 Processor -- start 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 -- start start 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4628100050 0x7f46281a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4628100a10 0x7f46281a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4628106a50 0x7f462819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4628118600 con 0x7f4628106a50 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f4628118480 con 0x7f4628100a10 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.675+0000 7f462fc64640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4628118780 con 0x7f4628100050 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462e201640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4628106a50 0x7f462819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462da00640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4628100050 0x7f46281a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462d1ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4628100a10 0x7f46281a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462da00640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4628100050 0x7f46281a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:48300/0 (socket says 172.21.15.16:48300) 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462da00640 1 -- 172.21.15.16:0/2681006120 learned_addr learned my addr 172.21.15.16:0/2681006120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462da00640 1 -- 172.21.15.16:0/2681006120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4628100a10 msgr2=0x7f46281a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462da00640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4628100a10 0x7f46281a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462da00640 1 -- 172.21.15.16:0/2681006120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4628106a50 msgr2=0x7f462819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462da00640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4628106a50 0x7f462819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462da00640 1 -- 172.21.15.16:0/2681006120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462819f000 con 0x7f4628100050 2024-03-27T06:45:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462e201640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4628106a50 0x7f462819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:51.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462da00640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4628100050 0x7f46281a46d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4620009d50 tx=0x7f462000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:51.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f4626ffd640 1 -- 172.21.15.16:0/2681006120 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620003b70 con 0x7f4628100050 2024-03-27T06:45:51.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f4626ffd640 1 -- 172.21.15.16:0/2681006120 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4620003d10 con 0x7f4628100050 2024-03-27T06:45:51.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462fc64640 1 -- 172.21.15.16:0/2681006120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462819f2f0 con 0x7f4628100050 2024-03-27T06:45:51.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f4626ffd640 1 -- 172.21.15.16:0/2681006120 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620010a90 con 0x7f4628100050 2024-03-27T06:45:51.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.676+0000 7f462fc64640 1 -- 172.21.15.16:0/2681006120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46281ab590 con 0x7f4628100050 2024-03-27T06:45:51.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.677+0000 7f462fc64640 1 -- 172.21.15.16:0/2681006120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4628068d90 con 0x7f4628100050 2024-03-27T06:45:51.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.678+0000 7f4626ffd640 1 -- 172.21.15.16:0/2681006120 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f4620004a40 con 0x7f4628100050 2024-03-27T06:45:51.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.679+0000 7f4626ffd640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f46100a3800 0x7f46100a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:51.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.679+0000 7f462d1ff640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f46100a3800 0x7f46100a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:51.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.679+0000 7f4626ffd640 1 -- 172.21.15.16:0/2681006120 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f46200d0c70 con 0x7f4628100050 2024-03-27T06:45:51.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.679+0000 7f462d1ff640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f46100a3800 0x7f46100a5cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f461c006010 tx=0x7f461c005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:51.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.683+0000 7f4626ffd640 1 -- 172.21.15.16:0/2681006120 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f462009d530 con 0x7f4628100050 2024-03-27T06:45:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.908+0000 7f462fc64640 1 -- 172.21.15.16:0/2681006120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f462819fdb0 con 0x7f4628100050 2024-03-27T06:45:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.909+0000 7f4626ffd640 1 -- 172.21.15.16:0/2681006120 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11940 (secure 0 0 0) 0x7f462009ce30 con 0x7f4628100050 2024-03-27T06:45:51.904 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:45:51.904 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:45:28.505075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":67,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":65,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":65,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":64,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25852":{"gid":25852,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/681514902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":681514902},{"type":"v1","addr":"172.21.15.170:6835","nonce":681514902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527,24467,24500,24493,24472]},"id":1}]} 2024-03-27T06:45:51.904 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 31 2024-03-27T06:45:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.911+0000 7f462fc64640 1 -- 172.21.15.16:0/2681006120 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f46100a3800 msgr2=0x7f46100a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.911+0000 7f462fc64640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f46100a3800 0x7f46100a5cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f461c006010 tx=0x7f461c005e40 comp rx=0 tx=0).stop 2024-03-27T06:45:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.911+0000 7f462fc64640 1 -- 172.21.15.16:0/2681006120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4628100050 msgr2=0x7f46281a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.911+0000 7f462fc64640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4628100050 0x7f46281a46d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4620009d50 tx=0x7f462000c6a0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.911+0000 7f462fc64640 1 -- 172.21.15.16:0/2681006120 shutdown_connections 2024-03-27T06:45:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.911+0000 7f462fc64640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4628106a50 0x7f462819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.911+0000 7f462fc64640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4628100a10 0x7f46281a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.911+0000 7f462fc64640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f46100a3800 0x7f46100a5cc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.911+0000 7f462fc64640 1 --2- 172.21.15.16:0/2681006120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4628100050 0x7f46281a46d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.911+0000 7f462fc64640 1 -- 172.21.15.16:0/2681006120 >> 172.21.15.16:0/2681006120 conn(0x7f4628076fe0 msgr2=0x7f4628102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.911+0000 7f462fc64640 1 -- 172.21.15.16:0/2681006120 shutdown_connections 2024-03-27T06:45:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:51.912+0000 7f462fc64640 1 -- 172.21.15.16:0/2681006120 wait complete. 2024-03-27T06:45:51.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-27T06:45:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.027+0000 7faa12276640 1 -- 172.21.15.16:0/318152849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c101af0 msgr2=0x7faa0c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.027+0000 7faa12276640 1 --2- 172.21.15.16:0/318152849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c101af0 0x7faa0c101f70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009890 tx=0x7fa9f402bb70 comp rx=0 tx=0).stop 2024-03-27T06:45:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.027+0000 7faa12276640 1 -- 172.21.15.16:0/318152849 shutdown_connections 2024-03-27T06:45:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.027+0000 7faa12276640 1 --2- 172.21.15.16:0/318152849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c1024b0 0x7faa0c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.027+0000 7faa12276640 1 --2- 172.21.15.16:0/318152849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c101af0 0x7faa0c101f70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.027+0000 7faa12276640 1 --2- 172.21.15.16:0/318152849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa0c107b90 0x7faa0c107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.027+0000 7faa12276640 1 -- 172.21.15.16:0/318152849 >> 172.21.15.16:0/318152849 conn(0x7faa0c0fd580 msgr2=0x7faa0c0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.027+0000 7faa12276640 1 -- 172.21.15.16:0/318152849 shutdown_connections 2024-03-27T06:45:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.027+0000 7faa12276640 1 -- 172.21.15.16:0/318152849 wait complete. 2024-03-27T06:45:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa12276640 1 Processor -- start 2024-03-27T06:45:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa12276640 1 -- start start 2024-03-27T06:45:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa12276640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c101af0 0x7faa0c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa12276640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c1024b0 0x7faa0c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa12276640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa0c107b90 0x7faa0c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa12276640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c112030 con 0x7faa0c101af0 2024-03-27T06:45:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa12276640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c111eb0 con 0x7faa0c107b90 2024-03-27T06:45:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa12276640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1121b0 con 0x7faa0c1024b0 2024-03-27T06:45:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c101af0 0x7faa0c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c101af0 0x7faa0c198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60910/0 (socket says 172.21.15.16:60910) 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa0bfff640 1 -- 172.21.15.16:0/2476142218 learned_addr learned my addr 172.21.15.16:0/2476142218 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa0b7fe640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c1024b0 0x7faa0c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.028+0000 7faa10813640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa0c107b90 0x7faa0c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.029+0000 7faa0bfff640 1 -- 172.21.15.16:0/2476142218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c1024b0 msgr2=0x7faa0c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.029+0000 7faa0bfff640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c1024b0 0x7faa0c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.029+0000 7faa0bfff640 1 -- 172.21.15.16:0/2476142218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa0c107b90 msgr2=0x7faa0c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.029+0000 7faa0bfff640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa0c107b90 0x7faa0c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.029+0000 7faa0bfff640 1 -- 172.21.15.16:0/2476142218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c19d540 con 0x7faa0c101af0 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.029+0000 7faa0bfff640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c101af0 0x7faa0c198550 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00e9f0 tx=0x7fa9fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.029+0000 7faa097fa640 1 -- 172.21.15.16:0/2476142218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc00cd80 con 0x7faa0c101af0 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.029+0000 7faa12276640 1 -- 172.21.15.16:0/2476142218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c19d830 con 0x7faa0c101af0 2024-03-27T06:45:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.029+0000 7faa12276640 1 -- 172.21.15.16:0/2476142218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c1a50c0 con 0x7faa0c101af0 2024-03-27T06:45:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.029+0000 7faa097fa640 1 -- 172.21.15.16:0/2476142218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa9fc005520 con 0x7faa0c101af0 2024-03-27T06:45:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.029+0000 7faa097fa640 1 -- 172.21.15.16:0/2476142218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc018650 con 0x7faa0c101af0 2024-03-27T06:45:52.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.031+0000 7faa097fa640 1 -- 172.21.15.16:0/2476142218 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fa9fc0046c0 con 0x7faa0c101af0 2024-03-27T06:45:52.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.031+0000 7faa14e63740 1 Processor -- start 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.031+0000 7faa14e63740 1 -- start start 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa097fa640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9f80a3780 0x7fa9f80a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa14e63740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eecbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa14e63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa14e63740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eed70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa14e63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56217ee2f710 con 0x56217eecc0d0 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa14e63740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x56217edd7af0 con 0x56217eecb7b0 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa14e63740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56217ee2b6c0 con 0x56217eed4cb0 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa10813640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9f80a3780 0x7fa9f80a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa0b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa0bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eecbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa0b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60920/0 (socket says 172.21.15.16:60920) 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa10813640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eed70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa0b7fe640 1 -- 172.21.15.16:0/1340941617 learned_addr learned my addr 172.21.15.16:0/1340941617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa097fa640 1 -- 172.21.15.16:0/2476142218 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fa9fc0d05e0 con 0x7faa0c101af0 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa0b7fe640 1 -- 172.21.15.16:0/1340941617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 msgr2=0x56217eed70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa0b7fe640 1 --2- 172.21.15.16:0/1340941617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eed70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa0b7fe640 1 -- 172.21.15.16:0/1340941617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 msgr2=0x56217eecbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa0b7fe640 1 --2- 172.21.15.16:0/1340941617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eecbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa0b7fe640 1 -- 172.21.15.16:0/1340941617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56217eb433f0 con 0x56217eecc0d0 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa0bfff640 1 --2- 172.21.15.16:0/1340941617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eecbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa10813640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9f80a3780 0x7fa9f80a5c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faa04005fd0 tx=0x7faa04009f00 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa0b7fe640 1 --2- 172.21.15.16:0/1340941617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed4770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa9f402c0b0 tx=0x7fa9f4005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672058a1dcb5f535 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.032+0000 7faa027fc640 1 -- 172.21.15.16:0/1340941617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4043020 con 0x56217eecc0d0 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.033+0000 7faa027fc640 1 -- 172.21.15.16:0/1340941617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa9f4034d50 con 0x56217eecc0d0 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.033+0000 7faa027fc640 1 -- 172.21.15.16:0/1340941617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4039040 con 0x56217eecc0d0 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.033+0000 7faa14e63740 1 -- 172.21.15.16:0/1340941617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 msgr2=0x56217eed4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.033+0000 7faa14e63740 1 --2- 172.21.15.16:0/1340941617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed4770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa9f402c0b0 tx=0x7fa9f4005b20 comp rx=0 tx=0).stop 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.033+0000 7faa14e63740 1 -- 172.21.15.16:0/1340941617 shutdown_connections 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.033+0000 7faa14e63740 1 --2- 172.21.15.16:0/1340941617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eed70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.033+0000 7faa14e63740 1 --2- 172.21.15.16:0/1340941617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed4770 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.033+0000 7faa14e63740 1 --2- 172.21.15.16:0/1340941617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eecbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.033+0000 7faa14e63740 1 -- 172.21.15.16:0/1340941617 >> 172.21.15.16:0/1340941617 conn(0x56217eec5ee0 msgr2=0x56217eec62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.033+0000 7faa14e63740 1 -- 172.21.15.16:0/1340941617 shutdown_connections 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.033+0000 7faa14e63740 1 -- 172.21.15.16:0/1340941617 wait complete. 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa14e63740 1 -- start start 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa14e63740 1 Processor -- start 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa14e63740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eed1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa14e63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa14e63740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eed2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa14e63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56217eed7a40 con 0x56217eecc0d0 2024-03-27T06:45:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa14e63740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x56217edeaeb0 con 0x56217eecb7b0 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa14e63740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56217eed7bc0 con 0x56217eed4cb0 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa14e63740 10 client.0 _ll_register_callbacks cb 0x56217ee276a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa0b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa0bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eed1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa10813640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eed2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa0b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60926/0 (socket says 172.21.15.16:60926) 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa0bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eed1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:45230/0 (socket says 172.21.15.16:45230) 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.034+0000 7faa0b7fe640 1 -- 172.21.15.16:0/52359759 learned_addr learned my addr 172.21.15.16:0/52359759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.035+0000 7faa10813640 1 -- 172.21.15.16:0/52359759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 msgr2=0x56217eed1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.035+0000 7faa10813640 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eed1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.035+0000 7faa10813640 1 -- 172.21.15.16:0/52359759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 msgr2=0x56217eed21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.035+0000 7faa10813640 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.035+0000 7faa10813640 1 -- 172.21.15.16:0/52359759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56217ec0d7d0 con 0x56217eed4cb0 2024-03-27T06:45:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.035+0000 7faa0b7fe640 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.035+0000 7faa10813640 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eed2710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa0400e620 tx=0x7faa04004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b386734d31e08098 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.035+0000 7fa9eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:45:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.036+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/52359759 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0402d060 con 0x56217eed4cb0 2024-03-27T06:45:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.036+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/52359759 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7faa04028620 con 0x56217eed4cb0 2024-03-27T06:45:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.036+0000 7faa14e63740 1 -- 172.21.15.16:0/52359759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f4034d50 con 0x56217eed4cb0 2024-03-27T06:45:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.040+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/52359759 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa04017040 con 0x56217eed4cb0 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.040+0000 7faa14e63740 10 client.26086 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.040+0000 7faa14e63740 1 -- 172.21.15.16:0/52359759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56217eee3cf0 con 0x56217eed4cb0 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.040+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/52359759 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa9f4034d50 con 0x56217eed4cb0 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.040+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/52359759 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7faa0400b040 con 0x56217eed4cb0 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.041+0000 7faa14e63740 10 client.26086 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.041+0000 7faa14e63740 10 client.26086 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.041+0000 7faa14e63740 20 client.26086 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.041+0000 7faa14e63740 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x56217eee5200 0x56217eee7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.041+0000 7faa14e63740 4 client.26086 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.041+0000 7faa14e63740 1 -- 172.21.15.16:0/52359759 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56217ee3cc50 con 0x56217eee5200 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.041+0000 7faa0bfff640 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x56217eee5200 0x56217eee7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.041+0000 7fa9eb7fe640 10 client.26086 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:45:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.041+0000 7faa0bfff640 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x56217eee5200 0x56217eee7640 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d8edff4bfe731ef9 server_cookie=8126741b8289f278 in_seq=0 out_seq=0 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/52359759 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56217ee3cc50 con 0x56217eee5200 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7fa9eb7fe640 10 client.26086 handle_command_reply: tid=0 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 1 client.26086 shutdown 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 1 -- 172.21.15.16:0/52359759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 msgr2=0x56217eed2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eed2710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa0400e620 tx=0x7faa04004ab0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 1 -- 172.21.15.16:0/52359759 shutdown_connections 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x56217eee5200 0x56217eee7640 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eed2710 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecc0d0 0x56217eed21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 1 --2- 172.21.15.16:0/52359759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eed1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7fa9eb7fe640 0 client.26086 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 1 -- 172.21.15.16:0/52359759 >> 172.21.15.16:0/52359759 conn(0x56217eec5ee0 msgr2=0x56217eec7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 1 -- 172.21.15.16:0/52359759 shutdown_connections 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 1 -- 172.21.15.16:0/52359759 wait complete. 2024-03-27T06:45:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.042+0000 7faa14e63740 20 client.26086 trim_cache size 0 max 16384 2024-03-27T06:45:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa14e63740 1 Processor -- start 2024-03-27T06:45:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa14e63740 1 -- start start 2024-03-27T06:45:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa14e63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecb7b0 0x56217eed43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa14e63740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecc0d0 0x56217eee0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa14e63740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eeecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa14e63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1ad890 con 0x56217eecb7b0 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa14e63740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1af490 con 0x56217eecc0d0 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa14e63740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1adf60 con 0x56217eed4cb0 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecb7b0 0x56217eed43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecb7b0 0x56217eed43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60934/0 (socket says 172.21.15.16:60934) 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa0bfff640 1 -- 172.21.15.16:0/124637054 learned_addr learned my addr 172.21.15.16:0/124637054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa0b7fe640 1 --2- 172.21.15.16:0/124637054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecc0d0 0x56217eee0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa10813640 1 --2- 172.21.15.16:0/124637054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eeecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa0bfff640 1 -- 172.21.15.16:0/124637054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 msgr2=0x56217eeecf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa0bfff640 1 --2- 172.21.15.16:0/124637054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eeecf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa0bfff640 1 -- 172.21.15.16:0/124637054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecc0d0 msgr2=0x56217eee0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa0bfff640 1 --2- 172.21.15.16:0/124637054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecc0d0 0x56217eee0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa0bfff640 1 -- 172.21.15.16:0/124637054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56217eec5d10 con 0x56217eecb7b0 2024-03-27T06:45:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa0bfff640 1 --2- 172.21.15.16:0/124637054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecb7b0 0x56217eed43c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc09c4a0 tx=0x7fa9fc004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93a4aef5f3502eb server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.056+0000 7faa01ffb640 1 -- 172.21.15.16:0/124637054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc09cea0 con 0x56217eecb7b0 2024-03-27T06:45:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.057+0000 7faa01ffb640 1 -- 172.21.15.16:0/124637054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa9fc09c7a0 con 0x56217eecb7b0 2024-03-27T06:45:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.057+0000 7faa01ffb640 1 -- 172.21.15.16:0/124637054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0cdfe0 con 0x56217eecb7b0 2024-03-27T06:45:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.057+0000 7faa14e63740 1 -- 172.21.15.16:0/124637054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecb7b0 msgr2=0x56217eed43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.057+0000 7faa14e63740 1 --2- 172.21.15.16:0/124637054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecb7b0 0x56217eed43c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc09c4a0 tx=0x7fa9fc004c10 comp rx=0 tx=0).stop 2024-03-27T06:45:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.057+0000 7faa14e63740 1 -- 172.21.15.16:0/124637054 shutdown_connections 2024-03-27T06:45:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.057+0000 7faa14e63740 1 --2- 172.21.15.16:0/124637054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eed4cb0 0x56217eeecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.057+0000 7faa14e63740 1 --2- 172.21.15.16:0/124637054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecc0d0 0x56217eee0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.057+0000 7faa14e63740 1 --2- 172.21.15.16:0/124637054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eecb7b0 0x56217eed43c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.057+0000 7faa14e63740 1 -- 172.21.15.16:0/124637054 >> 172.21.15.16:0/124637054 conn(0x56217eec5ee0 msgr2=0x56217eec4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.057+0000 7faa14e63740 1 -- 172.21.15.16:0/124637054 shutdown_connections 2024-03-27T06:45:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.057+0000 7faa14e63740 1 -- 172.21.15.16:0/124637054 wait complete. 2024-03-27T06:45:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa14e63740 1 -- start start 2024-03-27T06:45:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa14e63740 1 Processor -- start 2024-03-27T06:45:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa14e63740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eece950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa14e63740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eecc0d0 0x56217eecee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa14e63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eed4cb0 0x56217eecf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa14e63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1aedc0 con 0x56217eed4cb0 2024-03-27T06:45:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa14e63740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1ae630 con 0x56217eecb7b0 2024-03-27T06:45:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa14e63740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1ab870 con 0x56217eecc0d0 2024-03-27T06:45:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa14e63740 10 client.0 _ll_register_callbacks cb 0x7faa0400eea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa10813640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eed4cb0 0x56217eecf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa10813640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eed4cb0 0x56217eecf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60936/0 (socket says 172.21.15.16:60936) 2024-03-27T06:45:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa10813640 1 -- 172.21.15.16:0/4082555448 learned_addr learned my addr 172.21.15.16:0/4082555448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa0bfff640 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eece950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa0b7fe640 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eecc0d0 0x56217eecee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa10813640 1 -- 172.21.15.16:0/4082555448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eecc0d0 msgr2=0x56217eecee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa10813640 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eecc0d0 0x56217eecee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa10813640 1 -- 172.21.15.16:0/4082555448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 msgr2=0x56217eece950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa10813640 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eece950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.058+0000 7faa10813640 1 -- 172.21.15.16:0/4082555448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56217ef4a640 con 0x56217eed4cb0 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7faa10813640 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eed4cb0 0x56217eecf3d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faa04031b40 tx=0x7faa040092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e721b92b118446a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7fa9eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/4082555448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0402d040 con 0x56217eed4cb0 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/4082555448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7faa04032890 con 0x56217eed4cb0 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7faa14e63740 1 -- 172.21.15.16:0/4082555448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9fc09c7a0 con 0x56217eed4cb0 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/4082555448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa0400e770 con 0x56217eed4cb0 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7faa14e63740 10 client.16266 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7faa14e63740 1 -- 172.21.15.16:0/4082555448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56217ef4b940 con 0x56217eed4cb0 2024-03-27T06:45:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/4082555448 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa9fc09c7a0 con 0x56217eed4cb0 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/4082555448 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7faa0400b040 con 0x56217eed4cb0 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7faa14e63740 10 client.16266 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7faa14e63740 10 client.16266 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7faa14e63740 20 client.16266 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7faa14e63740 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x56217ef4c000 0x56217ef4e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7faa14e63740 4 client.16266 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.059+0000 7faa14e63740 1 -- 172.21.15.16:0/4082555448 --> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56217ee36fa0 con 0x56217ef4c000 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa0bfff640 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x56217ef4c000 0x56217ef4e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa0bfff640 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x56217ef4c000 0x56217ef4e420 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5a194f08feed61b server_cookie=935c6cbae45d81c1 in_seq=0 out_seq=0 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7fa9eb7fe640 10 client.16266 ms_handle_connect on v2:172.21.15.156:6834/4256913032 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/4082555448 <== mds.4 v2:172.21.15.156:6834/4256913032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+729 (crc 0 0 0) 0x56217ee36fa0 con 0x56217ef4c000 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7fa9eb7fe640 10 client.16266 handle_command_reply: tid=0 2024-03-27T06:45:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 1 client.16266 shutdown 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 1 -- 172.21.15.16:0/4082555448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eed4cb0 msgr2=0x56217eecf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eed4cb0 0x56217eecf3d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faa04031b40 tx=0x7faa040092f0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 1 -- 172.21.15.16:0/4082555448 shutdown_connections 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.156:6834/4256913032,v1:172.21.15.156:6835/4256913032] conn(0x56217ef4c000 0x56217ef4e420 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56217eed4cb0 0x56217eecf3d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56217eecc0d0 0x56217eecee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7fa9eb7fe640 0 client.16266 ms_handle_reset on v2:172.21.15.156:6834/4256913032 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 1 --2- 172.21.15.16:0/4082555448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217eecb7b0 0x56217eece950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 1 -- 172.21.15.16:0/4082555448 >> 172.21.15.16:0/4082555448 conn(0x56217eec5ee0 msgr2=0x56217eec5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 1 -- 172.21.15.16:0/4082555448 shutdown_connections 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 1 -- 172.21.15.16:0/4082555448 wait complete. 2024-03-27T06:45:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.060+0000 7faa14e63740 20 client.16266 trim_cache size 0 max 16384 2024-03-27T06:45:52.063 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-03-27T06:45:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 224002763, 2024-03-27T06:45:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511632166, 2024-03-27T06:45:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/00-INDEX" 2024-03-27T06:45:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.063 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 493159939, 2024-03-27T06:45:52.064 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511632167, 2024-03-27T06:45:52.064 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm" 2024-03-27T06:45:52.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.064 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.064 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.064 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2456918354, 2024-03-27T06:45:52.064 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631698, 2024-03-27T06:45:52.064 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/phy/samsung-usb2.txt" 2024-03-27T06:45:52.064 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:45:52.064 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-03-27T06:45:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 -- 172.21.15.16:0/2476142218 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9f80a3780 msgr2=0x7fa9f80a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9f80a3780 0x7fa9f80a5c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faa04005fd0 tx=0x7faa04009f00 comp rx=0 tx=0).stop 2024-03-27T06:45:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 -- 172.21.15.16:0/2476142218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c101af0 msgr2=0x7faa0c198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c101af0 0x7faa0c198550 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00e9f0 tx=0x7fa9fc00edf0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 -- 172.21.15.16:0/2476142218 shutdown_connections 2024-03-27T06:45:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa0c107b90 0x7faa0c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9f80a3780 0x7fa9f80a5c40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c1024b0 0x7faa0c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 --2- 172.21.15.16:0/2476142218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa0c101af0 0x7faa0c198550 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 -- 172.21.15.16:0/2476142218 >> 172.21.15.16:0/2476142218 conn(0x7faa0c0fd580 msgr2=0x7faa0c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 -- 172.21.15.16:0/2476142218 shutdown_connections 2024-03-27T06:45:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.061+0000 7faa12276640 1 -- 172.21.15.16:0/2476142218 wait complete. 2024-03-27T06:45:52.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-27T06:45:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.177+0000 7f412c302640 1 -- 172.21.15.16:0/3539813677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41241024a0 msgr2=0x7f412410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.177+0000 7f412c302640 1 --2- 172.21.15.16:0/3539813677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41241024a0 0x7f412410e9b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4114009c10 tx=0x7f411402bef0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.177+0000 7f412c302640 1 -- 172.21.15.16:0/3539813677 shutdown_connections 2024-03-27T06:45:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.177+0000 7f412c302640 1 --2- 172.21.15.16:0/3539813677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41241024a0 0x7f412410e9b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.177+0000 7f412c302640 1 --2- 172.21.15.16:0/3539813677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4124101ae0 0x7f4124101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.177+0000 7f412c302640 1 --2- 172.21.15.16:0/3539813677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4124107b80 0x7f4124107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.177+0000 7f412c302640 1 -- 172.21.15.16:0/3539813677 >> 172.21.15.16:0/3539813677 conn(0x7f4124078f50 msgr2=0x7f4124079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.177+0000 7f412c302640 1 -- 172.21.15.16:0/3539813677 shutdown_connections 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.177+0000 7f412c302640 1 -- 172.21.15.16:0/3539813677 wait complete. 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412c302640 1 Processor -- start 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412c302640 1 -- start start 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412c302640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4124101ae0 0x7f4124198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412c302640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41241024a0 0x7f41241989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412c302640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4124107b80 0x7f412419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412c302640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4124112020 con 0x7f4124107b80 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412c302640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f4124111ea0 con 0x7f41241024a0 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412c302640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f41241121a0 con 0x7f4124101ae0 2024-03-27T06:45:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412a89f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4124107b80 0x7f412419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412a89f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4124107b80 0x7f412419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60948/0 (socket says 172.21.15.16:60948) 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412a89f640 1 -- 172.21.15.16:0/1879322438 learned_addr learned my addr 172.21.15.16:0/1879322438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412989d640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41241024a0 0x7f41241989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.178+0000 7f412a09e640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4124101ae0 0x7f4124198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.179+0000 7f412a89f640 1 -- 172.21.15.16:0/1879322438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4124101ae0 msgr2=0x7f4124198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.179+0000 7f412a89f640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4124101ae0 0x7f4124198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.179+0000 7f412a89f640 1 -- 172.21.15.16:0/1879322438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41241024a0 msgr2=0x7f41241989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.179+0000 7f412a89f640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41241024a0 0x7f41241989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.179+0000 7f412a89f640 1 -- 172.21.15.16:0/1879322438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412419d450 con 0x7f4124107b80 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.179+0000 7f412a89f640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4124107b80 0x7f412419cd20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4114033490 tx=0x7f4114033720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.179+0000 7f41237fe640 1 -- 172.21.15.16:0/1879322438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114043020 con 0x7f4124107b80 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.179+0000 7f41237fe640 1 -- 172.21.15.16:0/1879322438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f411402ca60 con 0x7f4124107b80 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.179+0000 7f412c302640 1 -- 172.21.15.16:0/1879322438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412419d6e0 con 0x7f4124107b80 2024-03-27T06:45:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.179+0000 7f41237fe640 1 -- 172.21.15.16:0/1879322438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411403e6b0 con 0x7f4124107b80 2024-03-27T06:45:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.179+0000 7f412c302640 1 -- 172.21.15.16:0/1879322438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41241a4fd0 con 0x7f4124107b80 2024-03-27T06:45:52.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.181+0000 7f41237fe640 1 -- 172.21.15.16:0/1879322438 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f4114046020 con 0x7f4124107b80 2024-03-27T06:45:52.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.181+0000 7f412efc9740 1 Processor -- start 2024-03-27T06:45:52.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412efc9740 1 -- start start 2024-03-27T06:45:52.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f41237fe640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f410c0a3780 0x7f410c0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412efc9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742d7b0 0x55e30742db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412efc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 0x55e307436770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412efc9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 0x55e3074390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412efc9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e307391710 con 0x55e30742e0d0 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412efc9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e307339af0 con 0x55e30742d7b0 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412efc9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e30738d6c0 con 0x55e307436cb0 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412989d640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f410c0a3780 0x7f410c0a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f41237fe640 1 -- 172.21.15.16:0/1879322438 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f41140f2550 con 0x7f4124107b80 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412a89f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 0x55e3074390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412989d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 0x55e307436770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412989d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 0x55e307436770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60952/0 (socket says 172.21.15.16:60952) 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412989d640 1 -- 172.21.15.16:0/505831737 learned_addr learned my addr 172.21.15.16:0/505831737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412a09e640 1 --2- 172.21.15.16:0/505831737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742d7b0 0x55e30742db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412a89f640 1 -- 172.21.15.16:0/505831737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742d7b0 msgr2=0x55e30742db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412a89f640 1 --2- 172.21.15.16:0/505831737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742d7b0 0x55e30742db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412a89f640 1 -- 172.21.15.16:0/505831737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 msgr2=0x55e307436770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412a89f640 1 --2- 172.21.15.16:0/505831737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 0x55e307436770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.182+0000 7f412a89f640 1 -- 172.21.15.16:0/505831737 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3070a53f0 con 0x55e307436cb0 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.183+0000 7f412989d640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f410c0a3780 0x7f410c0a5c40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41180119d0 tx=0x7f4118009c00 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.183+0000 7f412a89f640 1 --2- 172.21.15.16:0/505831737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 0x55e3074390a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f41140efd80 tx=0x7f41140be300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f5bac4c39c8b23 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.183+0000 7f4120bfe640 1 -- 172.21.15.16:0/505831737 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41140475a0 con 0x55e307436cb0 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.183+0000 7f4120bfe640 1 -- 172.21.15.16:0/505831737 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4114033910 con 0x55e307436cb0 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.183+0000 7f4120bfe640 1 -- 172.21.15.16:0/505831737 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114048400 con 0x55e307436cb0 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 -- 172.21.15.16:0/505831737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 msgr2=0x55e3074390a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 --2- 172.21.15.16:0/505831737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 0x55e3074390a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f41140efd80 tx=0x7f41140be300 comp rx=0 tx=0).stop 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 -- 172.21.15.16:0/505831737 shutdown_connections 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 --2- 172.21.15.16:0/505831737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 0x55e3074390a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 --2- 172.21.15.16:0/505831737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 0x55e307436770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 --2- 172.21.15.16:0/505831737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742d7b0 0x55e30742db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 -- 172.21.15.16:0/505831737 >> 172.21.15.16:0/505831737 conn(0x55e307427ee0 msgr2=0x55e3074282c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 -- 172.21.15.16:0/505831737 shutdown_connections 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 -- 172.21.15.16:0/505831737 wait complete. 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 -- start start 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 Processor -- start 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742d7b0 0x55e307433cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 0x55e307434210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 0x55e307434750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e307439860 con 0x55e30742e0d0 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e30734ceb0 con 0x55e30742d7b0 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e3074399e0 con 0x55e307436cb0 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412efc9740 10 client.0 _ll_register_callbacks cb 0x55e3073896a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.184+0000 7f412989d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 0x55e307434210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412a09e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742d7b0 0x55e307433cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412989d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 0x55e307434210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60960/0 (socket says 172.21.15.16:60960) 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412a89f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 0x55e307434750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412989d640 1 -- 172.21.15.16:0/2364696616 learned_addr learned my addr 172.21.15.16:0/2364696616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412989d640 1 -- 172.21.15.16:0/2364696616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 msgr2=0x55e307434750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412989d640 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 0x55e307434750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412989d640 1 -- 172.21.15.16:0/2364696616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742d7b0 msgr2=0x55e307433cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412989d640 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742d7b0 0x55e307433cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412989d640 1 -- 172.21.15.16:0/2364696616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e30716f7d0 con 0x55e30742e0d0 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412a09e640 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742d7b0 0x55e307433cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412a89f640 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 0x55e307434750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412989d640 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 0x55e307434210 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4118009780 tx=0x7f411801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f64cf53a93c4c23 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f4109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f4109ffb640 1 -- 172.21.15.16:0/2364696616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411800fe50 con 0x55e30742e0d0 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f4109ffb640 1 -- 172.21.15.16:0/2364696616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f4118003800 con 0x55e30742e0d0 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412efc9740 1 -- 172.21.15.16:0/2364696616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e30739ec50 con 0x55e30742e0d0 2024-03-27T06:45:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f4109ffb640 1 -- 172.21.15.16:0/2364696616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4118003a40 con 0x55e30742e0d0 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412efc9740 10 client.16278 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.185+0000 7f412efc9740 1 -- 172.21.15.16:0/2364696616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e307445a10 con 0x55e30742e0d0 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.186+0000 7f4109ffb640 1 -- 172.21.15.16:0/2364696616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55e30739ec50 con 0x55e30742e0d0 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.186+0000 7f4109ffb640 1 -- 172.21.15.16:0/2364696616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f4118010430 con 0x55e30742e0d0 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.186+0000 7f412efc9740 10 client.16278 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.186+0000 7f412efc9740 10 client.16278 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.186+0000 7f412efc9740 20 client.16278 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.186+0000 7f412efc9740 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e307446f20 0x55e307449360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.186+0000 7f412efc9740 4 client.16278 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.186+0000 7f412efc9740 1 -- 172.21.15.16:0/2364696616 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e307398fa0 con 0x55e307446f20 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.186+0000 7f412a09e640 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e307446f20 0x55e307449360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.186+0000 7f4109ffb640 10 client.16278 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.186+0000 7f412a09e640 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e307446f20 0x55e307449360 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc9b54d3c152740f server_cookie=189acbead3188dd2 in_seq=0 out_seq=0 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f4109ffb640 1 -- 172.21.15.16:0/2364696616 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e307398fa0 con 0x55e307446f20 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f4109ffb640 10 client.16278 handle_command_reply: tid=0 2024-03-27T06:45:52.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 1 client.16278 shutdown 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 1 -- 172.21.15.16:0/2364696616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 msgr2=0x55e307434210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 0x55e307434210 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4118009780 tx=0x7f411801d000 comp rx=0 tx=0).stop 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 1 -- 172.21.15.16:0/2364696616 shutdown_connections 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e307446f20 0x55e307449360 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e307436cb0 0x55e307434750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742e0d0 0x55e307434210 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 1 --2- 172.21.15.16:0/2364696616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742d7b0 0x55e307433cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f4109ffb640 0 client.16278 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 1 -- 172.21.15.16:0/2364696616 >> 172.21.15.16:0/2364696616 conn(0x55e307427ee0 msgr2=0x55e307429c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 1 -- 172.21.15.16:0/2364696616 shutdown_connections 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 1 -- 172.21.15.16:0/2364696616 wait complete. 2024-03-27T06:45:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.187+0000 7f412efc9740 20 client.16278 trim_cache size 0 max 16384 2024-03-27T06:45:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412efc9740 1 Processor -- start 2024-03-27T06:45:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412efc9740 1 -- start start 2024-03-27T06:45:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412efc9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e30742d7b0 0x55e307436400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412efc9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742e0d0 0x55e307432e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412efc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e307436cb0 0x55e3074424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412efc9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41241ad7d0 con 0x55e307436cb0 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412efc9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f41241af3d0 con 0x55e30742e0d0 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412efc9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f41241adea0 con 0x55e30742d7b0 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412a89f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e307436cb0 0x55e3074424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412989d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742e0d0 0x55e307432e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412a89f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e307436cb0 0x55e3074424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60972/0 (socket says 172.21.15.16:60972) 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412a09e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e30742d7b0 0x55e307436400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412a89f640 1 -- 172.21.15.16:0/245943191 learned_addr learned my addr 172.21.15.16:0/245943191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412a89f640 1 -- 172.21.15.16:0/245943191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e30742d7b0 msgr2=0x55e307436400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412a89f640 1 --2- 172.21.15.16:0/245943191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e30742d7b0 0x55e307436400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412a89f640 1 -- 172.21.15.16:0/245943191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742e0d0 msgr2=0x55e307432e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412a89f640 1 --2- 172.21.15.16:0/245943191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742e0d0 0x55e307432e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.201+0000 7f412a89f640 1 -- 172.21.15.16:0/245943191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3074390a0 con 0x55e307436cb0 2024-03-27T06:45:52.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f412a89f640 1 --2- 172.21.15.16:0/245943191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e307436cb0 0x55e3074424b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f41140bb140 tx=0x7f411402bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55be6b83ba10feb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f410bfff640 1 -- 172.21.15.16:0/245943191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41140473b0 con 0x55e307436cb0 2024-03-27T06:45:52.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f410bfff640 1 -- 172.21.15.16:0/245943191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f41140beec0 con 0x55e307436cb0 2024-03-27T06:45:52.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f410bfff640 1 -- 172.21.15.16:0/245943191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114003c10 con 0x55e307436cb0 2024-03-27T06:45:52.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f412efc9740 1 -- 172.21.15.16:0/245943191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e307436cb0 msgr2=0x55e3074424b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f412efc9740 1 --2- 172.21.15.16:0/245943191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e307436cb0 0x55e3074424b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f41140bb140 tx=0x7f411402bef0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f412efc9740 1 -- 172.21.15.16:0/245943191 shutdown_connections 2024-03-27T06:45:52.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f412efc9740 1 --2- 172.21.15.16:0/245943191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e307436cb0 0x55e3074424b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f412efc9740 1 --2- 172.21.15.16:0/245943191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e30742e0d0 0x55e307432e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f412efc9740 1 --2- 172.21.15.16:0/245943191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e30742d7b0 0x55e307436400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f412efc9740 1 -- 172.21.15.16:0/245943191 >> 172.21.15.16:0/245943191 conn(0x55e307427ee0 msgr2=0x55e307433c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f412efc9740 1 -- 172.21.15.16:0/245943191 shutdown_connections 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.202+0000 7f412efc9740 1 -- 172.21.15.16:0/245943191 wait complete. 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412efc9740 1 -- start start 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412efc9740 1 Processor -- start 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412efc9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742d7b0 0x55e30743ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412efc9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e30742e0d0 0x55e307430350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412efc9740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e307436cb0 0x55e307430890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412efc9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41241aed00 con 0x55e30742d7b0 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412efc9740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f41241ae570 con 0x55e307436cb0 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412efc9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f41241ab7b0 con 0x55e30742e0d0 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412efc9740 10 client.0 _ll_register_callbacks cb 0x7f4118009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:52.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412a09e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742d7b0 0x55e30743ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412a09e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742d7b0 0x55e30743ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60980/0 (socket says 172.21.15.16:60980) 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412a09e640 1 -- 172.21.15.16:0/3314850745 learned_addr learned my addr 172.21.15.16:0/3314850745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412989d640 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e30742e0d0 0x55e307430350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412a89f640 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e307436cb0 0x55e307430890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412a09e640 1 -- 172.21.15.16:0/3314850745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e30742e0d0 msgr2=0x55e307430350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412a09e640 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e30742e0d0 0x55e307430350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412a09e640 1 -- 172.21.15.16:0/3314850745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e307436cb0 msgr2=0x55e307430890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412a09e640 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e307436cb0 0x55e307430890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412a09e640 1 -- 172.21.15.16:0/3314850745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e307430e20 con 0x55e30742d7b0 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f412a09e640 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742d7b0 0x55e30743ecd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f411c000c00 tx=0x7f411c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edba101cee1f87a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.203+0000 7f4109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f4109ffb640 1 -- 172.21.15.16:0/3314850745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c024070 con 0x55e30742d7b0 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f4109ffb640 1 -- 172.21.15.16:0/3314850745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f411c005220 con 0x55e30742d7b0 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f412efc9740 1 -- 172.21.15.16:0/3314850745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41140beec0 con 0x55e30742d7b0 2024-03-27T06:45:52.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f4109ffb640 1 -- 172.21.15.16:0/3314850745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f411c02f050 con 0x55e30742d7b0 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f412efc9740 10 client.16284 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f412efc9740 1 -- 172.21.15.16:0/3314850745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e3074ad080 con 0x55e30742d7b0 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f4109ffb640 1 -- 172.21.15.16:0/3314850745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f41140beec0 con 0x55e30742d7b0 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f4109ffb640 1 -- 172.21.15.16:0/3314850745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f411c020070 con 0x55e30742d7b0 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f412efc9740 10 client.16284 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f412efc9740 10 client.16284 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f412efc9740 20 client.16284 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f412efc9740 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e3074ad780 0x55e3074afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f412efc9740 4 client.16284 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f412efc9740 1 -- 172.21.15.16:0/3314850745 --> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3073166a0 con 0x55e3074ad780 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.204+0000 7f412989d640 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e3074ad780 0x55e3074afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f4109ffb640 10 client.16284 ms_handle_connect on v2:172.21.15.170:6836/2408489138 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412989d640 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e3074ad780 0x55e3074afba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6a7e22b05437cc5a server_cookie=c428fc132cd9f031 in_seq=0 out_seq=0 2024-03-27T06:45:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f4109ffb640 1 -- 172.21.15.16:0/3314850745 <== mds.3 v2:172.21.15.170:6836/2408489138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+491 (crc 0 0 0) 0x55e3073166a0 con 0x55e3074ad780 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f4109ffb640 10 client.16284 handle_command_reply: tid=0 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 1 client.16284 shutdown 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 1 -- 172.21.15.16:0/3314850745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742d7b0 msgr2=0x55e30743ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742d7b0 0x55e30743ecd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f411c000c00 tx=0x7f411c004410 comp rx=0 tx=0).stop 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 1 -- 172.21.15.16:0/3314850745 shutdown_connections 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.170:6836/2408489138,v1:172.21.15.170:6837/2408489138] conn(0x55e3074ad780 0x55e3074afba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e307436cb0 0x55e307430890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e30742e0d0 0x55e307430350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f4109ffb640 0 client.16284 ms_handle_reset on v2:172.21.15.170:6836/2408489138 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 1 --2- 172.21.15.16:0/3314850745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e30742d7b0 0x55e30743ecd0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 1 -- 172.21.15.16:0/3314850745 >> 172.21.15.16:0/3314850745 conn(0x55e307427ee0 msgr2=0x55e307427000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 1 -- 172.21.15.16:0/3314850745 shutdown_connections 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 1 -- 172.21.15.16:0/3314850745 wait complete. 2024-03-27T06:45:52.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.205+0000 7f412efc9740 20 client.16284 trim_cache size 0 max 16384 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 518453461, 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631491, 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/00-INDEX" 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3536147696, 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631492, 2024-03-27T06:45:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/AU1xxx_IDE.README" 2024-03-27T06:45:52.208 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:45:52.208 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-03-27T06:45:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 -- 172.21.15.16:0/1879322438 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f410c0a3780 msgr2=0x7f410c0a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f410c0a3780 0x7f410c0a5c40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41180119d0 tx=0x7f4118009c00 comp rx=0 tx=0).stop 2024-03-27T06:45:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 -- 172.21.15.16:0/1879322438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4124107b80 msgr2=0x7f412419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4124107b80 0x7f412419cd20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4114033490 tx=0x7f4114033720 comp rx=0 tx=0).stop 2024-03-27T06:45:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 -- 172.21.15.16:0/1879322438 shutdown_connections 2024-03-27T06:45:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4124107b80 0x7f412419cd20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41241024a0 0x7f41241989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f410c0a3780 0x7f410c0a5c40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 --2- 172.21.15.16:0/1879322438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4124101ae0 0x7f4124198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 -- 172.21.15.16:0/1879322438 >> 172.21.15.16:0/1879322438 conn(0x7f4124078f50 msgr2=0x7f4124104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 -- 172.21.15.16:0/1879322438 shutdown_connections 2024-03-27T06:45:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.206+0000 7f412c302640 1 -- 172.21.15.16:0/1879322438 wait complete. 2024-03-27T06:45:52.210 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:52 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2681006120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:45:52.211 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-27T06:45:52.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.321+0000 7f10042a3640 1 -- 172.21.15.16:0/351735053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc106a10 msgr2=0x7f0ffc106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.321+0000 7f10042a3640 1 --2- 172.21.15.16:0/351735053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc106a10 0x7f0ffc106df0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009930 tx=0x7f0ff802bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.321+0000 7f10042a3640 1 -- 172.21.15.16:0/351735053 shutdown_connections 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.321+0000 7f10042a3640 1 --2- 172.21.15.16:0/351735053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ffc1013c0 0x7f0ffc10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.321+0000 7f10042a3640 1 --2- 172.21.15.16:0/351735053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc100a00 0x7f0ffc100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.321+0000 7f10042a3640 1 --2- 172.21.15.16:0/351735053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc106a10 0x7f0ffc106df0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.321+0000 7f10042a3640 1 -- 172.21.15.16:0/351735053 >> 172.21.15.16:0/351735053 conn(0x7f0ffc076dd0 msgr2=0x7f0ffc0771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.321+0000 7f10042a3640 1 -- 172.21.15.16:0/351735053 shutdown_connections 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.321+0000 7f10042a3640 1 -- 172.21.15.16:0/351735053 wait complete. 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f10042a3640 1 Processor -- start 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f10042a3640 1 -- start start 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f10042a3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ffc100a00 0x7f0ffc198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f10042a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc1013c0 0x7f0ffc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f10042a3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc106a10 0x7f0ffc19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f10042a3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc112000 con 0x7f0ffc1013c0 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f10042a3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc111e80 con 0x7f0ffc100a00 2024-03-27T06:45:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f10042a3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc112180 con 0x7f0ffc106a10 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f100183e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc1013c0 0x7f0ffc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f100183e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc1013c0 0x7f0ffc198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32768/0 (socket says 172.21.15.16:32768) 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f100183e640 1 -- 172.21.15.16:0/1155649954 learned_addr learned my addr 172.21.15.16:0/1155649954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f100203f640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ffc100a00 0x7f0ffc198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f1002840640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc106a10 0x7f0ffc19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f100183e640 1 -- 172.21.15.16:0/1155649954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc106a10 msgr2=0x7f0ffc19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f100183e640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc106a10 0x7f0ffc19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f100183e640 1 -- 172.21.15.16:0/1155649954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ffc100a00 msgr2=0x7f0ffc198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f100183e640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ffc100a00 0x7f0ffc198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.322+0000 7f100183e640 1 -- 172.21.15.16:0/1155649954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc19d520 con 0x7f0ffc1013c0 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.323+0000 7f100183e640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc1013c0 0x7f0ffc198ac0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00e9f0 tx=0x7f0fec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.323+0000 7f0ff37fe640 1 -- 172.21.15.16:0/1155649954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec00cd80 con 0x7f0ffc1013c0 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.323+0000 7f10042a3640 1 -- 172.21.15.16:0/1155649954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc19d810 con 0x7f0ffc1013c0 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.323+0000 7f0ff37fe640 1 -- 172.21.15.16:0/1155649954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0fec005520 con 0x7f0ffc1013c0 2024-03-27T06:45:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.323+0000 7f0ff37fe640 1 -- 172.21.15.16:0/1155649954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec010690 con 0x7f0ffc1013c0 2024-03-27T06:45:52.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.323+0000 7f10042a3640 1 -- 172.21.15.16:0/1155649954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1a50a0 con 0x7f0ffc1013c0 2024-03-27T06:45:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.325+0000 7f0ff37fe640 1 -- 172.21.15.16:0/1155649954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f0fec0046c0 con 0x7f0ffc1013c0 2024-03-27T06:45:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.325+0000 7f1006e85740 1 Processor -- start 2024-03-27T06:45:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f1006e85740 1 -- start start 2024-03-27T06:45:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f1006e85740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f1006e85740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f1006e85740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f1006e85740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558b6731f710 con 0x558b673c4cb0 2024-03-27T06:45:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f1006e85740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x558b672c7af0 con 0x558b673bc0d0 2024-03-27T06:45:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f0ff37fe640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0fe80a3780 0x7f0fe80a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f1006e85740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558b6731b6c0 con 0x558b673bb7b0 2024-03-27T06:45:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f1002840640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f100203f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f100203f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:45322/0 (socket says 172.21.15.16:45322) 2024-03-27T06:45:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f1002840640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673bbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:48406/0 (socket says 172.21.15.16:48406) 2024-03-27T06:45:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f100183e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f100203f640 1 -- 172.21.15.16:0/3863760808 learned_addr learned my addr 172.21.15.16:0/3863760808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f1002840640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0fe80a3780 0x7f0fe80a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f0ff37fe640 1 -- 172.21.15.16:0/1155649954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f0fec0d0570 con 0x7f0ffc1013c0 2024-03-27T06:45:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f100203f640 1 -- 172.21.15.16:0/3863760808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 msgr2=0x558b673bbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f100203f640 1 --2- 172.21.15.16:0/3863760808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673bbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f100203f640 1 -- 172.21.15.16:0/3863760808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 msgr2=0x558b673c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f100203f640 1 --2- 172.21.15.16:0/3863760808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.326+0000 7f100203f640 1 -- 172.21.15.16:0/3863760808 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b670333f0 con 0x558b673bc0d0 2024-03-27T06:45:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.327+0000 7f1002840640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0fe80a3780 0x7f0fe80a5c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005fd0 tx=0x7f0ff4009500 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.327+0000 7f100203f640 1 --2- 172.21.15.16:0/3863760808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c4770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8036040 tx=0x7f0ff8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa94acbd8362ba72 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.327+0000 7f0ff0bfe640 1 -- 172.21.15.16:0/3863760808 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff802c710 con 0x558b673bc0d0 2024-03-27T06:45:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.327+0000 7f0ff0bfe640 1 -- 172.21.15.16:0/3863760808 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0ff8004040 con 0x558b673bc0d0 2024-03-27T06:45:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.327+0000 7f0ff0bfe640 1 -- 172.21.15.16:0/3863760808 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80046c0 con 0x558b673bc0d0 2024-03-27T06:45:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 -- 172.21.15.16:0/3863760808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 msgr2=0x558b673c4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 --2- 172.21.15.16:0/3863760808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c4770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8036040 tx=0x7f0ff8009560 comp rx=0 tx=0).stop 2024-03-27T06:45:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 -- 172.21.15.16:0/3863760808 shutdown_connections 2024-03-27T06:45:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 --2- 172.21.15.16:0/3863760808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 --2- 172.21.15.16:0/3863760808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c4770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 --2- 172.21.15.16:0/3863760808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673bbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 -- 172.21.15.16:0/3863760808 >> 172.21.15.16:0/3863760808 conn(0x558b673b5ee0 msgr2=0x558b673b62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 -- 172.21.15.16:0/3863760808 shutdown_connections 2024-03-27T06:45:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 -- 172.21.15.16:0/3863760808 wait complete. 2024-03-27T06:45:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 -- start start 2024-03-27T06:45:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 Processor -- start 2024-03-27T06:45:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558b673c7a40 con 0x558b673c4cb0 2024-03-27T06:45:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x558b672daeb0 con 0x558b673bc0d0 2024-03-27T06:45:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558b673c7bc0 con 0x558b673bb7b0 2024-03-27T06:45:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1006e85740 10 client.0 _ll_register_callbacks cb 0x558b673176a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f100203f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f1002840640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f100203f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673c1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:48418/0 (socket says 172.21.15.16:48418) 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.328+0000 7f100203f640 1 -- 172.21.15.16:0/4132069199 learned_addr learned my addr 172.21.15.16:0/4132069199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f100183e640 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f100203f640 1 -- 172.21.15.16:0/4132069199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 msgr2=0x558b673c21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f100203f640 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f100203f640 1 -- 172.21.15.16:0/4132069199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 msgr2=0x558b673c2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f100203f640 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f100203f640 1 -- 172.21.15.16:0/4132069199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b670fd7d0 con 0x558b673bb7b0 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f1002840640 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f100203f640 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673c1c80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8005700 tx=0x7f0ff802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b4f7606ae9b297 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f0fe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/4132069199 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff804f030 con 0x558b673bb7b0 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/4132069199 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f0ff8004400 con 0x558b673bb7b0 2024-03-27T06:45:52.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f1006e85740 1 -- 172.21.15.16:0/4132069199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff8004040 con 0x558b673bb7b0 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/4132069199 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ff802c9d0 con 0x558b673bb7b0 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f1006e85740 10 client.26122 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.329+0000 7f1006e85740 1 -- 172.21.15.16:0/4132069199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558b673c7a40 con 0x558b673bb7b0 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.330+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/4132069199 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0ff8004040 con 0x558b673bb7b0 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.330+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/4132069199 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f0ff80046c0 con 0x558b673bb7b0 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.330+0000 7f1006e85740 10 client.26122 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.330+0000 7f1006e85740 10 client.26122 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.330+0000 7f1006e85740 20 client.26122 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.330+0000 7f1006e85740 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x558b673d4e00 0x558b673d71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.330+0000 7f1006e85740 4 client.26122 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.330+0000 7f1006e85740 1 -- 172.21.15.16:0/4132069199 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b6732cc50 con 0x558b673d4e00 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.330+0000 7f100183e640 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x558b673d4e00 0x558b673d71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.330+0000 7f0fe5ffb640 10 client.26122 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.330+0000 7f100183e640 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x558b673d4e00 0x558b673d71f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5322b5f64b2718a1 server_cookie=ccc946e4b178ce19 in_seq=0 out_seq=0 2024-03-27T06:45:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.331+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/4132069199 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558b6732cc50 con 0x558b673d4e00 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.331+0000 7f0fe5ffb640 10 client.26122 handle_command_reply: tid=0 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.331+0000 7f1006e85740 1 client.26122 shutdown 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f1006e85740 1 -- 172.21.15.16:0/4132069199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 msgr2=0x558b673c1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f1006e85740 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673c1c80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8005700 tx=0x7f0ff802bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f1006e85740 1 -- 172.21.15.16:0/4132069199 shutdown_connections 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f1006e85740 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x558b673d4e00 0x558b673d71f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f1006e85740 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f0fe5ffb640 0 client.26122 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f1006e85740 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f1006e85740 1 --2- 172.21.15.16:0/4132069199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673c1c80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f1006e85740 1 -- 172.21.15.16:0/4132069199 >> 172.21.15.16:0/4132069199 conn(0x558b673b5ee0 msgr2=0x558b673b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f1006e85740 1 -- 172.21.15.16:0/4132069199 shutdown_connections 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f1006e85740 1 -- 172.21.15.16:0/4132069199 wait complete. 2024-03-27T06:45:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.332+0000 7f1006e85740 20 client.26122 trim_cache size 0 max 16384 2024-03-27T06:45:52.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:52 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2681006120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f1006e85740 1 Processor -- start 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f1006e85740 1 -- start start 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f1006e85740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bb7b0 0x558b673bcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f1006e85740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673bc0d0 0x558b673d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f1006e85740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673c4cb0 0x558b673c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f1006e85740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1ad840 con 0x558b673bc0d0 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f1006e85740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1af440 con 0x558b673bb7b0 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f1006e85740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1adf10 con 0x558b673c4cb0 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f100183e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673bc0d0 0x558b673d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f100183e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673bc0d0 0x558b673d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32784/0 (socket says 172.21.15.16:32784) 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f100183e640 1 -- 172.21.15.16:0/2476758634 learned_addr learned my addr 172.21.15.16:0/2476758634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f100203f640 1 --2- 172.21.15.16:0/2476758634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bb7b0 0x558b673bcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f1002840640 1 --2- 172.21.15.16:0/2476758634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673c4cb0 0x558b673c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f100183e640 1 -- 172.21.15.16:0/2476758634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673c4cb0 msgr2=0x558b673c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f100183e640 1 --2- 172.21.15.16:0/2476758634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673c4cb0 0x558b673c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f100183e640 1 -- 172.21.15.16:0/2476758634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bb7b0 msgr2=0x558b673bcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.345+0000 7f100183e640 1 --2- 172.21.15.16:0/2476758634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bb7b0 0x558b673bcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f100183e640 1 -- 172.21.15.16:0/2476758634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b673d0d20 con 0x558b673bc0d0 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f100183e640 1 --2- 172.21.15.16:0/2476758634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673bc0d0 0x558b673d07e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c6a0 tx=0x7f0fec01bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ad2ab7d8eeffd server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f0fe7fff640 1 -- 172.21.15.16:0/2476758634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0870d0 con 0x558b673bc0d0 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f0fe7fff640 1 -- 172.21.15.16:0/2476758634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0fec09c6d0 con 0x558b673bc0d0 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f0fe7fff640 1 -- 172.21.15.16:0/2476758634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0bff80 con 0x558b673bc0d0 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f1006e85740 1 -- 172.21.15.16:0/2476758634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673bc0d0 msgr2=0x558b673d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f1006e85740 1 --2- 172.21.15.16:0/2476758634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673bc0d0 0x558b673d07e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c6a0 tx=0x7f0fec01bb30 comp rx=0 tx=0).stop 2024-03-27T06:45:52.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f1006e85740 1 -- 172.21.15.16:0/2476758634 shutdown_connections 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f1006e85740 1 --2- 172.21.15.16:0/2476758634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673c4cb0 0x558b673c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f1006e85740 1 --2- 172.21.15.16:0/2476758634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673bc0d0 0x558b673d07e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f1006e85740 1 --2- 172.21.15.16:0/2476758634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bb7b0 0x558b673bcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f1006e85740 1 -- 172.21.15.16:0/2476758634 >> 172.21.15.16:0/2476758634 conn(0x558b673b5ee0 msgr2=0x558b673b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f1006e85740 1 -- 172.21.15.16:0/2476758634 shutdown_connections 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.346+0000 7f1006e85740 1 -- 172.21.15.16:0/2476758634 wait complete. 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1006e85740 1 -- start start 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1006e85740 1 Processor -- start 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1006e85740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1006e85740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1006e85740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1006e85740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1aed70 con 0x558b673c4cb0 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1006e85740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1ae5e0 con 0x558b673bc0d0 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1006e85740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1ab820 con 0x558b673bb7b0 2024-03-27T06:45:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1006e85740 10 client.0 _ll_register_callbacks cb 0x7f0fec009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1002840640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1002840640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32794/0 (socket says 172.21.15.16:32794) 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1002840640 1 -- 172.21.15.16:0/282801557 learned_addr learned my addr 172.21.15.16:0/282801557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f100183e640 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f100203f640 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1002840640 1 -- 172.21.15.16:0/282801557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 msgr2=0x558b673cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1002840640 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1002840640 1 -- 172.21.15.16:0/282801557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 msgr2=0x558b673c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1002840640 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.347+0000 7f1002840640 1 -- 172.21.15.16:0/282801557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b67439f90 con 0x558b673c4cb0 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f1002840640 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c8d20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0ff401f960 tx=0x7f0ff4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706cfab843ce9e66 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f0fe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/282801557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff402f060 con 0x558b673c4cb0 2024-03-27T06:45:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/282801557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f0ff402a680 con 0x558b673c4cb0 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f1006e85740 1 -- 172.21.15.16:0/282801557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fec09c6d0 con 0x558b673c4cb0 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/282801557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ff402a880 con 0x558b673c4cb0 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f1006e85740 10 client.16302 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f1006e85740 1 -- 172.21.15.16:0/282801557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558b6743b1b0 con 0x558b673c4cb0 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/282801557 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0fec09c6d0 con 0x558b673c4cb0 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/282801557 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f0ff401d070 con 0x558b673c4cb0 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f1006e85740 10 client.16302 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f1006e85740 10 client.16302 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f1006e85740 20 client.16302 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f1006e85740 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x558b6743b930 0x558b6743dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f1006e85740 4 client.16302 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.348+0000 7f1006e85740 1 -- 172.21.15.16:0/282801557 --> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b67326fa0 con 0x558b6743b930 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f100203f640 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x558b6743b930 0x558b6743dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f100203f640 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x558b6743b930 0x558b6743dd50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c30c68c7bb5a575 server_cookie=ade94f65a914a36b in_seq=0 out_seq=0 2024-03-27T06:45:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f0fe5ffb640 10 client.16302 ms_handle_connect on v2:172.21.15.170:6838/246078270 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f0fe5ffb640 1 -- 172.21.15.16:0/282801557 <== mds.2 v2:172.21.15.170:6838/246078270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1732 (crc 0 0 0) 0x558b67326fa0 con 0x558b6743b930 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f0fe5ffb640 10 client.16302 handle_command_reply: tid=0 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 1 client.16302 shutdown 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 1 -- 172.21.15.16:0/282801557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 msgr2=0x558b673c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c8d20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0ff401f960 tx=0x7f0ff4004a40 comp rx=0 tx=0).stop 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 1 -- 172.21.15.16:0/282801557 shutdown_connections 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.170:6838/246078270,v1:172.21.15.170:6839/246078270] conn(0x558b6743b930 0x558b6743dd50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b673c4cb0 0x558b673c8d20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b673bc0d0 0x558b673c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f0fe5ffb640 0 client.16302 ms_handle_reset on v2:172.21.15.170:6838/246078270 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 1 --2- 172.21.15.16:0/282801557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b673bb7b0 0x558b673cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 1 -- 172.21.15.16:0/282801557 >> 172.21.15.16:0/282801557 conn(0x558b673b5ee0 msgr2=0x558b673cf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 1 -- 172.21.15.16:0/282801557 shutdown_connections 2024-03-27T06:45:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 1 -- 172.21.15.16:0/282801557 wait complete. 2024-03-27T06:45:52.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.349+0000 7f1006e85740 20 client.16302 trim_cache size 0 max 16384 2024-03-27T06:45:52.352 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-03-27T06:45:52.352 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.352 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.352 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 288873844, 2024-03-27T06:45:52.352 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631404, 2024-03-27T06:45:52.352 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/makefiles.txt" 2024-03-27T06:45:52.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.352 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.352 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.352 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1317136046, 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631403, 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig.txt" 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1541207138, 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631400, 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/headers_install.txt" 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1572893329, 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631402, 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig-language.txt" 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.353 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1813746460, 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631399, 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/00-INDEX" 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2804642534, 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631401, 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kbuild.txt" 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.355 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2953760589, 2024-03-27T06:45:52.355 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631405, 2024-03-27T06:45:52.355 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/modules.txt" 2024-03-27T06:45:52.355 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:45:52.355 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-03-27T06:45:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 -- 172.21.15.16:0/1155649954 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0fe80a3780 msgr2=0x7f0fe80a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0fe80a3780 0x7f0fe80a5c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005fd0 tx=0x7f0ff4009500 comp rx=0 tx=0).stop 2024-03-27T06:45:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 -- 172.21.15.16:0/1155649954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc1013c0 msgr2=0x7f0ffc198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc1013c0 0x7f0ffc198ac0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00e9f0 tx=0x7f0fec00edf0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 -- 172.21.15.16:0/1155649954 shutdown_connections 2024-03-27T06:45:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f0fe80a3780 0x7f0fe80a5c40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ffc106a10 0x7f0ffc19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ffc1013c0 0x7f0ffc198ac0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 --2- 172.21.15.16:0/1155649954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ffc100a00 0x7f0ffc198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 -- 172.21.15.16:0/1155649954 >> 172.21.15.16:0/1155649954 conn(0x7f0ffc076dd0 msgr2=0x7f0ffc0fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 -- 172.21.15.16:0/1155649954 shutdown_connections 2024-03-27T06:45:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.350+0000 7f10042a3640 1 -- 172.21.15.16:0/1155649954 wait complete. 2024-03-27T06:45:52.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-27T06:45:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.469+0000 7fc9e6655640 1 -- 172.21.15.16:0/866251758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01024a0 msgr2=0x7fc9e010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.469+0000 7fc9e6655640 1 --2- 172.21.15.16:0/866251758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01024a0 0x7fc9e010e9b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009cd0 tx=0x7fc9d802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.469+0000 7fc9e6655640 1 -- 172.21.15.16:0/866251758 shutdown_connections 2024-03-27T06:45:52.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.469+0000 7fc9e6655640 1 --2- 172.21.15.16:0/866251758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01024a0 0x7fc9e010e9b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.469+0000 7fc9e6655640 1 --2- 172.21.15.16:0/866251758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e0101ae0 0x7fc9e0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.469+0000 7fc9e6655640 1 --2- 172.21.15.16:0/866251758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e0107b80 0x7fc9e0107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.469+0000 7fc9e6655640 1 -- 172.21.15.16:0/866251758 >> 172.21.15.16:0/866251758 conn(0x7fc9e0078f50 msgr2=0x7fc9e0079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.469+0000 7fc9e6655640 1 -- 172.21.15.16:0/866251758 shutdown_connections 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.469+0000 7fc9e6655640 1 -- 172.21.15.16:0/866251758 wait complete. 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.469+0000 7fc9e6655640 1 Processor -- start 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9e6655640 1 -- start start 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9e6655640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e0101ae0 0x7fc9e0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9e6655640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01024a0 0x7fc9e01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9e6655640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e0107b80 0x7fc9e019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9e6655640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e0112020 con 0x7fc9e01024a0 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9e6655640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e0111ea0 con 0x7fc9e0101ae0 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9e6655640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01121a0 con 0x7fc9e0107b80 2024-03-27T06:45:52.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01024a0 0x7fc9e01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01024a0 0x7fc9e01989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32818/0 (socket says 172.21.15.16:32818) 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9df7fe640 1 -- 172.21.15.16:0/845991327 learned_addr learned my addr 172.21.15.16:0/845991327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9dffff640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e0101ae0 0x7fc9e0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e0107b80 0x7fc9e019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9df7fe640 1 -- 172.21.15.16:0/845991327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e0107b80 msgr2=0x7fc9e019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9df7fe640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e0107b80 0x7fc9e019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9df7fe640 1 -- 172.21.15.16:0/845991327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e0101ae0 msgr2=0x7fc9e0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9df7fe640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e0101ae0 0x7fc9e0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9df7fe640 1 -- 172.21.15.16:0/845991327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e019d470 con 0x7fc9e01024a0 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.470+0000 7fc9df7fe640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01024a0 0x7fc9e01989c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400e9f0 tx=0x7fc9d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.471+0000 7fc9dd7fa640 1 -- 172.21.15.16:0/845991327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400cd80 con 0x7fc9e01024a0 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.471+0000 7fc9dd7fa640 1 -- 172.21.15.16:0/845991327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc9d4005520 con 0x7fc9e01024a0 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.471+0000 7fc9e6655640 1 -- 172.21.15.16:0/845991327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e019d760 con 0x7fc9e01024a0 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.471+0000 7fc9dd7fa640 1 -- 172.21.15.16:0/845991327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4010690 con 0x7fc9e01024a0 2024-03-27T06:45:52.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.471+0000 7fc9e6655640 1 -- 172.21.15.16:0/845991327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e01a4ff0 con 0x7fc9e01024a0 2024-03-27T06:45:52.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:52 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2681006120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:45:52.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.473+0000 7fc9dd7fa640 1 -- 172.21.15.16:0/845991327 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fc9d40046c0 con 0x7fc9e01024a0 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.473+0000 7fc9e9243740 1 Processor -- start 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.473+0000 7fc9e9243740 1 -- start start 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.473+0000 7fc9dd7fa640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc9c80a3780 0x7fc9c80a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.473+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.473+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed19420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.473+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.473+0000 7fc9e9243740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e12ec78710 con 0x55e12ed26400 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.473+0000 7fc9e9243740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e12ec20af0 con 0x55e12ed18500 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.473+0000 7fc9e9243740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e12ec746c0 con 0x55e12ed18fa0 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.473+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc9c80a3780 0x7fc9c80a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9dd7fa640 1 -- 172.21.15.16:0/845991327 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fc9d40d0510 con 0x7fc9e01024a0 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9e4bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9dffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9dffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed189d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:45382/0 (socket says 172.21.15.16:45382) 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed19420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9dffff640 1 -- 172.21.15.16:0/894540268 learned_addr learned my addr 172.21.15.16:0/894540268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9df7fe640 1 -- 172.21.15.16:0/894540268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 msgr2=0x55e12ed189d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9df7fe640 1 --2- 172.21.15.16:0/894540268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed189d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9df7fe640 1 -- 172.21.15.16:0/894540268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 msgr2=0x55e12ed287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9df7fe640 1 --2- 172.21.15.16:0/894540268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed287f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9df7fe640 1 -- 172.21.15.16:0/894540268 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e12e98c3f0 con 0x55e12ed18fa0 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.474+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc9c80a3780 0x7fc9c80a5c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8005cc0 tx=0x7fc9d8032d30 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.475+0000 7fc9df7fe640 1 --2- 172.21.15.16:0/894540268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed19420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40cdc90 tx=0x7fc9d409c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99e81806c164f070 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.475+0000 7fc9c67fc640 1 -- 172.21.15.16:0/894540268 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d409cdd0 con 0x55e12ed18fa0 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.475+0000 7fc9c67fc640 1 -- 172.21.15.16:0/894540268 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc9d409c6d0 con 0x55e12ed18fa0 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.475+0000 7fc9c67fc640 1 -- 172.21.15.16:0/894540268 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40a50b0 con 0x55e12ed18fa0 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.475+0000 7fc9e9243740 1 -- 172.21.15.16:0/894540268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 msgr2=0x55e12ed19420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.475+0000 7fc9e9243740 1 --2- 172.21.15.16:0/894540268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed19420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40cdc90 tx=0x7fc9d409c2d0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.475+0000 7fc9e9243740 1 -- 172.21.15.16:0/894540268 shutdown_connections 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.475+0000 7fc9e9243740 1 --2- 172.21.15.16:0/894540268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.475+0000 7fc9e9243740 1 --2- 172.21.15.16:0/894540268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed19420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.475+0000 7fc9e9243740 1 --2- 172.21.15.16:0/894540268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 1 -- 172.21.15.16:0/894540268 >> 172.21.15.16:0/894540268 conn(0x55e12ed13230 msgr2=0x55e12ed13610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 1 -- 172.21.15.16:0/894540268 shutdown_connections 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 1 -- 172.21.15.16:0/894540268 wait complete. 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 1 -- start start 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 1 Processor -- start 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed0f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed0fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed101f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e12ed29210 con 0x55e12ed26400 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e12ec33eb0 con 0x55e12ed18500 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e12ed29390 con 0x55e12ed18fa0 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.476+0000 7fc9e9243740 10 client.0 _ll_register_callbacks cb 0x55e12ec706a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed0fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9e4bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed0fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:48468/0 (socket says 172.21.15.16:48468) 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9df7fe640 1 -- 172.21.15.16:0/4096025985 learned_addr learned my addr 172.21.15.16:0/4096025985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9dffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed0f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9e4bf2640 1 -- 172.21.15.16:0/4096025985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 msgr2=0x55e12ed0fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/4096025985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed0fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9e4bf2640 1 -- 172.21.15.16:0/4096025985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 msgr2=0x55e12ed0f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/4096025985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed0f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9e4bf2640 1 -- 172.21.15.16:0/4096025985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e12ea567d0 con 0x55e12ed26400 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/4096025985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed101f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc9d802c630 tx=0x7fc9d802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3ee24377f9b67a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/4096025985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8039040 con 0x55e12ed26400 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/4096025985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fc9d8004590 con 0x55e12ed26400 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9e9243740 1 -- 172.21.15.16:0/4096025985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d409c6d0 con 0x55e12ed26400 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/4096025985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc9d8004790 con 0x55e12ed26400 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9e9243740 10 client.16314 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.477+0000 7fc9e9243740 1 -- 172.21.15.16:0/4096025985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e12ed35410 con 0x55e12ed26400 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.478+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/4096025985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc9d409c6d0 con 0x55e12ed26400 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.478+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/4096025985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fc9d803f030 con 0x55e12ed26400 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.478+0000 7fc9e9243740 10 client.16314 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.478+0000 7fc9e9243740 10 client.16314 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T06:45:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.478+0000 7fc9e9243740 20 client.16314 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.478+0000 7fc9e9243740 1 --2- 172.21.15.16:0/4096025985 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55e12ed36920 0x55e12ed38d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.478+0000 7fc9e9243740 4 client.16314 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.478+0000 7fc9e9243740 1 -- 172.21.15.16:0/4096025985 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e12ec85c50 con 0x55e12ed36920 2024-03-27T06:45:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.478+0000 7fc9dffff640 1 --2- 172.21.15.16:0/4096025985 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55e12ed36920 0x55e12ed38d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.479+0000 7fc9dffff640 1 --2- 172.21.15.16:0/4096025985 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55e12ed36920 0x55e12ed38d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49b08fdaa6a7b6d8 server_cookie=a63ecc7830cb3a67 in_seq=0 out_seq=0 2024-03-27T06:45:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.479+0000 7fc9bf7fe640 10 client.16314 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:45:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.480+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/4096025985 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e12ec85c50 con 0x55e12ed36920 2024-03-27T06:45:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.480+0000 7fc9bf7fe640 10 client.16314 handle_command_reply: tid=0 2024-03-27T06:45:52.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.480+0000 7fc9e9243740 1 client.16314 shutdown 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.480+0000 7fc9e9243740 1 -- 172.21.15.16:0/4096025985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 msgr2=0x55e12ed101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.480+0000 7fc9e9243740 1 --2- 172.21.15.16:0/4096025985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed101f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc9d802c630 tx=0x7fc9d802c0f0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.481+0000 7fc9e9243740 1 -- 172.21.15.16:0/4096025985 shutdown_connections 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.481+0000 7fc9e9243740 1 --2- 172.21.15.16:0/4096025985 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55e12ed36920 0x55e12ed38d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.481+0000 7fc9e9243740 1 --2- 172.21.15.16:0/4096025985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed101f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.481+0000 7fc9e9243740 1 --2- 172.21.15.16:0/4096025985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed0fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.481+0000 7fc9bf7fe640 0 client.16314 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.481+0000 7fc9e9243740 1 --2- 172.21.15.16:0/4096025985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed0f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.481+0000 7fc9e9243740 1 -- 172.21.15.16:0/4096025985 >> 172.21.15.16:0/4096025985 conn(0x55e12ed13230 msgr2=0x55e12ed15040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.481+0000 7fc9e9243740 1 -- 172.21.15.16:0/4096025985 shutdown_connections 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.481+0000 7fc9e9243740 1 -- 172.21.15.16:0/4096025985 wait complete. 2024-03-27T06:45:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.481+0000 7fc9e9243740 20 client.16314 trim_cache size 0 max 16384 2024-03-27T06:45:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9e9243740 1 Processor -- start 2024-03-27T06:45:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9e9243740 1 -- start start 2024-03-27T06:45:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed11ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed31ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed13610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9e9243740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01ab6c0 con 0x55e12ed26400 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9e9243740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01ad2c0 con 0x55e12ed18500 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9e9243740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01abd90 con 0x55e12ed18fa0 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9e4bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed13610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9e4bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed13610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32852/0 (socket says 172.21.15.16:32852) 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9e4bf2640 1 -- 172.21.15.16:0/3824826231 learned_addr learned my addr 172.21.15.16:0/3824826231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.499+0000 7fc9dffff640 1 --2- 172.21.15.16:0/3824826231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed11ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.500+0000 7fc9df7fe640 1 --2- 172.21.15.16:0/3824826231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed31ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.500+0000 7fc9e4bf2640 1 -- 172.21.15.16:0/3824826231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 msgr2=0x55e12ed31ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.500+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/3824826231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed31ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.500+0000 7fc9e4bf2640 1 -- 172.21.15.16:0/3824826231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 msgr2=0x55e12ed11ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.500+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/3824826231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed11ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.500+0000 7fc9e4bf2640 1 -- 172.21.15.16:0/3824826231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e12ed18d40 con 0x55e12ed26400 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.500+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/3824826231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed13610 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80095a0 tx=0x7fc9d80095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae071064eb712be5 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.500+0000 7fc9c5ffb640 1 -- 172.21.15.16:0/3824826231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d805b070 con 0x55e12ed26400 2024-03-27T06:45:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.500+0000 7fc9c5ffb640 1 -- 172.21.15.16:0/3824826231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc9d8032670 con 0x55e12ed26400 2024-03-27T06:45:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.500+0000 7fc9c5ffb640 1 -- 172.21.15.16:0/3824826231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8039040 con 0x55e12ed26400 2024-03-27T06:45:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.501+0000 7fc9e9243740 1 -- 172.21.15.16:0/3824826231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 msgr2=0x55e12ed13610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.501+0000 7fc9e9243740 1 --2- 172.21.15.16:0/3824826231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed13610 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80095a0 tx=0x7fc9d80095d0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.501+0000 7fc9e9243740 1 -- 172.21.15.16:0/3824826231 shutdown_connections 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.501+0000 7fc9e9243740 1 --2- 172.21.15.16:0/3824826231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed13610 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.501+0000 7fc9e9243740 1 --2- 172.21.15.16:0/3824826231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed31ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.501+0000 7fc9e9243740 1 --2- 172.21.15.16:0/3824826231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed11ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.501+0000 7fc9e9243740 1 -- 172.21.15.16:0/3824826231 >> 172.21.15.16:0/3824826231 conn(0x55e12ed13230 msgr2=0x55e12ed0e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.501+0000 7fc9e9243740 1 -- 172.21.15.16:0/3824826231 shutdown_connections 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.501+0000 7fc9e9243740 1 -- 172.21.15.16:0/3824826231 wait complete. 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.501+0000 7fc9e9243740 1 -- start start 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e9243740 1 Processor -- start 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed2e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed2b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e9243740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed2bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e9243740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01acbf0 con 0x55e12ed26400 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e9243740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01ac460 con 0x55e12ed18500 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e9243740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01a96a0 con 0x55e12ed18fa0 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e9243740 10 client.0 _ll_register_callbacks cb 0x7fc9d8003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e4bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed2bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e4bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed2bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32856/0 (socket says 172.21.15.16:32856) 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e4bf2640 1 -- 172.21.15.16:0/3371392584 learned_addr learned my addr 172.21.15.16:0/3371392584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9df7fe640 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed2b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9dffff640 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed2e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e4bf2640 1 -- 172.21.15.16:0/3371392584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 msgr2=0x55e12ed2b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed2b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e4bf2640 1 -- 172.21.15.16:0/3371392584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 msgr2=0x55e12ed2e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed2e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9e4bf2640 1 -- 172.21.15.16:0/3371392584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e12ed2c070 con 0x55e12ed26400 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.502+0000 7fc9df7fe640 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed2b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9e4bf2640 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed2bb30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc9d802c700 tx=0x7fc9d8004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0f19642a5d8e1a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/3371392584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8064030 con 0x55e12ed26400 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/3371392584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fc9d8004410 con 0x55e12ed26400 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9e9243740 1 -- 172.21.15.16:0/3371392584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d8032670 con 0x55e12ed26400 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/3371392584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc9d8070050 con 0x55e12ed26400 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9e9243740 10 client.16320 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9e9243740 1 -- 172.21.15.16:0/3371392584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e12ed9d8a0 con 0x55e12ed26400 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/3371392584 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc9d8032670 con 0x55e12ed26400 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/3371392584 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fc9d803f040 con 0x55e12ed26400 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9e9243740 10 client.16320 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9e9243740 10 client.16320 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9e9243740 20 client.16320 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9e9243740 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55e12ed9deb0 0x55e12eda02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9e9243740 4 client.16320 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9e9243740 1 -- 172.21.15.16:0/3371392584 --> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e12ec7ffa0 con 0x55e12ed9deb0 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.503+0000 7fc9dffff640 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55e12ed9deb0 0x55e12eda02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.504+0000 7fc9bf7fe640 10 client.16320 ms_handle_connect on v2:172.21.15.156:6836/3016592835 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.504+0000 7fc9dffff640 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55e12ed9deb0 0x55e12eda02d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=83f5f02bff00439f server_cookie=6e04cc0441cb834 in_seq=0 out_seq=0 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/3371392584 <== mds.1 v2:172.21.15.156:6836/3016592835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+19656 (crc 0 0 0) 0x55e12ec7ffa0 con 0x55e12ed9deb0 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9bf7fe640 10 client.16320 handle_command_reply: tid=0 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 1 client.16320 shutdown 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 1 -- 172.21.15.16:0/3371392584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 msgr2=0x55e12ed2bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed2bb30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc9d802c700 tx=0x7fc9d8004700 comp rx=0 tx=0).stop 2024-03-27T06:45:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 1 -- 172.21.15.16:0/3371392584 shutdown_connections 2024-03-27T06:45:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.156:6836/3016592835,v1:172.21.15.156:6837/3016592835] conn(0x55e12ed9deb0 0x55e12eda02d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e12ed26400 0x55e12ed2bb30 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e12ed18fa0 0x55e12ed2b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9bf7fe640 0 client.16320 ms_handle_reset on v2:172.21.15.156:6836/3016592835 2024-03-27T06:45:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 1 --2- 172.21.15.16:0/3371392584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e12ed18500 0x55e12ed2e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 1 -- 172.21.15.16:0/3371392584 >> 172.21.15.16:0/3371392584 conn(0x55e12ed13230 msgr2=0x55e12ed0f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 1 -- 172.21.15.16:0/3371392584 shutdown_connections 2024-03-27T06:45:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 1 -- 172.21.15.16:0/3371392584 wait complete. 2024-03-27T06:45:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.505+0000 7fc9e9243740 20 client.16320 trim_cache size 0 max 16384 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 39912893, 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631369, 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.CAPI" 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 46279368, 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631377, 2024-03-27T06:45:52.507 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.concap" 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 127420783, 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631320, 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/alps.txt" 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 142148348, 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631371, 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README" 2024-03-27T06:45:52.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 195051959, 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631326, 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cma3000_d0x.txt" 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 210741439, 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631333, 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gameport-programming.txt" 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.509 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 211627313, 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631380, 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.gigaset" 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 302368178, 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631365, 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/00-INDEX" 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.510 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 312998016, 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631344, 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ntrig.txt" 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 399275950, 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631343, 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/notifier.txt" 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 403304146, 2024-03-27T06:45:52.511 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631373, 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.HiSax" 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 444953834, 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631931, 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/codec.txt" 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 449203161, 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631329, 2024-03-27T06:45:52.512 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/elantech.txt" 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 461299106, 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631934, 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/machine.txt" 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 507114045, 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631330, 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/event-codes.txt" 2024-03-27T06:45:52.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 516297811, 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631384, 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.mISDN" 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 545987301, 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631322, 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/appletouch.txt" 2024-03-27T06:45:52.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 596050654, 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631323, 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/atarikbd.txt" 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 929697561, 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631769, 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.c" 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.515 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 942274796, 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631321, 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/amijoy.txt" 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1013470728, 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631328, 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/edt-ft5x06.txt" 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1026358664, 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631930, 2024-03-27T06:45:52.516 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt" 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1173100648, 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631367, 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/HiSax.cert" 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1178556972, 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631325, 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cd32.txt" 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.517 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1371754192, 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631372, 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.FAQ" 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1417922089, 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631932, 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt" 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1574168771, 2024-03-27T06:45:52.518 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631766, 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/.gitignore" 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1697867763, 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631346, 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/sentelic.txt" 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1705615949, 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631388, 2024-03-27T06:45:52.519 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.x25" 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1792980085, 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631385, 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.pcbit" 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1852874140, 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631928, 2024-03-27T06:45:52.520 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt" 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1867242192, 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511632172, 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/README" 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1868019874, 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631387, 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.syncppp" 2024-03-27T06:45:52.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1878406182, 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631382, 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hysdn" 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 1911771093, 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631389, 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/syncPPP.FAQ" 2024-03-27T06:45:52.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2034900017, 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631768, 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/ptp.txt" 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2045104049, 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511632171, 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices" 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.523 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2143235550, 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631332, 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gamepad.txt" 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2230377065, 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631378, 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.diversion" 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.524 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2371627781, 2024-03-27T06:45:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631375, 2024-03-27T06:45:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.audio" 2024-03-27T06:45:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2510805799, 2024-03-27T06:45:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631935, 2024-03-27T06:45:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/overview.txt" 2024-03-27T06:45:52.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2553737211, 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631348, 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/walkera0701.txt" 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2557761001, 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631775, 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/sysfs.txt" 2024-03-27T06:45:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2855466971, 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511632173, 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt" 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2901968678, 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631349, 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/xpad.txt" 2024-03-27T06:45:52.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2911035915, 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631347, 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/shape.fig" 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2912839053, 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631345, 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/rotary-encoder.txt" 2024-03-27T06:45:52.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2979598362, 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631334, 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gpio-tilt.txt" 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 2987985844, 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631331, 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ff.txt" 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.529 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3040112833, 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631368, 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE" 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3103857703, 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631386, 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.sc" 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3157954089, 2024-03-27T06:45:52.530 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631376, 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.avmb1" 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3174494685, 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631366, 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/CREDITS" 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3214597914, 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631342, 2024-03-27T06:45:52.531 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/multi-touch-protocol.txt" 2024-03-27T06:45:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3265400967, 2024-03-27T06:45:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631936, 2024-03-27T06:45:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/platform.txt" 2024-03-27T06:45:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.532 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3382087321, 2024-03-27T06:45:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511632168, 2024-03-27T06:45:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/00-INDEX" 2024-03-27T06:45:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3405024768, 2024-03-27T06:45:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631374, 2024-03-27T06:45:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.act2000" 2024-03-27T06:45:52.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3409586040, 2024-03-27T06:45:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631335, 2024-03-27T06:45:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/iforce-protocol.txt" 2024-03-27T06:45:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.534 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3435996754, 2024-03-27T06:45:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631336, 2024-03-27T06:45:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input-programming.txt" 2024-03-27T06:45:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3479849024, 2024-03-27T06:45:52.535 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631337, 2024-03-27T06:45:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input.txt" 2024-03-27T06:45:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3512580874, 2024-03-27T06:45:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631937, 2024-03-27T06:45:52.536 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt" 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3522456676, 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631327, 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cs461x.txt" 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3582788378, 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631381, 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hfc-pci" 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.537 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3598160325, 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631929, 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt" 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3644674861, 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631767, 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/Makefile" 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3775633484, 2024-03-27T06:45:52.538 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631324, 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/bcm5974.txt" 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3810628677, 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511632169, 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/api.txt" 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3841478924, 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631776, 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/tsi721.txt" 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.539 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3855060960, 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631383, 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.icn" 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3861329718, 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631933, 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/jack.txt" 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3959690012, 2024-03-27T06:45:52.540 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631379, 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.fax" 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3967530483, 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631339, 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-api.txt" 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3969809295, 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631338, 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/interactive.fig" 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 3971507176, 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631774, 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/rapidio.txt" 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4017030167, 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511632170, 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/cpuid.txt" 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.542 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4053817291, 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631340, 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-parport.txt" 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4122562986, 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631350, 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/yealink.txt" 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4192299098, 2024-03-27T06:45:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631341, 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick.txt" 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4215752132, 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631370, 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.fax" 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "damage_type": "backtrace", 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4252055089, 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1099511631770, 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.mk" 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:45:52.544 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-03-27T06:45:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 -- 172.21.15.16:0/845991327 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc9c80a3780 msgr2=0x7fc9c80a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc9c80a3780 0x7fc9c80a5c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8005cc0 tx=0x7fc9d8032d30 comp rx=0 tx=0).stop 2024-03-27T06:45:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 -- 172.21.15.16:0/845991327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01024a0 msgr2=0x7fc9e01989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01024a0 0x7fc9e01989c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400e9f0 tx=0x7fc9d400edf0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 -- 172.21.15.16:0/845991327 shutdown_connections 2024-03-27T06:45:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fc9c80a3780 0x7fc9c80a5c40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e0107b80 0x7fc9e019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01024a0 0x7fc9e01989c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 --2- 172.21.15.16:0/845991327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e0101ae0 0x7fc9e0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 -- 172.21.15.16:0/845991327 >> 172.21.15.16:0/845991327 conn(0x7fc9e0078f50 msgr2=0x7fc9e01043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 -- 172.21.15.16:0/845991327 shutdown_connections 2024-03-27T06:45:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.506+0000 7fc9e6655640 1 -- 172.21.15.16:0/845991327 wait complete. 2024-03-27T06:45:52.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-27T06:45:52.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.664+0000 7f18f5db5640 1 -- 172.21.15.16:0/797043345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01069d0 msgr2=0x7f18f0106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.664+0000 7f18f5db5640 1 --2- 172.21.15.16:0/797043345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01069d0 0x7f18f0106db0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009930 tx=0x7f18d802bc30 comp rx=0 tx=0).stop 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.664+0000 7f18f5db5640 1 -- 172.21.15.16:0/797043345 shutdown_connections 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.664+0000 7f18f5db5640 1 --2- 172.21.15.16:0/797043345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18f0101380 0x7f18f010d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.664+0000 7f18f5db5640 1 --2- 172.21.15.16:0/797043345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18f01009c0 0x7f18f0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.664+0000 7f18f5db5640 1 --2- 172.21.15.16:0/797043345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01069d0 0x7f18f0106db0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.664+0000 7f18f5db5640 1 -- 172.21.15.16:0/797043345 >> 172.21.15.16:0/797043345 conn(0x7f18f0077b50 msgr2=0x7f18f0077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.664+0000 7f18f5db5640 1 -- 172.21.15.16:0/797043345 shutdown_connections 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.664+0000 7f18f5db5640 1 -- 172.21.15.16:0/797043345 wait complete. 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.665+0000 7f18f5db5640 1 Processor -- start 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.665+0000 7f18f5db5640 1 -- start start 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.665+0000 7f18f5db5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01009c0 0x7f18f0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.665+0000 7f18f5db5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18f0101380 0x7f18f0199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.665+0000 7f18f5db5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18f01069d0 0x7f18f01a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.665+0000 7f18f5db5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0110eb0 con 0x7f18f01009c0 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.665+0000 7f18f5db5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0110d30 con 0x7f18f01069d0 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.665+0000 7f18f5db5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0111030 con 0x7f18f0101380 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01009c0 0x7f18f0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01009c0 0x7f18f0199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32876/0 (socket says 172.21.15.16:32876) 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18f0101380 0x7f18f0199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ef7fe640 1 -- 172.21.15.16:0/4179172418 learned_addr learned my addr 172.21.15.16:0/4179172418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18effff640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18f01069d0 0x7f18f01a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ef7fe640 1 -- 172.21.15.16:0/4179172418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18f0101380 msgr2=0x7f18f0199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ef7fe640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18f0101380 0x7f18f0199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ef7fe640 1 -- 172.21.15.16:0/4179172418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18f01069d0 msgr2=0x7f18f01a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ef7fe640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18f01069d0 0x7f18f01a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ef7fe640 1 -- 172.21.15.16:0/4179172418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f01a4910 con 0x7f18f01009c0 2024-03-27T06:45:52.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ef7fe640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01009c0 0x7f18f0199350 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009900 tx=0x7f18d8004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ecff9640 1 -- 172.21.15.16:0/4179172418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8043020 con 0x7f18f01009c0 2024-03-27T06:45:52.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18f5db5640 1 -- 172.21.15.16:0/4179172418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f01a4ae0 con 0x7f18f01009c0 2024-03-27T06:45:52.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ecff9640 1 -- 172.21.15.16:0/4179172418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f18d8004e50 con 0x7f18f01009c0 2024-03-27T06:45:52.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18ecff9640 1 -- 172.21.15.16:0/4179172418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8003a10 con 0x7f18f01009c0 2024-03-27T06:45:52.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.666+0000 7f18f5db5640 1 -- 172.21.15.16:0/4179172418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f01a5020 con 0x7f18f01009c0 2024-03-27T06:45:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.668+0000 7f18ecff9640 1 -- 172.21.15.16:0/4179172418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f18d8046020 con 0x7f18f01009c0 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18f899e740 1 Processor -- start 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18f899e740 1 -- start start 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18ecff9640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f18dc0a3780 0x7f18dc0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18effff640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f18dc0a3780 0x7f18dc0a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18ecff9640 1 -- 172.21.15.16:0/4179172418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f18d80f1ea0 con 0x7f18f01009c0 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18f899e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a67b0 0x5561c92a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18f899e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18f899e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92afcb0 0x5561c92b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18f899e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5561c920a710 con 0x5561c92a70d0 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18f899e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5561c91b2af0 con 0x5561c92afcb0 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18f899e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5561c92066c0 con 0x5561c92a67b0 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32890/0 (socket says 172.21.15.16:32890) 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18eeffd640 1 -- 172.21.15.16:0/413111523 learned_addr learned my addr 172.21.15.16:0/413111523 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18ef7fe640 1 --2- 172.21.15.16:0/413111523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a67b0 0x5561c92a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.669+0000 7f18effff640 1 --2- 172.21.15.16:0/413111523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92afcb0 0x5561c92b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.670+0000 7f18eeffd640 1 -- 172.21.15.16:0/413111523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a67b0 msgr2=0x5561c92a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.670+0000 7f18eeffd640 1 --2- 172.21.15.16:0/413111523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a67b0 0x5561c92a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.670+0000 7f18eeffd640 1 -- 172.21.15.16:0/413111523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92afcb0 msgr2=0x5561c92b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.670+0000 7f18eeffd640 1 --2- 172.21.15.16:0/413111523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92afcb0 0x5561c92b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.670+0000 7f18eeffd640 1 -- 172.21.15.16:0/413111523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561c8f1e3f0 con 0x5561c92a70d0 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.670+0000 7f18ef7fe640 1 --2- 172.21.15.16:0/413111523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a67b0 0x5561c92a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.670+0000 7f18effff640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f18dc0a3780 0x7f18dc0a5c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f18e8005fd0 tx=0x7f18e8011b40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.670+0000 7f18eeffd640 1 --2- 172.21.15.16:0/413111523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92af770 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009c90 tx=0x7f18e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1615647b54cde04 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.670+0000 7f18e5ffb640 1 -- 172.21.15.16:0/413111523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e001e070 con 0x5561c92a70d0 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.670+0000 7f18e5ffb640 1 -- 172.21.15.16:0/413111523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f18e0009d80 con 0x5561c92a70d0 2024-03-27T06:45:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.670+0000 7f18e5ffb640 1 -- 172.21.15.16:0/413111523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0018870 con 0x5561c92a70d0 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.671+0000 7f18f899e740 1 -- 172.21.15.16:0/413111523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 msgr2=0x5561c92af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.671+0000 7f18f899e740 1 --2- 172.21.15.16:0/413111523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92af770 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009c90 tx=0x7f18e0011810 comp rx=0 tx=0).stop 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.671+0000 7f18f899e740 1 -- 172.21.15.16:0/413111523 shutdown_connections 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.671+0000 7f18f899e740 1 --2- 172.21.15.16:0/413111523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92afcb0 0x5561c92b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.671+0000 7f18f899e740 1 --2- 172.21.15.16:0/413111523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92af770 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.671+0000 7f18f899e740 1 --2- 172.21.15.16:0/413111523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a67b0 0x5561c92a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.671+0000 7f18f899e740 1 -- 172.21.15.16:0/413111523 >> 172.21.15.16:0/413111523 conn(0x5561c92a0ee0 msgr2=0x5561c92a12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.671+0000 7f18f899e740 1 -- 172.21.15.16:0/413111523 shutdown_connections 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.671+0000 7f18f899e740 1 -- 172.21.15.16:0/413111523 wait complete. 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.671+0000 7f18f899e740 1 -- start start 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.671+0000 7f18f899e740 1 Processor -- start 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18f899e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92a67b0 0x5561c92accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18f899e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a70d0 0x5561c92ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18f899e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92afcb0 0x5561c92ad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18f899e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5561c92b2aa0 con 0x5561c92afcb0 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18f899e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5561c91c5eb0 con 0x5561c92a67b0 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18f899e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5561c92b2c20 con 0x5561c92a70d0 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18f899e740 10 client.0 _ll_register_callbacks cb 0x5561c92026a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92afcb0 0x5561c92ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18ef7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92a67b0 0x5561c92accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92afcb0 0x5561c92ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32892/0 (socket says 172.21.15.16:32892) 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18effff640 1 -- 172.21.15.16:0/4091523183 learned_addr learned my addr 172.21.15.16:0/4091523183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18ef7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92a67b0 0x5561c92accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:45456/0 (socket says 172.21.15.16:45456) 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18eeffd640 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a70d0 0x5561c92ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18effff640 1 -- 172.21.15.16:0/4091523183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a70d0 msgr2=0x5561c92ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18effff640 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a70d0 0x5561c92ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18effff640 1 -- 172.21.15.16:0/4091523183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92a67b0 msgr2=0x5561c92accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18effff640 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92a67b0 0x5561c92accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18effff640 1 -- 172.21.15.16:0/4091523183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561c8fe87d0 con 0x5561c92afcb0 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18effff640 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92afcb0 0x5561c92ad750 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f18e800ff40 tx=0x7f18e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ea1d1f4e964f58 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18ceffd640 1 -- 172.21.15.16:0/4091523183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e801a040 con 0x5561c92afcb0 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18ceffd640 1 -- 172.21.15.16:0/4091523183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f18e8004750 con 0x5561c92afcb0 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18f899e740 1 -- 172.21.15.16:0/4091523183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e0009d80 con 0x5561c92afcb0 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.672+0000 7f18ceffd640 1 -- 172.21.15.16:0/4091523183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f18e8020470 con 0x5561c92afcb0 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.673+0000 7f18f899e740 10 client.16332 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.673+0000 7f18f899e740 1 -- 172.21.15.16:0/4091523183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5561c92bed50 con 0x5561c92afcb0 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.673+0000 7f18ceffd640 1 -- 172.21.15.16:0/4091523183 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f18e0009d80 con 0x5561c92afcb0 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.673+0000 7f18ceffd640 1 -- 172.21.15.16:0/4091523183 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f18e801d030 con 0x5561c92afcb0 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.673+0000 7f18f899e740 10 client.16332 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.673+0000 7f18f899e740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.673+0000 7f18f899e740 20 client.16332 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.673+0000 7f18f899e740 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5561c92c02d0 0x5561c92c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.673+0000 7f18f899e740 4 client.16332 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:45:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.673+0000 7f18f899e740 1 -- 172.21.15.16:0/4091523183 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561c9217c50 con 0x5561c92c02d0 2024-03-27T06:45:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.673+0000 7f18ef7fe640 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5561c92c02d0 0x5561c92c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.674+0000 7f18ef7fe640 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5561c92c02d0 0x5561c92c2710 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b9e8dc1fe63a81 server_cookie=de04a63e8e6dbf70 in_seq=0 out_seq=0 2024-03-27T06:45:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.674+0000 7f18ceffd640 10 client.16332 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18ceffd640 1 -- 172.21.15.16:0/4091523183 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5561c9217c50 con 0x5561c92c02d0 2024-03-27T06:45:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18ceffd640 10 client.16332 handle_command_reply: tid=0 2024-03-27T06:45:52.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 1 client.16332 shutdown 2024-03-27T06:45:52.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 1 -- 172.21.15.16:0/4091523183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92afcb0 msgr2=0x5561c92ad750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92afcb0 0x5561c92ad750 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f18e800ff40 tx=0x7f18e8004ab0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 1 -- 172.21.15.16:0/4091523183 shutdown_connections 2024-03-27T06:45:52.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5561c92c02d0 0x5561c92c2710 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92afcb0 0x5561c92ad750 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a70d0 0x5561c92ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 1 --2- 172.21.15.16:0/4091523183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92a67b0 0x5561c92accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18ceffd640 0 client.16332 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:52.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 1 -- 172.21.15.16:0/4091523183 >> 172.21.15.16:0/4091523183 conn(0x5561c92a0ee0 msgr2=0x5561c92a2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 1 -- 172.21.15.16:0/4091523183 shutdown_connections 2024-03-27T06:45:52.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 1 -- 172.21.15.16:0/4091523183 wait complete. 2024-03-27T06:45:52.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.675+0000 7f18f899e740 20 client.16332 trim_cache size 0 max 16384 2024-03-27T06:45:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18f899e740 1 Processor -- start 2024-03-27T06:45:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18f899e740 1 -- start start 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18f899e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a67b0 0x5561c92af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18f899e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18f899e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92afcb0 0x5561c92c8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18f899e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01ad950 con 0x5561c92a70d0 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18f899e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01af550 con 0x5561c92afcb0 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18f899e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01ae020 con 0x5561c92a67b0 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92bb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32896/0 (socket says 172.21.15.16:32896) 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18eeffd640 1 -- 172.21.15.16:0/1342355670 learned_addr learned my addr 172.21.15.16:0/1342355670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18ef7fe640 1 --2- 172.21.15.16:0/1342355670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a67b0 0x5561c92af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.689+0000 7f18effff640 1 --2- 172.21.15.16:0/1342355670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92afcb0 0x5561c92c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.690+0000 7f18eeffd640 1 -- 172.21.15.16:0/1342355670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a67b0 msgr2=0x5561c92af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.690+0000 7f18eeffd640 1 --2- 172.21.15.16:0/1342355670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a67b0 0x5561c92af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.690+0000 7f18eeffd640 1 -- 172.21.15.16:0/1342355670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92afcb0 msgr2=0x5561c92c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.690+0000 7f18eeffd640 1 --2- 172.21.15.16:0/1342355670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92afcb0 0x5561c92c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.690+0000 7f18eeffd640 1 -- 172.21.15.16:0/1342355670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561c929b410 con 0x5561c92a70d0 2024-03-27T06:45:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.690+0000 7f18eeffd640 1 --2- 172.21.15.16:0/1342355670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92bb870 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009be0 tx=0x7f18e00183b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c84b06384d17f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.690+0000 7f18e57fa640 1 -- 172.21.15.16:0/1342355670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e001e040 con 0x5561c92a70d0 2024-03-27T06:45:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.690+0000 7f18e57fa640 1 -- 172.21.15.16:0/1342355670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f18e0009e00 con 0x5561c92a70d0 2024-03-27T06:45:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.690+0000 7f18e57fa640 1 -- 172.21.15.16:0/1342355670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0023460 con 0x5561c92a70d0 2024-03-27T06:45:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 -- 172.21.15.16:0/1342355670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 msgr2=0x5561c92bb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 --2- 172.21.15.16:0/1342355670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92bb870 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009be0 tx=0x7f18e00183b0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 -- 172.21.15.16:0/1342355670 shutdown_connections 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 --2- 172.21.15.16:0/1342355670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92afcb0 0x5561c92c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 --2- 172.21.15.16:0/1342355670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92bb870 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 --2- 172.21.15.16:0/1342355670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92a67b0 0x5561c92af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 -- 172.21.15.16:0/1342355670 >> 172.21.15.16:0/1342355670 conn(0x5561c92a0ee0 msgr2=0x5561c92a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 -- 172.21.15.16:0/1342355670 shutdown_connections 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 -- 172.21.15.16:0/1342355670 wait complete. 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 -- start start 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 Processor -- start 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92a67b0 0x5561c92a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92afcb0 0x5561c9325650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01aee80 con 0x5561c92a70d0 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01ae6f0 con 0x5561c92a67b0 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01ab930 con 0x5561c92afcb0 2024-03-27T06:45:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.691+0000 7f18f899e740 10 client.0 _ll_register_callbacks cb 0x7f18e8009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92aa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32904/0 (socket says 172.21.15.16:32904) 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18ef7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92a67b0 0x5561c92a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18eeffd640 1 -- 172.21.15.16:0/1520011509 learned_addr learned my addr 172.21.15.16:0/1520011509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18effff640 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92afcb0 0x5561c9325650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18eeffd640 1 -- 172.21.15.16:0/1520011509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92afcb0 msgr2=0x5561c9325650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18eeffd640 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92afcb0 0x5561c9325650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18eeffd640 1 -- 172.21.15.16:0/1520011509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92a67b0 msgr2=0x5561c92a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18eeffd640 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92a67b0 0x5561c92a9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18eeffd640 1 -- 172.21.15.16:0/1520011509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561c9325b90 con 0x5561c92a70d0 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18eeffd640 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92aa150 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009c90 tx=0x7f18e0003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda21e7d667d3e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:45:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18ceffd640 1 -- 172.21.15.16:0/1520011509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e001e030 con 0x5561c92a70d0 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18ceffd640 1 -- 172.21.15.16:0/1520011509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f18e00052d0 con 0x5561c92a70d0 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18f899e740 1 -- 172.21.15.16:0/1520011509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e0009e00 con 0x5561c92a70d0 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18ceffd640 1 -- 172.21.15.16:0/1520011509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f18e00054d0 con 0x5561c92a70d0 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18f899e740 10 client.16338 fetch_fsmap learned FSMap version 31 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18f899e740 1 -- 172.21.15.16:0/1520011509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5561c9326cf0 con 0x5561c92a70d0 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.692+0000 7f18ceffd640 1 -- 172.21.15.16:0/1520011509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f18e0009e00 con 0x5561c92a70d0 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.693+0000 7f18ceffd640 1 -- 172.21.15.16:0/1520011509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f18e0019410 con 0x5561c92a70d0 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.693+0000 7f18f899e740 10 client.16338 fetch_fsmap finished waiting for FSMap version 31 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.693+0000 7f18f899e740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.693+0000 7f18f899e740 20 client.16338 populate_metadata read hostname 'smithi016' 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.693+0000 7f18f899e740 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5561c93272c0 0x5561c93296e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.693+0000 7f18f899e740 4 client.16338 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.693+0000 7f18f899e740 1 -- 172.21.15.16:0/1520011509 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561c9211fa0 con 0x5561c93272c0 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.693+0000 7f18ef7fe640 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5561c93272c0 0x5561c93296e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.693+0000 7f18ef7fe640 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5561c93272c0 0x5561c93296e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ca363f6436ea68 server_cookie=d404b2c5b0da6934 in_seq=0 out_seq=0 2024-03-27T06:45:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.693+0000 7f18ceffd640 10 client.16338 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:52.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.694+0000 7f18ceffd640 1 -- 172.21.15.16:0/1520011509 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561c9211fa0 con 0x5561c93272c0 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.694+0000 7f18ceffd640 10 client.16338 handle_command_reply: tid=0 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.694+0000 7f18f899e740 1 client.16338 shutdown 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f899e740 1 -- 172.21.15.16:0/1520011509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 msgr2=0x5561c92aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f899e740 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92aa150 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009c90 tx=0x7f18e0003a40 comp rx=0 tx=0).stop 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f899e740 1 -- 172.21.15.16:0/1520011509 shutdown_connections 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f899e740 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5561c93272c0 0x5561c93296e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f899e740 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561c92afcb0 0x5561c9325650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f899e740 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5561c92a70d0 0x5561c92aa150 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f899e740 1 --2- 172.21.15.16:0/1520011509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5561c92a67b0 0x5561c92a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18ceffd640 0 client.16338 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f899e740 1 -- 172.21.15.16:0/1520011509 >> 172.21.15.16:0/1520011509 conn(0x5561c92a0ee0 msgr2=0x5561c92acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f899e740 1 -- 172.21.15.16:0/1520011509 shutdown_connections 2024-03-27T06:45:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f899e740 1 -- 172.21.15.16:0/1520011509 wait complete. 2024-03-27T06:45:52.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f899e740 20 client.16338 trim_cache size 0 max 16384 2024-03-27T06:45:52.696 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-03-27T06:45:52.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 -- 172.21.15.16:0/4179172418 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f18dc0a3780 msgr2=0x7f18dc0a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f18dc0a3780 0x7f18dc0a5c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f18e8005fd0 tx=0x7f18e8011b40 comp rx=0 tx=0).stop 2024-03-27T06:45:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 -- 172.21.15.16:0/4179172418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01009c0 msgr2=0x7f18f0199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:45:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01009c0 0x7f18f0199350 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009900 tx=0x7f18d8004c30 comp rx=0 tx=0).stop 2024-03-27T06:45:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 -- 172.21.15.16:0/4179172418 shutdown_connections 2024-03-27T06:45:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18f01069d0 0x7f18f01a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f18dc0a3780 0x7f18dc0a5c40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18f0101380 0x7f18f0199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 --2- 172.21.15.16:0/4179172418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01009c0 0x7f18f0199350 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:45:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 -- 172.21.15.16:0/4179172418 >> 172.21.15.16:0/4179172418 conn(0x7f18f0077b50 msgr2=0x7f18f0103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:45:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 -- 172.21.15.16:0/4179172418 shutdown_connections 2024-03-27T06:45:52.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:45:52.695+0000 7f18f5db5640 1 -- 172.21.15.16:0/4179172418 wait complete. 2024-03-27T06:45:52.701 ERROR:tasks.fwd_scrub:rank 4 damaged: [{'damage_type': 'backtrace', 'id': 224002763, 'ino': 1099511632166, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/00-INDEX'}, {'damage_type': 'backtrace', 'id': 493159939, 'ino': 1099511632167, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm'}, {'damage_type': 'backtrace', 'id': 2456918354, 'ino': 1099511631698, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/phy/samsung-usb2.txt'}] 2024-03-27T06:45:52.702 ERROR:tasks.fwd_scrub:rank 3 damaged: [{'damage_type': 'backtrace', 'id': 518453461, 'ino': 1099511631491, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/00-INDEX'}, {'damage_type': 'backtrace', 'id': 3536147696, 'ino': 1099511631492, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/AU1xxx_IDE.README'}] 2024-03-27T06:45:52.702 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 288873844, 'ino': 1099511631404, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/makefiles.txt'}, {'damage_type': 'backtrace', 'id': 1317136046, 'ino': 1099511631403, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig.txt'}, {'damage_type': 'backtrace', 'id': 1541207138, 'ino': 1099511631400, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/headers_install.txt'}, {'damage_type': 'backtrace', 'id': 1572893329, 'ino': 1099511631402, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kconfig-language.txt'}, {'damage_type': 'backtrace', 'id': 1813746460, 'ino': 1099511631399, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/00-INDEX'}, {'damage_type': 'backtrace', 'id': 2804642534, 'ino': 1099511631401, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/kbuild.txt'}, {'damage_type': 'backtrace', 'id': 2953760589, 'ino': 1099511631405, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/kbuild/modules.txt'}] 2024-03-27T06:45:52.702 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 39912893, 'ino': 1099511631369, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.CAPI'}, {'damage_type': 'backtrace', 'id': 46279368, 'ino': 1099511631377, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.concap'}, {'damage_type': 'backtrace', 'id': 127420783, 'ino': 1099511631320, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/alps.txt'}, {'damage_type': 'backtrace', 'id': 142148348, 'ino': 1099511631371, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README'}, {'damage_type': 'backtrace', 'id': 195051959, 'ino': 1099511631326, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cma3000_d0x.txt'}, {'damage_type': 'backtrace', 'id': 210741439, 'ino': 1099511631333, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gameport-programming.txt'}, {'damage_type': 'backtrace', 'id': 211627313, 'ino': 1099511631380, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.gigaset'}, {'damage_type': 'backtrace', 'id': 302368178, 'ino': 1099511631365, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/00-INDEX'}, {'damage_type': 'backtrace', 'id': 312998016, 'ino': 1099511631344, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ntrig.txt'}, {'damage_type': 'backtrace', 'id': 399275950, 'ino': 1099511631343, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/notifier.txt'}, {'damage_type': 'backtrace', 'id': 403304146, 'ino': 1099511631373, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.HiSax'}, {'damage_type': 'backtrace', 'id': 444953834, 'ino': 1099511631931, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/codec.txt'}, {'damage_type': 'backtrace', 'id': 449203161, 'ino': 1099511631329, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/elantech.txt'}, {'damage_type': 'backtrace', 'id': 461299106, 'ino': 1099511631934, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/machine.txt'}, {'damage_type': 'backtrace', 'id': 507114045, 'ino': 1099511631330, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/event-codes.txt'}, {'damage_type': 'backtrace', 'id': 516297811, 'ino': 1099511631384, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.mISDN'}, {'damage_type': 'backtrace', 'id': 545987301, 'ino': 1099511631322, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/appletouch.txt'}, {'damage_type': 'backtrace', 'id': 596050654, 'ino': 1099511631323, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/atarikbd.txt'}, {'damage_type': 'backtrace', 'id': 929697561, 'ino': 1099511631769, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.c'}, {'damage_type': 'backtrace', 'id': 942274796, 'ino': 1099511631321, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/amijoy.txt'}, {'damage_type': 'backtrace', 'id': 1013470728, 'ino': 1099511631328, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/edt-ft5x06.txt'}, {'damage_type': 'backtrace', 'id': 1026358664, 'ino': 1099511631930, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt'}, {'damage_type': 'backtrace', 'id': 1173100648, 'ino': 1099511631367, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/HiSax.cert'}, {'damage_type': 'backtrace', 'id': 1178556972, 'ino': 1099511631325, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cd32.txt'}, {'damage_type': 'backtrace', 'id': 1371754192, 'ino': 1099511631372, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.FAQ'}, {'damage_type': 'backtrace', 'id': 1417922089, 'ino': 1099511631932, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt'}, {'damage_type': 'backtrace', 'id': 1574168771, 'ino': 1099511631766, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/.gitignore'}, {'damage_type': 'backtrace', 'id': 1697867763, 'ino': 1099511631346, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/sentelic.txt'}, {'damage_type': 'backtrace', 'id': 1705615949, 'ino': 1099511631388, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.x25'}, {'damage_type': 'backtrace', 'id': 1792980085, 'ino': 1099511631385, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.pcbit'}, {'damage_type': 'backtrace', 'id': 1852874140, 'ino': 1099511631928, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt'}, {'damage_type': 'backtrace', 'id': 1867242192, 'ino': 1099511632172, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/README'}, {'damage_type': 'backtrace', 'id': 1868019874, 'ino': 1099511631387, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.syncppp'}, {'damage_type': 'backtrace', 'id': 1878406182, 'ino': 1099511631382, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hysdn'}, {'damage_type': 'backtrace', 'id': 1911771093, 'ino': 1099511631389, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/syncPPP.FAQ'}, {'damage_type': 'backtrace', 'id': 2034900017, 'ino': 1099511631768, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/ptp.txt'}, {'damage_type': 'backtrace', 'id': 2045104049, 'ino': 1099511632171, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices'}, {'damage_type': 'backtrace', 'id': 2143235550, 'ino': 1099511631332, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gamepad.txt'}, {'damage_type': 'backtrace', 'id': 2230377065, 'ino': 1099511631378, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.diversion'}, {'damage_type': 'backtrace', 'id': 2371627781, 'ino': 1099511631375, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.audio'}, {'damage_type': 'backtrace', 'id': 2510805799, 'ino': 1099511631935, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/overview.txt'}, {'damage_type': 'backtrace', 'id': 2553737211, 'ino': 1099511631348, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/walkera0701.txt'}, {'damage_type': 'backtrace', 'id': 2557761001, 'ino': 1099511631775, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/sysfs.txt'}, {'damage_type': 'backtrace', 'id': 2855466971, 'ino': 1099511632173, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt'}, {'damage_type': 'backtrace', 'id': 2901968678, 'ino': 1099511631349, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/xpad.txt'}, {'damage_type': 'backtrace', 'id': 2911035915, 'ino': 1099511631347, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/shape.fig'}, {'damage_type': 'backtrace', 'id': 2912839053, 'ino': 1099511631345, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/rotary-encoder.txt'}, {'damage_type': 'backtrace', 'id': 2979598362, 'ino': 1099511631334, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/gpio-tilt.txt'}, {'damage_type': 'backtrace', 'id': 2987985844, 'ino': 1099511631331, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/ff.txt'}, {'damage_type': 'backtrace', 'id': 3040112833, 'ino': 1099511631368, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE'}, {'damage_type': 'backtrace', 'id': 3103857703, 'ino': 1099511631386, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.sc'}, {'damage_type': 'backtrace', 'id': 3157954089, 'ino': 1099511631376, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.avmb1'}, {'damage_type': 'backtrace', 'id': 3174494685, 'ino': 1099511631366, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/CREDITS'}, {'damage_type': 'backtrace', 'id': 3214597914, 'ino': 1099511631342, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/multi-touch-protocol.txt'}, {'damage_type': 'backtrace', 'id': 3265400967, 'ino': 1099511631936, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/platform.txt'}, {'damage_type': 'backtrace', 'id': 3382087321, 'ino': 1099511632168, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/00-INDEX'}, {'damage_type': 'backtrace', 'id': 3405024768, 'ino': 1099511631374, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.act2000'}, {'damage_type': 'backtrace', 'id': 3409586040, 'ino': 1099511631335, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/iforce-protocol.txt'}, {'damage_type': 'backtrace', 'id': 3435996754, 'ino': 1099511631336, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input-programming.txt'}, {'damage_type': 'backtrace', 'id': 3479849024, 'ino': 1099511631337, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/input.txt'}, {'damage_type': 'backtrace', 'id': 3512580874, 'ino': 1099511631937, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt'}, {'damage_type': 'backtrace', 'id': 3522456676, 'ino': 1099511631327, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/cs461x.txt'}, {'damage_type': 'backtrace', 'id': 3582788378, 'ino': 1099511631381, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.hfc-pci'}, {'damage_type': 'backtrace', 'id': 3598160325, 'ino': 1099511631929, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt'}, {'damage_type': 'backtrace', 'id': 3644674861, 'ino': 1099511631767, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/Makefile'}, {'damage_type': 'backtrace', 'id': 3775633484, 'ino': 1099511631324, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/bcm5974.txt'}, {'damage_type': 'backtrace', 'id': 3810628677, 'ino': 1099511632169, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/api.txt'}, {'damage_type': 'backtrace', 'id': 3841478924, 'ino': 1099511631776, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/tsi721.txt'}, {'damage_type': 'backtrace', 'id': 3855060960, 'ino': 1099511631383, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.icn'}, {'damage_type': 'backtrace', 'id': 3861329718, 'ino': 1099511631933, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/alsa/soc/jack.txt'}, {'damage_type': 'backtrace', 'id': 3959690012, 'ino': 1099511631379, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/README.fax'}, {'damage_type': 'backtrace', 'id': 3967530483, 'ino': 1099511631339, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-api.txt'}, {'damage_type': 'backtrace', 'id': 3969809295, 'ino': 1099511631338, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/interactive.fig'}, {'damage_type': 'backtrace', 'id': 3971507176, 'ino': 1099511631774, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/rapidio/rapidio.txt'}, {'damage_type': 'backtrace', 'id': 4017030167, 'ino': 1099511632170, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/virtual/kvm/cpuid.txt'}, {'damage_type': 'backtrace', 'id': 4053817291, 'ino': 1099511631340, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick-parport.txt'}, {'damage_type': 'backtrace', 'id': 4122562986, 'ino': 1099511631350, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/yealink.txt'}, {'damage_type': 'backtrace', 'id': 4192299098, 'ino': 1099511631341, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/input/joystick.txt'}, {'damage_type': 'backtrace', 'id': 4215752132, 'ino': 1099511631370, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/isdn/INTERFACE.fax'}, {'damage_type': 'backtrace', 'id': 4252055089, 'ino': 1099511631770, 'path': '/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/ptp/testptp.mk'}] 2024-03-27T06:45:52.702 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-03-27T06:45:53.092 INFO:tasks.workunit.client.0.smithi016.stderr:+ tar c linux-4.0.5 2024-03-27T06:45:53.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:53 smithi170 ceph-mon[36438]: pgmap v73: 97 pgs: 97 active+clean; 147 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 451 op/s 2024-03-27T06:45:53.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:53 smithi016 ceph-mon[31908]: pgmap v73: 97 pgs: 97 active+clean; 147 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 451 op/s 2024-03-27T06:45:53.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:53 smithi156 ceph-mon[36261]: pgmap v73: 97 pgs: 97 active+clean; 147 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 451 op/s 2024-03-27T06:45:53.666 INFO:tasks.workunit.client.0.smithi016.stderr:+ rm -rf linux-4.0.5 2024-03-27T06:45:55.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:55 smithi170 ceph-mon[36438]: pgmap v74: 97 pgs: 97 active+clean; 147 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 392 op/s 2024-03-27T06:45:55.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:55 smithi016 ceph-mon[31908]: pgmap v74: 97 pgs: 97 active+clean; 147 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 392 op/s 2024-03-27T06:45:55.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:55 smithi156 ceph-mon[36261]: pgmap v74: 97 pgs: 97 active+clean; 147 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 392 op/s 2024-03-27T06:45:57.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:57 smithi170 ceph-mon[36438]: pgmap v75: 97 pgs: 97 active+clean; 133 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 945 op/s 2024-03-27T06:45:57.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:57 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:57.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:57 smithi016 ceph-mon[31908]: pgmap v75: 97 pgs: 97 active+clean; 133 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 945 op/s 2024-03-27T06:45:57.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:57 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:57.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:57 smithi156 ceph-mon[36261]: pgmap v75: 97 pgs: 97 active+clean; 133 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 945 op/s 2024-03-27T06:45:57.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:57 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:45:59.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:45:59 smithi170 ceph-mon[36438]: pgmap v76: 97 pgs: 97 active+clean; 133 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 875 op/s 2024-03-27T06:45:59.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:45:59 smithi016 ceph-mon[31908]: pgmap v76: 97 pgs: 97 active+clean; 133 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 875 op/s 2024-03-27T06:45:59.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:45:59 smithi156 ceph-mon[36261]: pgmap v76: 97 pgs: 97 active+clean; 133 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 875 op/s 2024-03-27T06:46:01.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:01 smithi170 ceph-mon[36438]: pgmap v77: 97 pgs: 97 active+clean; 135 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 927 op/s 2024-03-27T06:46:01.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:01 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:01.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:01 smithi016 ceph-mon[31908]: pgmap v77: 97 pgs: 97 active+clean; 135 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 927 op/s 2024-03-27T06:46:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:01 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:01.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:01 smithi156 ceph-mon[36261]: pgmap v77: 97 pgs: 97 active+clean; 135 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 927 op/s 2024-03-27T06:46:01.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:01 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:02.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:02 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:02.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:02 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:02.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:02 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:03.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:03 smithi170 ceph-mon[36438]: pgmap v78: 97 pgs: 97 active+clean; 136 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 880 op/s 2024-03-27T06:46:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:03 smithi016 ceph-mon[31908]: pgmap v78: 97 pgs: 97 active+clean; 136 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 880 op/s 2024-03-27T06:46:03.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:03 smithi156 ceph-mon[36261]: pgmap v78: 97 pgs: 97 active+clean; 136 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 880 op/s 2024-03-27T06:46:04.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:04 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:04.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:04 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:04.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:04 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.617+0000 7f9258fd7640 1 -- 172.21.15.16:0/1458256269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92541024b0 msgr2=0x7f925410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.617+0000 7f9258fd7640 1 --2- 172.21.15.16:0/1458256269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92541024b0 0x7f925410e9c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9244009930 tx=0x7f924402bc30 comp rx=0 tx=0).stop 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.617+0000 7f9258fd7640 1 -- 172.21.15.16:0/1458256269 shutdown_connections 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.617+0000 7f9258fd7640 1 --2- 172.21.15.16:0/1458256269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92541024b0 0x7f925410e9c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.617+0000 7f9258fd7640 1 --2- 172.21.15.16:0/1458256269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254101af0 0x7f9254101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.617+0000 7f9258fd7640 1 --2- 172.21.15.16:0/1458256269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9254107b90 0x7f9254107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.617+0000 7f9258fd7640 1 -- 172.21.15.16:0/1458256269 >> 172.21.15.16:0/1458256269 conn(0x7f92540fd5e0 msgr2=0x7f92540ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.617+0000 7f9258fd7640 1 -- 172.21.15.16:0/1458256269 shutdown_connections 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.617+0000 7f9258fd7640 1 -- 172.21.15.16:0/1458256269 wait complete. 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.618+0000 7f9258fd7640 1 Processor -- start 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.618+0000 7f9258fd7640 1 -- start start 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.618+0000 7f9258fd7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9254101af0 0x7f9254198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.618+0000 7f9258fd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92541024b0 0x7f9254198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.618+0000 7f9258fd7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254107b90 0x7f925419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.618+0000 7f9258fd7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9254111fb0 con 0x7f92541024b0 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.618+0000 7f9258fd7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f9254111e30 con 0x7f9254107b90 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.618+0000 7f9258fd7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9254112130 con 0x7f9254101af0 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.618+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92541024b0 0x7f9254198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92541024b0 0x7f9254198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41234/0 (socket says 172.21.15.16:41234) 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f9251d9b640 1 -- 172.21.15.16:0/3584179205 learned_addr learned my addr 172.21.15.16:0/3584179205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f925259c640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9254101af0 0x7f9254198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f9252d9d640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254107b90 0x7f925419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f9251d9b640 1 -- 172.21.15.16:0/3584179205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9254101af0 msgr2=0x7f9254198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f9251d9b640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9254101af0 0x7f9254198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f9251d9b640 1 -- 172.21.15.16:0/3584179205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254107b90 msgr2=0x7f925419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f9251d9b640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254107b90 0x7f925419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f9251d9b640 1 -- 172.21.15.16:0/3584179205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925419d420 con 0x7f92541024b0 2024-03-27T06:46:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f9251d9b640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92541024b0 0x7f9254198970 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f923c00e9f0 tx=0x7f923c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f924b7fe640 1 -- 172.21.15.16:0/3584179205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c00cd80 con 0x7f92541024b0 2024-03-27T06:46:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f9258fd7640 1 -- 172.21.15.16:0/3584179205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925419d710 con 0x7f92541024b0 2024-03-27T06:46:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f924b7fe640 1 -- 172.21.15.16:0/3584179205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f923c010930 con 0x7f92541024b0 2024-03-27T06:46:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f924b7fe640 1 -- 172.21.15.16:0/3584179205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c018b00 con 0x7f92541024b0 2024-03-27T06:46:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.619+0000 7f9258fd7640 1 -- 172.21.15.16:0/3584179205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92541a4fa0 con 0x7f92541024b0 2024-03-27T06:46:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.621+0000 7f924b7fe640 1 -- 172.21.15.16:0/3584179205 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f923c018ca0 con 0x7f92541024b0 2024-03-27T06:46:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f925bbbe740 1 Processor -- start 2024-03-27T06:46:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f925bbbe740 1 -- start start 2024-03-27T06:46:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9374960f0 0x55e9374964d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e937496a10 0x55e93749f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93749f7e0 0x55e9374a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f925bbbe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e93745ccd0 con 0x55e93749f7e0 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f925bbbe740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e9373a1950 con 0x55e937496a10 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f925bbbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e9373f9e50 con 0x55e9374960f0 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f924b7fe640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92400a3820 0x7f92400a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93749f7e0 0x55e9374a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f925259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e937496a10 0x55e93749f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93749f7e0 0x55e9374a1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41246/0 (socket says 172.21.15.16:41246) 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f925259c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e937496a10 0x55e93749f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:58790/0 (socket says 172.21.15.16:58790) 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f9251d9b640 1 -- 172.21.15.16:0/2464427950 learned_addr learned my addr 172.21.15.16:0/2464427950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f9252d9d640 1 --2- 172.21.15.16:0/2464427950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9374960f0 0x55e9374964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f9252d9d640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92400a3820 0x7f92400a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f924b7fe640 1 -- 172.21.15.16:0/3584179205 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f923c0d0460 con 0x7f92541024b0 2024-03-27T06:46:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.622+0000 7f925259c640 1 -- 172.21.15.16:0/2464427950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9374960f0 msgr2=0x55e9374964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.623+0000 7f925259c640 1 --2- 172.21.15.16:0/2464427950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9374960f0 0x55e9374964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.623+0000 7f925259c640 1 -- 172.21.15.16:0/2464427950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93749f7e0 msgr2=0x55e9374a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.623+0000 7f925259c640 1 --2- 172.21.15.16:0/2464427950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93749f7e0 0x55e9374a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.623+0000 7f925259c640 1 -- 172.21.15.16:0/2464427950 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93710e210 con 0x55e937496a10 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.623+0000 7f9252d9d640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92400a3820 0x7f92400a5ce0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9244005940 tx=0x7f92440058d0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.623+0000 7f925259c640 1 --2- 172.21.15.16:0/2464427950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e937496a10 0x55e93749f2a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f924c009fa0 tx=0x7f924c015ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7585205118403e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.623+0000 7f9248ff9640 1 -- 172.21.15.16:0/2464427950 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c003c30 con 0x55e937496a10 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.623+0000 7f9248ff9640 1 -- 172.21.15.16:0/2464427950 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f924c00fe70 con 0x55e937496a10 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.623+0000 7f9248ff9640 1 -- 172.21.15.16:0/2464427950 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c01d3e0 con 0x55e937496a10 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 -- 172.21.15.16:0/2464427950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e937496a10 msgr2=0x55e93749f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 --2- 172.21.15.16:0/2464427950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e937496a10 0x55e93749f2a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f924c009fa0 tx=0x7f924c015ce0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 -- 172.21.15.16:0/2464427950 shutdown_connections 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 --2- 172.21.15.16:0/2464427950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93749f7e0 0x55e9374a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 --2- 172.21.15.16:0/2464427950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e937496a10 0x55e93749f2a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 --2- 172.21.15.16:0/2464427950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9374960f0 0x55e9374964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 -- 172.21.15.16:0/2464427950 >> 172.21.15.16:0/2464427950 conn(0x55e937490a10 msgr2=0x55e937490df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 -- 172.21.15.16:0/2464427950 shutdown_connections 2024-03-27T06:46:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 -- 172.21.15.16:0/2464427950 wait complete. 2024-03-27T06:46:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 -- start start 2024-03-27T06:46:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 Processor -- start 2024-03-27T06:46:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.624+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9374960f0 0x55e93749c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e93749cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e93749f7e0 0x55e93749d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f925bbbe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9373f5e00 con 0x55e937496a10 2024-03-27T06:46:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f925bbbe740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e9373a2520 con 0x55e93749f7e0 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f925bbbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e9373b58e0 con 0x55e9374960f0 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f925bbbe740 10 client.0 _ll_register_callbacks cb 0x55e9373f0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e93749cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f925259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9374960f0 0x55e93749c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e93749cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41254/0 (socket says 172.21.15.16:41254) 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f925259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9374960f0 0x55e93749c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.16:42190/0 (socket says 172.21.15.16:42190) 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9251d9b640 1 -- 172.21.15.16:0/4073686255 learned_addr learned my addr 172.21.15.16:0/4073686255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9252d9d640 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e93749f7e0 0x55e93749d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9251d9b640 1 -- 172.21.15.16:0/4073686255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9374960f0 msgr2=0x55e93749c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9251d9b640 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9374960f0 0x55e93749c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9251d9b640 1 -- 172.21.15.16:0/4073686255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e93749f7e0 msgr2=0x55e93749d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9251d9b640 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e93749f7e0 0x55e93749d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9251d9b640 1 -- 172.21.15.16:0/4073686255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9371da620 con 0x55e937496a10 2024-03-27T06:46:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9251d9b640 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e93749cce0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f923c009230 tx=0x7f923c09c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc01d767c9f9dc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9239ffb640 1 -- 172.21.15.16:0/4073686255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c09c6d0 con 0x55e937496a10 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9239ffb640 1 -- 172.21.15.16:0/4073686255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f923c010ae0 con 0x55e937496a10 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f925bbbe740 1 -- 172.21.15.16:0/4073686255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f924c00fe70 con 0x55e937496a10 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f9239ffb640 1 -- 172.21.15.16:0/4073686255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f923c0ba050 con 0x55e937496a10 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.625+0000 7f925bbbe740 10 client.16350 fetch_fsmap learned FSMap version 35 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.626+0000 7f925bbbe740 1 -- 172.21.15.16:0/4073686255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e9374ae380 con 0x55e937496a10 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.626+0000 7f9239ffb640 1 -- 172.21.15.16:0/4073686255 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f924c00fe70 con 0x55e937496a10 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.626+0000 7f9239ffb640 1 -- 172.21.15.16:0/4073686255 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f923c0b5cd0 con 0x55e937496a10 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.626+0000 7f925bbbe740 10 client.16350 fetch_fsmap finished waiting for FSMap version 35 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.626+0000 7f925bbbe740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.626+0000 7f925bbbe740 20 client.16350 populate_metadata read hostname 'smithi016' 2024-03-27T06:46:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.626+0000 7f925bbbe740 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e9374af840 0x55e9374b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.626+0000 7f925bbbe740 4 client.16350 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.626+0000 7f925bbbe740 1 -- 172.21.15.16:0/4073686255 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e937407390 con 0x55e9374af840 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.626+0000 7f925259c640 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e9374af840 0x55e9374b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.627+0000 7f925259c640 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e9374af840 0x55e9374b1c60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77c3beb2c92c7bd6 server_cookie=ef6cb52a5624e583 in_seq=0 out_seq=0 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.627+0000 7f9239ffb640 10 client.16350 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.628+0000 7f9239ffb640 1 -- 172.21.15.16:0/4073686255 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e937407390 con 0x55e9374af840 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.628+0000 7f9239ffb640 10 client.16350 handle_command_reply: tid=0 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.628+0000 7f925bbbe740 1 client.16350 shutdown 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.628+0000 7f925bbbe740 1 -- 172.21.15.16:0/4073686255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 msgr2=0x55e93749cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.628+0000 7f925bbbe740 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e93749cce0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f923c009230 tx=0x7f923c09c320 comp rx=0 tx=0).stop 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.629+0000 7f925bbbe740 1 -- 172.21.15.16:0/4073686255 shutdown_connections 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.629+0000 7f925bbbe740 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e9374af840 0x55e9374b1c60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.629+0000 7f925bbbe740 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e93749f7e0 0x55e93749d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.629+0000 7f9239ffb640 0 client.16350 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.629+0000 7f925bbbe740 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e93749cce0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.629+0000 7f925bbbe740 1 --2- 172.21.15.16:0/4073686255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9374960f0 0x55e93749c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.629+0000 7f925bbbe740 1 -- 172.21.15.16:0/4073686255 >> 172.21.15.16:0/4073686255 conn(0x55e937490a10 msgr2=0x55e937492760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.629+0000 7f925bbbe740 1 -- 172.21.15.16:0/4073686255 shutdown_connections 2024-03-27T06:46:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.629+0000 7f925bbbe740 1 -- 172.21.15.16:0/4073686255 wait complete. 2024-03-27T06:46:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.629+0000 7f925bbbe740 20 client.16350 trim_cache size 0 max 16384 2024-03-27T06:46:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f925bbbe740 1 Processor -- start 2024-03-27T06:46:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f925bbbe740 1 -- start start 2024-03-27T06:46:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9374960f0 0x55e93749c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e9374aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e93749f7e0 0x55e9374b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f925bbbe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92541ab5c0 con 0x55e937496a10 2024-03-27T06:46:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f925bbbe740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f92541ad220 con 0x55e9374960f0 2024-03-27T06:46:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f925bbbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f92541abcc0 con 0x55e93749f7e0 2024-03-27T06:46:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e9374aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e9374aaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41270/0 (socket says 172.21.15.16:41270) 2024-03-27T06:46:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f9251d9b640 1 -- 172.21.15.16:0/351090858 learned_addr learned my addr 172.21.15.16:0/351090858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f925259c640 1 --2- 172.21.15.16:0/351090858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9374960f0 0x55e93749c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.645+0000 7f9252d9d640 1 --2- 172.21.15.16:0/351090858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e93749f7e0 0x55e9374b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.646+0000 7f9251d9b640 1 -- 172.21.15.16:0/351090858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e93749f7e0 msgr2=0x55e9374b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.646+0000 7f9251d9b640 1 --2- 172.21.15.16:0/351090858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e93749f7e0 0x55e9374b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.646+0000 7f9251d9b640 1 -- 172.21.15.16:0/351090858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9374960f0 msgr2=0x55e93749c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.646+0000 7f9251d9b640 1 --2- 172.21.15.16:0/351090858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9374960f0 0x55e93749c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.646+0000 7f9251d9b640 1 -- 172.21.15.16:0/351090858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e937497570 con 0x55e937496a10 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.646+0000 7f9251d9b640 1 --2- 172.21.15.16:0/351090858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e9374aaeb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f923c09c660 tx=0x7f923c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b15355d0e725e17 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.646+0000 7f923bfff640 1 -- 172.21.15.16:0/351090858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c098d40 con 0x55e937496a10 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.646+0000 7f923bfff640 1 -- 172.21.15.16:0/351090858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f923c010ae0 con 0x55e937496a10 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.646+0000 7f923bfff640 1 -- 172.21.15.16:0/351090858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c098ee0 con 0x55e937496a10 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 -- 172.21.15.16:0/351090858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 msgr2=0x55e9374aaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 --2- 172.21.15.16:0/351090858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e9374aaeb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f923c09c660 tx=0x7f923c009310 comp rx=0 tx=0).stop 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 -- 172.21.15.16:0/351090858 shutdown_connections 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 --2- 172.21.15.16:0/351090858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e93749f7e0 0x55e9374b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 --2- 172.21.15.16:0/351090858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e9374aaeb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 --2- 172.21.15.16:0/351090858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9374960f0 0x55e93749c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 -- 172.21.15.16:0/351090858 >> 172.21.15.16:0/351090858 conn(0x55e937490a10 msgr2=0x55e93749e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 -- 172.21.15.16:0/351090858 shutdown_connections 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 -- 172.21.15.16:0/351090858 wait complete. 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 -- start start 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 Processor -- start 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9374960f0 0x55e937499680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e937499bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e93749f7e0 0x55e937514b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92541acb50 con 0x55e937496a10 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f92541ac3c0 con 0x55e9374960f0 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f92541a95a0 con 0x55e93749f7e0 2024-03-27T06:46:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.647+0000 7f925bbbe740 10 client.0 _ll_register_callbacks cb 0x7f923c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e937499bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e937499bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41286/0 (socket says 172.21.15.16:41286) 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9251d9b640 1 -- 172.21.15.16:0/264602931 learned_addr learned my addr 172.21.15.16:0/264602931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f925259c640 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9374960f0 0x55e937499680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9252d9d640 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e93749f7e0 0x55e937514b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9251d9b640 1 -- 172.21.15.16:0/264602931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e93749f7e0 msgr2=0x55e937514b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9251d9b640 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e93749f7e0 0x55e937514b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9251d9b640 1 -- 172.21.15.16:0/264602931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9374960f0 msgr2=0x55e937499680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9251d9b640 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9374960f0 0x55e937499680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9251d9b640 1 -- 172.21.15.16:0/264602931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9375150c0 con 0x55e937496a10 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9251d9b640 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e937499bc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f923c009830 tx=0x7f923c098aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a8f8e3643776e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9239ffb640 1 -- 172.21.15.16:0/264602931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c09c6d0 con 0x55e937496a10 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9239ffb640 1 -- 172.21.15.16:0/264602931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f923c0b5bb0 con 0x55e937496a10 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f925bbbe740 1 -- 172.21.15.16:0/264602931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f923c010ae0 con 0x55e937496a10 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9239ffb640 1 -- 172.21.15.16:0/264602931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f923c0b5e50 con 0x55e937496a10 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f925bbbe740 10 client.16356 fetch_fsmap learned FSMap version 35 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f925bbbe740 1 -- 172.21.15.16:0/264602931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e9375161b0 con 0x55e937496a10 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.648+0000 7f9239ffb640 1 -- 172.21.15.16:0/264602931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f923c010ae0 con 0x55e937496a10 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.649+0000 7f9239ffb640 1 -- 172.21.15.16:0/264602931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f923c0a1b60 con 0x55e937496a10 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.649+0000 7f925bbbe740 10 client.16356 fetch_fsmap finished waiting for FSMap version 35 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.649+0000 7f925bbbe740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.649+0000 7f925bbbe740 20 client.16356 populate_metadata read hostname 'smithi016' 2024-03-27T06:46:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.649+0000 7f925bbbe740 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e937516710 0x55e937518b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.649+0000 7f925bbbe740 4 client.16356 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:46:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.649+0000 7f925bbbe740 1 -- 172.21.15.16:0/264602931 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e9374016e0 con 0x55e937516710 2024-03-27T06:46:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.649+0000 7f925259c640 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e937516710 0x55e937518b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.649+0000 7f925259c640 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e937516710 0x55e937518b30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=647c132e09afc49d server_cookie=b71aee4ab5bc0bb1 in_seq=0 out_seq=0 2024-03-27T06:46:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.649+0000 7f9239ffb640 10 client.16356 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f9239ffb640 1 -- 172.21.15.16:0/264602931 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e9374016e0 con 0x55e937516710 2024-03-27T06:46:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f9239ffb640 10 client.16356 handle_command_reply: tid=0 2024-03-27T06:46:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f925bbbe740 1 client.16356 shutdown 2024-03-27T06:46:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f925bbbe740 1 -- 172.21.15.16:0/264602931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 msgr2=0x55e937499bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f925bbbe740 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e937499bc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f923c009830 tx=0x7f923c098aa0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f925bbbe740 1 -- 172.21.15.16:0/264602931 shutdown_connections 2024-03-27T06:46:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f925bbbe740 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e937516710 0x55e937518b30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f925bbbe740 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e93749f7e0 0x55e937514b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f9239ffb640 0 client.16356 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f925bbbe740 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e937496a10 0x55e937499bc0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f925bbbe740 1 --2- 172.21.15.16:0/264602931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e9374960f0 0x55e937499680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.938+0000 7f925bbbe740 1 -- 172.21.15.16:0/264602931 >> 172.21.15.16:0/264602931 conn(0x55e937490a10 msgr2=0x55e93748f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f925bbbe740 1 -- 172.21.15.16:0/264602931 shutdown_connections 2024-03-27T06:46:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f925bbbe740 1 -- 172.21.15.16:0/264602931 wait complete. 2024-03-27T06:46:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f925bbbe740 20 client.16356 trim_cache size 0 max 16384 2024-03-27T06:46:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:46:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:46:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:46:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:46:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 -- 172.21.15.16:0/3584179205 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92400a3820 msgr2=0x7f92400a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92400a3820 0x7f92400a5ce0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9244005940 tx=0x7f92440058d0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 -- 172.21.15.16:0/3584179205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92541024b0 msgr2=0x7f9254198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92541024b0 0x7f9254198970 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f923c00e9f0 tx=0x7f923c00eeb0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 -- 172.21.15.16:0/3584179205 shutdown_connections 2024-03-27T06:46:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9254107b90 0x7f925419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92541024b0 0x7f9254198970 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f92400a3820 0x7f92400a5ce0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 --2- 172.21.15.16:0/3584179205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9254101af0 0x7f9254198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 -- 172.21.15.16:0/3584179205 >> 172.21.15.16:0/3584179205 conn(0x7f92540fd5e0 msgr2=0x7f92540fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 -- 172.21.15.16:0/3584179205 shutdown_connections 2024-03-27T06:46:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:04.939+0000 7f9258fd7640 1 -- 172.21.15.16:0/3584179205 wait complete. 2024-03-27T06:46:05.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:05 smithi170 ceph-mon[36438]: pgmap v79: 97 pgs: 97 active+clean; 136 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 684 op/s 2024-03-27T06:46:05.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:05 smithi016 ceph-mon[31908]: pgmap v79: 97 pgs: 97 active+clean; 136 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 684 op/s 2024-03-27T06:46:05.373 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:05 smithi156 ceph-mon[36261]: pgmap v79: 97 pgs: 97 active+clean; 136 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 684 op/s 2024-03-27T06:46:07.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:07 smithi170 ceph-mon[36438]: pgmap v80: 97 pgs: 97 active+clean; 142 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 907 op/s 2024-03-27T06:46:07.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:07 smithi016 ceph-mon[31908]: pgmap v80: 97 pgs: 97 active+clean; 142 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 907 op/s 2024-03-27T06:46:07.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:07 smithi156 ceph-mon[36261]: pgmap v80: 97 pgs: 97 active+clean; 142 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 907 op/s 2024-03-27T06:46:09.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:09 smithi170 ceph-mon[36438]: pgmap v81: 97 pgs: 97 active+clean; 142 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 354 op/s 2024-03-27T06:46:09.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:09 smithi016 ceph-mon[31908]: pgmap v81: 97 pgs: 97 active+clean; 142 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 354 op/s 2024-03-27T06:46:09.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:09 smithi156 ceph-mon[36261]: pgmap v81: 97 pgs: 97 active+clean; 142 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 354 op/s 2024-03-27T06:46:11.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:11 smithi170 ceph-mon[36438]: pgmap v82: 97 pgs: 97 active+clean; 136 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 402 op/s 2024-03-27T06:46:11.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:11 smithi016 ceph-mon[31908]: pgmap v82: 97 pgs: 97 active+clean; 136 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 402 op/s 2024-03-27T06:46:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:11 smithi156 ceph-mon[36261]: pgmap v82: 97 pgs: 97 active+clean; 136 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 402 op/s 2024-03-27T06:46:12.540 INFO:tasks.workunit.client.0.smithi016.stderr:+ tar xf doc.tar 2024-03-27T06:46:13.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:13 smithi170 ceph-mon[36438]: pgmap v83: 97 pgs: 97 active+clean; 132 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 426 op/s 2024-03-27T06:46:13.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:13 smithi016 ceph-mon[31908]: pgmap v83: 97 pgs: 97 active+clean; 132 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 426 op/s 2024-03-27T06:46:13.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:13 smithi156 ceph-mon[36261]: pgmap v83: 97 pgs: 97 active+clean; 132 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 426 op/s 2024-03-27T06:46:14.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:14 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:14.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:14 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:14.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:14 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:15 smithi016 ceph-mon[31908]: pgmap v84: 97 pgs: 97 active+clean; 132 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.2 MiB/s wr, 346 op/s 2024-03-27T06:46:15.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:15 smithi156 ceph-mon[36261]: pgmap v84: 97 pgs: 97 active+clean; 132 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.2 MiB/s wr, 346 op/s 2024-03-27T06:46:15.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:15 smithi170 ceph-mon[36438]: pgmap v84: 97 pgs: 97 active+clean; 132 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.2 MiB/s wr, 346 op/s 2024-03-27T06:46:17.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:17 smithi016 ceph-mon[31908]: pgmap v85: 97 pgs: 97 active+clean; 139 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 434 op/s 2024-03-27T06:46:17.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:17 smithi156 ceph-mon[36261]: pgmap v85: 97 pgs: 97 active+clean; 139 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 434 op/s 2024-03-27T06:46:17.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:17 smithi170 ceph-mon[36438]: pgmap v85: 97 pgs: 97 active+clean; 139 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 434 op/s 2024-03-27T06:46:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.037+0000 7f82252af640 1 -- 172.21.15.16:0/1932043387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82201036d0 msgr2=0x7f822010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.037+0000 7f82252af640 1 --2- 172.21.15.16:0/1932043387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82201036d0 0x7f822010bbb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8214009900 tx=0x7f821402bc30 comp rx=0 tx=0).stop 2024-03-27T06:46:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.037+0000 7f82252af640 1 -- 172.21.15.16:0/1932043387 shutdown_connections 2024-03-27T06:46:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.037+0000 7f82252af640 1 --2- 172.21.15.16:0/1932043387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f822010c180 0x7f822010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.037+0000 7f82252af640 1 --2- 172.21.15.16:0/1932043387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82201036d0 0x7f822010bbb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.037+0000 7f82252af640 1 --2- 172.21.15.16:0/1932043387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8220102db0 0x7f8220103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.037+0000 7f82252af640 1 -- 172.21.15.16:0/1932043387 >> 172.21.15.16:0/1932043387 conn(0x7f8220077b80 msgr2=0x7f8220077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.037+0000 7f82252af640 1 -- 172.21.15.16:0/1932043387 shutdown_connections 2024-03-27T06:46:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.037+0000 7f82252af640 1 -- 172.21.15.16:0/1932043387 wait complete. 2024-03-27T06:46:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.037+0000 7f82252af640 1 Processor -- start 2024-03-27T06:46:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f82252af640 1 -- start start 2024-03-27T06:46:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f82252af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220102db0 0x7f82201983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f82252af640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201036d0 0x7f8220198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f82252af640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f822010c180 0x7f822019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f82252af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8220112030 con 0x7f8220102db0 2024-03-27T06:46:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f82252af640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f8220111eb0 con 0x7f822010c180 2024-03-27T06:46:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f82252af640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f82201121b0 con 0x7f82201036d0 2024-03-27T06:46:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f821effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220102db0 0x7f82201983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f821effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220102db0 0x7f82201983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53590/0 (socket says 172.21.15.16:53590) 2024-03-27T06:46:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f821effd640 1 -- 172.21.15.16:0/1277405821 learned_addr learned my addr 172.21.15.16:0/1277405821 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f821f7fe640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f822010c180 0x7f822019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f821e7fc640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201036d0 0x7f8220198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f821effd640 1 -- 172.21.15.16:0/1277405821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201036d0 msgr2=0x7f8220198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f821effd640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201036d0 0x7f8220198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f821effd640 1 -- 172.21.15.16:0/1277405821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f822010c180 msgr2=0x7f822019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f821effd640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f822010c180 0x7f822019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f821effd640 1 -- 172.21.15.16:0/1277405821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822019d3e0 con 0x7f8220102db0 2024-03-27T06:46:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.038+0000 7f821e7fc640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201036d0 0x7f8220198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:46:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.039+0000 7f821effd640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220102db0 0x7f82201983f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f821000e9f0 tx=0x7f821000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.039+0000 7f8207fff640 1 -- 172.21.15.16:0/1277405821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821000cd80 con 0x7f8220102db0 2024-03-27T06:46:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.039+0000 7f8207fff640 1 -- 172.21.15.16:0/1277405821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8210010930 con 0x7f8220102db0 2024-03-27T06:46:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.039+0000 7f82252af640 1 -- 172.21.15.16:0/1277405821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822019d6d0 con 0x7f8220102db0 2024-03-27T06:46:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.039+0000 7f8207fff640 1 -- 172.21.15.16:0/1277405821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210018b00 con 0x7f8220102db0 2024-03-27T06:46:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.039+0000 7f82252af640 1 -- 172.21.15.16:0/1277405821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82201a4f60 con 0x7f8220102db0 2024-03-27T06:46:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f8207fff640 1 -- 172.21.15.16:0/1277405821 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f8210018ca0 con 0x7f8220102db0 2024-03-27T06:46:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f8227f72740 1 Processor -- start 2024-03-27T06:46:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f8227f72740 1 -- start start 2024-03-27T06:46:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f8227f72740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd8110f0 0x5577dd8114d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f8227f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd81a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f8227f72740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd81a7e0 0x5577dd81cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f8227f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5577dd7d7cd0 con 0x5577dd811a10 2024-03-27T06:46:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f8227f72740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5577dd71c950 con 0x5577dd81a7e0 2024-03-27T06:46:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f8227f72740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5577dd774e50 con 0x5577dd8110f0 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f821effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd81a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f8207fff640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f82080a3820 0x7f82080a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f821effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd81a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53600/0 (socket says 172.21.15.16:53600) 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f821effd640 1 -- 172.21.15.16:0/2802831716 learned_addr learned my addr 172.21.15.16:0/2802831716 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.042+0000 7f821e7fc640 1 --2- 172.21.15.16:0/2802831716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd81a7e0 0x5577dd81cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f821f7fe640 1 --2- 172.21.15.16:0/2802831716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd8110f0 0x5577dd8114d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f821f7fe640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f82080a3820 0x7f82080a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f8207fff640 1 -- 172.21.15.16:0/1277405821 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f82100d0460 con 0x7f8220102db0 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f821e7fc640 1 -- 172.21.15.16:0/2802831716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd8110f0 msgr2=0x5577dd8114d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f821e7fc640 1 --2- 172.21.15.16:0/2802831716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd8110f0 0x5577dd8114d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f821e7fc640 1 -- 172.21.15.16:0/2802831716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 msgr2=0x5577dd81a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f821e7fc640 1 --2- 172.21.15.16:0/2802831716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd81a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f821e7fc640 1 -- 172.21.15.16:0/2802831716 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577dd489210 con 0x5577dd81a7e0 2024-03-27T06:46:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f821e7fc640 1 --2- 172.21.15.16:0/2802831716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd81a7e0 0x5577dd81cbd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8214000c00 tx=0x7f82140095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3476fa9e0cd0a3a server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f821f7fe640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f82080a3820 0x7f82080a5ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8218005c40 tx=0x7f8218009500 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f82057fa640 1 -- 172.21.15.16:0/2802831716 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214043070 con 0x5577dd81a7e0 2024-03-27T06:46:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f82057fa640 1 -- 172.21.15.16:0/2802831716 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f821402c6f0 con 0x5577dd81a7e0 2024-03-27T06:46:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.043+0000 7f82057fa640 1 -- 172.21.15.16:0/2802831716 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214003970 con 0x5577dd81a7e0 2024-03-27T06:46:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.044+0000 7f8227f72740 1 -- 172.21.15.16:0/2802831716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd81a7e0 msgr2=0x5577dd81cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.044+0000 7f8227f72740 1 --2- 172.21.15.16:0/2802831716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd81a7e0 0x5577dd81cbd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8214000c00 tx=0x7f82140095f0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.044+0000 7f8227f72740 1 -- 172.21.15.16:0/2802831716 shutdown_connections 2024-03-27T06:46:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.044+0000 7f8227f72740 1 --2- 172.21.15.16:0/2802831716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd81a7e0 0x5577dd81cbd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.044+0000 7f8227f72740 1 --2- 172.21.15.16:0/2802831716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd81a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.044+0000 7f8227f72740 1 --2- 172.21.15.16:0/2802831716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd8110f0 0x5577dd8114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.044+0000 7f8227f72740 1 -- 172.21.15.16:0/2802831716 >> 172.21.15.16:0/2802831716 conn(0x5577dd80ba10 msgr2=0x5577dd80bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.044+0000 7f8227f72740 1 -- 172.21.15.16:0/2802831716 shutdown_connections 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.044+0000 7f8227f72740 1 -- 172.21.15.16:0/2802831716 wait complete. 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.044+0000 7f8227f72740 1 -- start start 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f8227f72740 1 Processor -- start 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f8227f72740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd8177d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f8227f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd817d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f8227f72740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd81a7e0 0x5577dd818250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f8227f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5577dd770e00 con 0x5577dd811a10 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f8227f72740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x5577dd71d520 con 0x5577dd8110f0 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f8227f72740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5577dd7308e0 con 0x5577dd81a7e0 2024-03-27T06:46:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f8227f72740 10 client.0 _ll_register_callbacks cb 0x5577dd76ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:46:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f821effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd8177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f821effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd8177d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:37034/0 (socket says 172.21.15.16:37034) 2024-03-27T06:46:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f821effd640 1 -- 172.21.15.16:0/3349692033 learned_addr learned my addr 172.21.15.16:0/3349692033 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f821f7fe640 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd81a7e0 0x5577dd818250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f821effd640 1 -- 172.21.15.16:0/3349692033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd81a7e0 msgr2=0x5577dd818250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f821effd640 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd81a7e0 0x5577dd818250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f821effd640 1 -- 172.21.15.16:0/3349692033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 msgr2=0x5577dd817d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f821effd640 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd817d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f821effd640 1 -- 172.21.15.16:0/3349692033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577dd555620 con 0x5577dd8110f0 2024-03-27T06:46:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.045+0000 7f821f7fe640 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd81a7e0 0x5577dd818250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f821effd640 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd8177d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f821009c660 tx=0x7f821009c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=823e549e8b53e275 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8202ff5640 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8202ff5640 1 -- 172.21.15.16:0/3349692033 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210026040 con 0x5577dd8110f0 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8227f72740 1 -- 172.21.15.16:0/3349692033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f821402c6f0 con 0x5577dd8110f0 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8202ff5640 1 -- 172.21.15.16:0/3349692033 <== mon.2 v2:172.21.15.156:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f821009bb20 con 0x5577dd8110f0 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8202ff5640 1 -- 172.21.15.16:0/3349692033 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8210010ce0 con 0x5577dd8110f0 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8227f72740 10 client.26192 fetch_fsmap learned FSMap version 36 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8227f72740 1 -- 172.21.15.16:0/3349692033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5577dd829350 con 0x5577dd8110f0 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8202ff5640 1 -- 172.21.15.16:0/3349692033 <== mon.2 v2:172.21.15.156:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f821402c6f0 con 0x5577dd8110f0 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8202ff5640 1 -- 172.21.15.16:0/3349692033 <== mon.2 v2:172.21.15.156:3300/0 5 ==== fsmap(e 36) ==== 8390+0+0 (secure 0 0 0) 0x7f8210098420 con 0x5577dd8110f0 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8227f72740 10 client.26192 fetch_fsmap finished waiting for FSMap version 36 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8227f72740 10 client.26192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:46:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8227f72740 20 client.26192 populate_metadata read hostname 'smithi016' 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8227f72740 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5577dd82a8d0 0x5577dd82cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8227f72740 4 client.26192 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.046+0000 7f8227f72740 1 -- 172.21.15.16:0/3349692033 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577dd782390 con 0x5577dd82a8d0 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.047+0000 7f821e7fc640 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5577dd82a8d0 0x5577dd82cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.047+0000 7f821e7fc640 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5577dd82a8d0 0x5577dd82cd90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=136f687aa0ab387f server_cookie=36b3ea88e106a203 in_seq=0 out_seq=0 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.047+0000 7f8202ff5640 10 client.26192 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8202ff5640 1 -- 172.21.15.16:0/3349692033 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5577dd782390 con 0x5577dd82a8d0 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8202ff5640 10 client.26192 handle_command_reply: tid=0 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 1 client.26192 shutdown 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 1 -- 172.21.15.16:0/3349692033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 msgr2=0x5577dd8177d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd8177d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f821009c660 tx=0x7f821009c6d0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 1 -- 172.21.15.16:0/3349692033 shutdown_connections 2024-03-27T06:46:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5577dd82a8d0 0x5577dd82cd90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd81a7e0 0x5577dd818250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8202ff5640 0 client.26192 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd817d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 1 --2- 172.21.15.16:0/3349692033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd8177d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 1 -- 172.21.15.16:0/3349692033 >> 172.21.15.16:0/3349692033 conn(0x5577dd80ba10 msgr2=0x5577dd80d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 1 -- 172.21.15.16:0/3349692033 shutdown_connections 2024-03-27T06:46:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 1 -- 172.21.15.16:0/3349692033 wait complete. 2024-03-27T06:46:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.048+0000 7f8227f72740 20 client.26192 trim_cache size 0 max 16384 2024-03-27T06:46:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f8227f72740 1 Processor -- start 2024-03-27T06:46:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f8227f72740 1 -- start start 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f8227f72740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd819f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f8227f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd825e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f8227f72740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd81a7e0 0x5577dd832660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f8227f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82201ad680 con 0x5577dd811a10 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f8227f72740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f82201af310 con 0x5577dd8110f0 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f8227f72740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f82201add80 con 0x5577dd81a7e0 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f821e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd825e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f821e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd825e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53632/0 (socket says 172.21.15.16:53632) 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f821effd640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd819f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f821e7fc640 1 -- 172.21.15.16:0/629315041 learned_addr learned my addr 172.21.15.16:0/629315041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.063+0000 7f821f7fe640 1 --2- 172.21.15.16:0/629315041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd81a7e0 0x5577dd832660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f821e7fc640 1 -- 172.21.15.16:0/629315041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd81a7e0 msgr2=0x5577dd832660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f821e7fc640 1 --2- 172.21.15.16:0/629315041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd81a7e0 0x5577dd832660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f821e7fc640 1 -- 172.21.15.16:0/629315041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 msgr2=0x5577dd819f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f821e7fc640 1 --2- 172.21.15.16:0/629315041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd819f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f821e7fc640 1 -- 172.21.15.16:0/629315041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577dd8133b0 con 0x5577dd811a10 2024-03-27T06:46:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f821effd640 1 --2- 172.21.15.16:0/629315041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd819f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:46:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f821e7fc640 1 --2- 172.21.15.16:0/629315041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd825e50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8214009900 tx=0x7f8214004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886e9e0d8adcefe4 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f8204ff9640 1 -- 172.21.15.16:0/629315041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214043040 con 0x5577dd811a10 2024-03-27T06:46:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f8204ff9640 1 -- 172.21.15.16:0/629315041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f82140349e0 con 0x5577dd811a10 2024-03-27T06:46:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f8204ff9640 1 -- 172.21.15.16:0/629315041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214048940 con 0x5577dd811a10 2024-03-27T06:46:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f8227f72740 1 -- 172.21.15.16:0/629315041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 msgr2=0x5577dd825e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.064+0000 7f8227f72740 1 --2- 172.21.15.16:0/629315041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd825e50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8214009900 tx=0x7f8214004580 comp rx=0 tx=0).stop 2024-03-27T06:46:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 -- 172.21.15.16:0/629315041 shutdown_connections 2024-03-27T06:46:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 --2- 172.21.15.16:0/629315041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd81a7e0 0x5577dd832660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 --2- 172.21.15.16:0/629315041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd811a10 0x5577dd825e50 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 --2- 172.21.15.16:0/629315041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd819f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 -- 172.21.15.16:0/629315041 >> 172.21.15.16:0/629315041 conn(0x5577dd80ba10 msgr2=0x5577dd8065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 -- 172.21.15.16:0/629315041 shutdown_connections 2024-03-27T06:46:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 -- 172.21.15.16:0/629315041 wait complete. 2024-03-27T06:46:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 -- start start 2024-03-27T06:46:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 Processor -- start 2024-03-27T06:46:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd814540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd811a10 0x5577dd814a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd81a7e0 0x5577dd814fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82201aec40 con 0x5577dd81a7e0 2024-03-27T06:46:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f82201ae4b0 con 0x5577dd8110f0 2024-03-27T06:46:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f82201ab660 con 0x5577dd811a10 2024-03-27T06:46:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f8227f72740 10 client.0 _ll_register_callbacks cb 0x7f821009c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:46:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f821f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd81a7e0 0x5577dd814fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f821f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd81a7e0 0x5577dd814fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53646/0 (socket says 172.21.15.16:53646) 2024-03-27T06:46:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f821f7fe640 1 -- 172.21.15.16:0/1696429478 learned_addr learned my addr 172.21.15.16:0/1696429478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f821effd640 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd814540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.065+0000 7f821e7fc640 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd811a10 0x5577dd814a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f821f7fe640 1 -- 172.21.15.16:0/1696429478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd811a10 msgr2=0x5577dd814a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f821f7fe640 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd811a10 0x5577dd814a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f821f7fe640 1 -- 172.21.15.16:0/1696429478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 msgr2=0x5577dd814540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f821f7fe640 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd814540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f821f7fe640 1 -- 172.21.15.16:0/1696429478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577dd88fe60 con 0x5577dd81a7e0 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f821f7fe640 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd81a7e0 0x5577dd814fc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8218003ec0 tx=0x7f8218004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5938ea19ea4268e server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f8202ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f8202ff5640 1 -- 172.21.15.16:0/1696429478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218030060 con 0x5577dd81a7e0 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f8202ff5640 1 -- 172.21.15.16:0/1696429478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7f8218020860 con 0x5577dd81a7e0 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f8227f72740 1 -- 172.21.15.16:0/1696429478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82140349e0 con 0x5577dd81a7e0 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f8202ff5640 1 -- 172.21.15.16:0/1696429478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8218020a60 con 0x5577dd81a7e0 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f8227f72740 10 client.16371 fetch_fsmap learned FSMap version 36 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f8227f72740 1 -- 172.21.15.16:0/1696429478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5577dd8911b0 con 0x5577dd81a7e0 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.066+0000 7f8202ff5640 1 -- 172.21.15.16:0/1696429478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f82140349e0 con 0x5577dd81a7e0 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.067+0000 7f8202ff5640 1 -- 172.21.15.16:0/1696429478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 36) ==== 8390+0+0 (secure 0 0 0) 0x7f821802a070 con 0x5577dd81a7e0 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.067+0000 7f8227f72740 10 client.16371 fetch_fsmap finished waiting for FSMap version 36 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.067+0000 7f8227f72740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:46:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.067+0000 7f8227f72740 20 client.16371 populate_metadata read hostname 'smithi016' 2024-03-27T06:46:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.067+0000 7f8227f72740 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5577dd891a20 0x5577dd893e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.067+0000 7f8227f72740 4 client.16371 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:46:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.067+0000 7f8227f72740 1 -- 172.21.15.16:0/1696429478 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577dd77c6e0 con 0x5577dd891a20 2024-03-27T06:46:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.067+0000 7f821effd640 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5577dd891a20 0x5577dd893e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.067+0000 7f821effd640 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5577dd891a20 0x5577dd893e40 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a21937aa42c8c89a server_cookie=4d9493f774abc238 in_seq=0 out_seq=0 2024-03-27T06:46:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.067+0000 7f8202ff5640 10 client.16371 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8202ff5640 1 -- 172.21.15.16:0/1696429478 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577dd77c6e0 con 0x5577dd891a20 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8202ff5640 10 client.16371 handle_command_reply: tid=0 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 1 client.16371 shutdown 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 1 -- 172.21.15.16:0/1696429478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd81a7e0 msgr2=0x5577dd814fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd81a7e0 0x5577dd814fc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8218003ec0 tx=0x7f8218004a40 comp rx=0 tx=0).stop 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 1 -- 172.21.15.16:0/1696429478 shutdown_connections 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x5577dd891a20 0x5577dd893e40 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8202ff5640 0 client.16371 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577dd81a7e0 0x5577dd814fc0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577dd811a10 0x5577dd814a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 1 --2- 172.21.15.16:0/1696429478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577dd8110f0 0x5577dd814540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 1 -- 172.21.15.16:0/1696429478 >> 172.21.15.16:0/1696429478 conn(0x5577dd80ba10 msgr2=0x5577dd80a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 1 -- 172.21.15.16:0/1696429478 shutdown_connections 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 1 -- 172.21.15.16:0/1696429478 wait complete. 2024-03-27T06:46:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.875+0000 7f8227f72740 20 client.16371 trim_cache size 0 max 16384 2024-03-27T06:46:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:46:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:46:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:46:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:46:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 -- 172.21.15.16:0/1277405821 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f82080a3820 msgr2=0x7f82080a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f82080a3820 0x7f82080a5ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8218005c40 tx=0x7f8218009500 comp rx=0 tx=0).stop 2024-03-27T06:46:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 -- 172.21.15.16:0/1277405821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220102db0 msgr2=0x7f82201983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220102db0 0x7f82201983f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f821000e9f0 tx=0x7f821000eeb0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 -- 172.21.15.16:0/1277405821 shutdown_connections 2024-03-27T06:46:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f822010c180 0x7f822019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f82080a3820 0x7f82080a5ce0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201036d0 0x7f8220198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 --2- 172.21.15.16:0/1277405821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8220102db0 0x7f82201983f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 -- 172.21.15.16:0/1277405821 >> 172.21.15.16:0/1277405821 conn(0x7f8220077b80 msgr2=0x7f82200fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 -- 172.21.15.16:0/1277405821 shutdown_connections 2024-03-27T06:46:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:18.876+0000 7f82252af640 1 -- 172.21.15.16:0/1277405821 wait complete. 2024-03-27T06:46:19.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:19 smithi016 ceph-mon[31908]: pgmap v86: 97 pgs: 97 active+clean; 139 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-03-27T06:46:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:19 smithi156 ceph-mon[36261]: pgmap v86: 97 pgs: 97 active+clean; 139 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-03-27T06:46:19.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:19 smithi170 ceph-mon[36438]: pgmap v86: 97 pgs: 97 active+clean; 139 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-03-27T06:46:21.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:21 smithi016 ceph-mon[31908]: pgmap v87: 97 pgs: 97 active+clean; 133 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 313 op/s 2024-03-27T06:46:21.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:21 smithi156 ceph-mon[36261]: pgmap v87: 97 pgs: 97 active+clean; 133 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 313 op/s 2024-03-27T06:46:21.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:21 smithi170 ceph-mon[36438]: pgmap v87: 97 pgs: 97 active+clean; 133 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 313 op/s 2024-03-27T06:46:23.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:23 smithi016 ceph-mon[31908]: pgmap v88: 97 pgs: 97 active+clean; 126 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 375 op/s 2024-03-27T06:46:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:23 smithi156 ceph-mon[36261]: pgmap v88: 97 pgs: 97 active+clean; 126 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 375 op/s 2024-03-27T06:46:23.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:23 smithi170 ceph-mon[36438]: pgmap v88: 97 pgs: 97 active+clean; 126 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 375 op/s 2024-03-27T06:46:25.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:25 smithi016 ceph-mon[31908]: pgmap v89: 97 pgs: 97 active+clean; 126 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.0 MiB/s wr, 299 op/s 2024-03-27T06:46:25.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:25 smithi156 ceph-mon[36261]: pgmap v89: 97 pgs: 97 active+clean; 126 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.0 MiB/s wr, 299 op/s 2024-03-27T06:46:25.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:25 smithi170 ceph-mon[36438]: pgmap v89: 97 pgs: 97 active+clean; 126 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.0 MiB/s wr, 299 op/s 2024-03-27T06:46:27.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:27 smithi156 ceph-mon[36261]: pgmap v90: 97 pgs: 97 active+clean; 131 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 443 op/s 2024-03-27T06:46:27.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:27 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:27.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:27 smithi170 ceph-mon[36438]: pgmap v90: 97 pgs: 97 active+clean; 131 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 443 op/s 2024-03-27T06:46:27.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:27 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:27 smithi016 ceph-mon[31908]: pgmap v90: 97 pgs: 97 active+clean; 131 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 443 op/s 2024-03-27T06:46:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:27 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:29.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:29 smithi156 ceph-mon[36261]: pgmap v91: 97 pgs: 97 active+clean; 131 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 950 KiB/s wr, 355 op/s 2024-03-27T06:46:29.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:29 smithi170 ceph-mon[36438]: pgmap v91: 97 pgs: 97 active+clean; 131 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 950 KiB/s wr, 355 op/s 2024-03-27T06:46:29.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:29 smithi016 ceph-mon[31908]: pgmap v91: 97 pgs: 97 active+clean; 131 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 950 KiB/s wr, 355 op/s 2024-03-27T06:46:30.042 INFO:tasks.workunit.client.0.smithi016.stderr:+ sync 2024-03-27T06:46:31.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:31 smithi156 ceph-mon[36261]: pgmap v92: 97 pgs: 97 active+clean; 132 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 998 KiB/s wr, 363 op/s 2024-03-27T06:46:31.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:31 smithi170 ceph-mon[36438]: pgmap v92: 97 pgs: 97 active+clean; 132 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 998 KiB/s wr, 363 op/s 2024-03-27T06:46:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:31 smithi016 ceph-mon[31908]: pgmap v92: 97 pgs: 97 active+clean; 132 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 998 KiB/s wr, 363 op/s 2024-03-27T06:46:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.972+0000 7fbae1bac640 1 -- 172.21.15.16:0/3646448684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc1059f0 msgr2=0x7fbadc105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.972+0000 7fbae1bac640 1 --2- 172.21.15.16:0/3646448684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc1059f0 0x7fbadc105dd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009900 tx=0x7fbacc02bc30 comp rx=0 tx=0).stop 2024-03-27T06:46:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.972+0000 7fbae1bac640 1 -- 172.21.15.16:0/3646448684 shutdown_connections 2024-03-27T06:46:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.972+0000 7fbae1bac640 1 --2- 172.21.15.16:0/3646448684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbadc101660 0x7fbadc10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.972+0000 7fbae1bac640 1 --2- 172.21.15.16:0/3646448684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc100ca0 0x7fbadc101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.972+0000 7fbae1bac640 1 --2- 172.21.15.16:0/3646448684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc1059f0 0x7fbadc105dd0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.972+0000 7fbae1bac640 1 -- 172.21.15.16:0/3646448684 >> 172.21.15.16:0/3646448684 conn(0x7fbadc0778a0 msgr2=0x7fbadc077cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.972+0000 7fbae1bac640 1 -- 172.21.15.16:0/3646448684 shutdown_connections 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.972+0000 7fbae1bac640 1 -- 172.21.15.16:0/3646448684 wait complete. 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.973+0000 7fbae1bac640 1 Processor -- start 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.973+0000 7fbae1bac640 1 -- start start 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.973+0000 7fbae1bac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc100ca0 0x7fbadc1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.973+0000 7fbae1bac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc101660 0x7fbadc1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.973+0000 7fbae1bac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbadc1059f0 0x7fbadc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.973+0000 7fbae1bac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc110ff0 con 0x7fbadc100ca0 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.973+0000 7fbae1bac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc110e70 con 0x7fbadc101660 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.973+0000 7fbae1bac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc111170 con 0x7fbadc1059f0 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.973+0000 7fbadb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc100ca0 0x7fbadc1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbadb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc100ca0 0x7fbadc1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39422/0 (socket says 172.21.15.16:39422) 2024-03-27T06:46:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbadb7fe640 1 -- 172.21.15.16:0/1019322304 learned_addr learned my addr 172.21.15.16:0/1019322304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbadaffd640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc101660 0x7fbadc1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbadbfff640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbadc1059f0 0x7fbadc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbadb7fe640 1 -- 172.21.15.16:0/1019322304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbadc1059f0 msgr2=0x7fbadc19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbadc1059f0 0x7fbadc19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbadb7fe640 1 -- 172.21.15.16:0/1019322304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc101660 msgr2=0x7fbadc1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc101660 0x7fbadc1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbadb7fe640 1 -- 172.21.15.16:0/1019322304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc19d490 con 0x7fbadc100ca0 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbadaffd640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc101660 0x7fbadc1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc100ca0 0x7fbadc1984a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbacc005940 tx=0x7fbacc02dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbad8ff9640 1 -- 172.21.15.16:0/1019322304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc043020 con 0x7fbadc100ca0 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbad8ff9640 1 -- 172.21.15.16:0/1019322304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbacc004250 con 0x7fbadc100ca0 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbae1bac640 1 -- 172.21.15.16:0/1019322304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbadc19d720 con 0x7fbadc100ca0 2024-03-27T06:46:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbad8ff9640 1 -- 172.21.15.16:0/1019322304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc003a10 con 0x7fbadc100ca0 2024-03-27T06:46:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.974+0000 7fbae1bac640 1 -- 172.21.15.16:0/1019322304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbadc1a5010 con 0x7fbadc100ca0 2024-03-27T06:46:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbad8ff9640 1 -- 172.21.15.16:0/1019322304 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fbacc046020 con 0x7fbadc100ca0 2024-03-27T06:46:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbae4792740 1 Processor -- start 2024-03-27T06:46:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbae4792740 1 -- start start 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbae4792740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbae4792740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbae4792740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbae4792740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d1e287acd0 con 0x55d1e28b4a10 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbae4792740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55d1e27bf950 con 0x55d1e28bd7e0 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbae4792740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d1e2817e50 con 0x55d1e28b40f0 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbadb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbadb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39426/0 (socket says 172.21.15.16:39426) 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbadb7fe640 1 -- 172.21.15.16:0/3464378622 learned_addr learned my addr 172.21.15.16:0/3464378622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbadbfff640 1 --2- 172.21.15.16:0/3464378622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.977+0000 7fbadaffd640 1 --2- 172.21.15.16:0/3464378622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbad8ff9640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbac40a3820 0x7fbac40a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbadbfff640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbac40a3820 0x7fbac40a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbadaffd640 1 -- 172.21.15.16:0/3464378622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 msgr2=0x55d1e28b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbadaffd640 1 --2- 172.21.15.16:0/3464378622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbadaffd640 1 -- 172.21.15.16:0/3464378622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 msgr2=0x55d1e28bd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbad8ff9640 1 -- 172.21.15.16:0/1019322304 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fbacc0f1f90 con 0x7fbadc100ca0 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbadaffd640 1 --2- 172.21.15.16:0/3464378622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbadaffd640 1 -- 172.21.15.16:0/3464378622 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e252c210 con 0x55d1e28bd7e0 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/3464378622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbadaffd640 1 --2- 172.21.15.16:0/3464378622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28bfbd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009fa0 tx=0x7fbad00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677fb5da44fad16c server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbadbfff640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbac40a3820 0x7fbac40a5ce0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009fd0 tx=0x7fbad4004ab0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbac1ffb640 1 -- 172.21.15.16:0/3464378622 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0004030 con 0x55d1e28bd7e0 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbac1ffb640 1 -- 172.21.15.16:0/3464378622 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbad0005520 con 0x55d1e28bd7e0 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.978+0000 7fbac1ffb640 1 -- 172.21.15.16:0/3464378622 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0004dd0 con 0x55d1e28bd7e0 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.979+0000 7fbae4792740 1 -- 172.21.15.16:0/3464378622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 msgr2=0x55d1e28bfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.979+0000 7fbae4792740 1 --2- 172.21.15.16:0/3464378622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28bfbd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009fa0 tx=0x7fbad00118d0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.979+0000 7fbae4792740 1 -- 172.21.15.16:0/3464378622 shutdown_connections 2024-03-27T06:46:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.979+0000 7fbae4792740 1 --2- 172.21.15.16:0/3464378622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28bfbd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.979+0000 7fbae4792740 1 --2- 172.21.15.16:0/3464378622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.979+0000 7fbae4792740 1 --2- 172.21.15.16:0/3464378622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.979+0000 7fbae4792740 1 -- 172.21.15.16:0/3464378622 >> 172.21.15.16:0/3464378622 conn(0x55d1e28aea10 msgr2=0x55d1e28aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.979+0000 7fbae4792740 1 -- 172.21.15.16:0/3464378622 shutdown_connections 2024-03-27T06:46:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.979+0000 7fbae4792740 1 -- 172.21.15.16:0/3464378622 wait complete. 2024-03-27T06:46:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbae4792740 1 -- start start 2024-03-27T06:46:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbae4792740 1 Processor -- start 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbae4792740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28ba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbae4792740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbae4792740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28bb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbae4792740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d1e2813e00 con 0x55d1e28b4a10 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbae4792740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55d1e27c0520 con 0x55d1e28bd7e0 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbae4792740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d1e27d38e0 con 0x55d1e28b40f0 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbae4792740 10 client.0 _ll_register_callbacks cb 0x55d1e280ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbadaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbadaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39434/0 (socket says 172.21.15.16:39434) 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbadb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbadaffd640 1 -- 172.21.15.16:0/3468356315 learned_addr learned my addr 172.21.15.16:0/3468356315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.980+0000 7fbadbfff640 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28bb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbadaffd640 1 -- 172.21.15.16:0/3468356315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 msgr2=0x55d1e28ba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbadaffd640 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28ba810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbadaffd640 1 -- 172.21.15.16:0/3468356315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 msgr2=0x55d1e28bb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbadaffd640 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28bb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbadaffd640 1 -- 172.21.15.16:0/3468356315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e25f8620 con 0x55d1e28b4a10 2024-03-27T06:46:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28ba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:46:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbadaffd640 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bad50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbad0000c00 tx=0x7fbad0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59cb1ffae654213 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbabaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:46:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbabaffd640 1 -- 172.21.15.16:0/3468356315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad001d050 con 0x55d1e28b4a10 2024-03-27T06:46:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbabaffd640 1 -- 172.21.15.16:0/3468356315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fbad0005bd0 con 0x55d1e28b4a10 2024-03-27T06:46:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbae4792740 1 -- 172.21.15.16:0/3468356315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad0005520 con 0x55d1e28b4a10 2024-03-27T06:46:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbabaffd640 1 -- 172.21.15.16:0/3468356315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbad0032050 con 0x55d1e28b4a10 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbae4792740 10 client.16383 fetch_fsmap learned FSMap version 38 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbae4792740 1 -- 172.21.15.16:0/3468356315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d1e28cc3b0 con 0x55d1e28b4a10 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.981+0000 7fbabaffd640 1 -- 172.21.15.16:0/3468356315 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbad0005520 con 0x55d1e28b4a10 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.982+0000 7fbabaffd640 1 -- 172.21.15.16:0/3468356315 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8402+0+0 (secure 0 0 0) 0x7fbad0004fc0 con 0x55d1e28b4a10 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.982+0000 7fbae4792740 10 client.16383 fetch_fsmap finished waiting for FSMap version 38 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.982+0000 7fbae4792740 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.982+0000 7fbae4792740 20 client.16383 populate_metadata read hostname 'smithi016' 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.982+0000 7fbae4792740 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55d1e28cd760 0x55d1e28cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.982+0000 7fbae4792740 4 client.16383 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.982+0000 7fbae4792740 1 -- 172.21.15.16:0/3468356315 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1e2825390 con 0x55d1e28cd760 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.982+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55d1e28cd760 0x55d1e28cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.982+0000 7fbabaffd640 10 client.16383 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.982+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55d1e28cd760 0x55d1e28cfb50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1fa8198a2be49ed server_cookie=285652a6d53fc44d in_seq=0 out_seq=0 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.983+0000 7fbabaffd640 1 -- 172.21.15.16:0/3468356315 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d1e2825390 con 0x55d1e28cd760 2024-03-27T06:46:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.983+0000 7fbabaffd640 10 client.16383 handle_command_reply: tid=0 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.983+0000 7fbae4792740 1 client.16383 shutdown 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.983+0000 7fbae4792740 1 -- 172.21.15.16:0/3468356315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 msgr2=0x55d1e28bad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.983+0000 7fbae4792740 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bad50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbad0000c00 tx=0x7fbad0009310 comp rx=0 tx=0).stop 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.984+0000 7fbae4792740 1 -- 172.21.15.16:0/3468356315 shutdown_connections 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.984+0000 7fbae4792740 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55d1e28cd760 0x55d1e28cfb50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.984+0000 7fbae4792740 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28bb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.984+0000 7fbae4792740 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28bad50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.984+0000 7fbabaffd640 0 client.16383 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.984+0000 7fbae4792740 1 --2- 172.21.15.16:0/3468356315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28ba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.984+0000 7fbae4792740 1 -- 172.21.15.16:0/3468356315 >> 172.21.15.16:0/3468356315 conn(0x55d1e28aea10 msgr2=0x55d1e28b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.984+0000 7fbae4792740 1 -- 172.21.15.16:0/3468356315 shutdown_connections 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.984+0000 7fbae4792740 1 -- 172.21.15.16:0/3468356315 wait complete. 2024-03-27T06:46:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.984+0000 7fbae4792740 20 client.16383 trim_cache size 0 max 16384 2024-03-27T06:46:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbae4792740 1 Processor -- start 2024-03-27T06:46:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbae4792740 1 -- start start 2024-03-27T06:46:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbae4792740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbae4792740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbae4792740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbae4792740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1ad7c0 con 0x55d1e28b4a10 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbae4792740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1af450 con 0x55d1e28bd7e0 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbae4792740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1adec0 con 0x55d1e28b40f0 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbadaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbadaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39438/0 (socket says 172.21.15.16:39438) 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbadaffd640 1 -- 172.21.15.16:0/1178266993 learned_addr learned my addr 172.21.15.16:0/1178266993 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/1178266993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbadbfff640 1 --2- 172.21.15.16:0/1178266993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbadaffd640 1 -- 172.21.15.16:0/1178266993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 msgr2=0x55d1e28ba0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbadaffd640 1 --2- 172.21.15.16:0/1178266993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28ba0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbadaffd640 1 -- 172.21.15.16:0/1178266993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 msgr2=0x55d1e28b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbadaffd640 1 --2- 172.21.15.16:0/1178266993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbadaffd640 1 -- 172.21.15.16:0/1178266993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e28b8d60 con 0x55d1e28b4a10 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:31.999+0000 7fbadaffd640 1 --2- 172.21.15.16:0/1178266993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28c8eb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fbad0005460 tx=0x7fbad0009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec3163e38cc2d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbac17fa640 1 -- 172.21.15.16:0/1178266993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad001d040 con 0x55d1e28b4a10 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbac17fa640 1 -- 172.21.15.16:0/1178266993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbad003b070 con 0x55d1e28b4a10 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbac17fa640 1 -- 172.21.15.16:0/1178266993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0003800 con 0x55d1e28b4a10 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbae4792740 1 -- 172.21.15.16:0/1178266993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 msgr2=0x55d1e28c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbae4792740 1 --2- 172.21.15.16:0/1178266993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28c8eb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fbad0005460 tx=0x7fbad0009e20 comp rx=0 tx=0).stop 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbae4792740 1 -- 172.21.15.16:0/1178266993 shutdown_connections 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbae4792740 1 --2- 172.21.15.16:0/1178266993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28bd7e0 0x55d1e28b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbae4792740 1 --2- 172.21.15.16:0/1178266993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28b4a10 0x55d1e28c8eb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbae4792740 1 --2- 172.21.15.16:0/1178266993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b40f0 0x55d1e28ba0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbae4792740 1 -- 172.21.15.16:0/1178266993 >> 172.21.15.16:0/1178266993 conn(0x55d1e28aea10 msgr2=0x55d1e28ada70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbae4792740 1 -- 172.21.15.16:0/1178266993 shutdown_connections 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.000+0000 7fbae4792740 1 -- 172.21.15.16:0/1178266993 wait complete. 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbae4792740 1 -- start start 2024-03-27T06:46:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbae4792740 1 Processor -- start 2024-03-27T06:46:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbae4792740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28b40f0 0x55d1e28c06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbae4792740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b4a10 0x55d1e28c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbae4792740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28bd7e0 0x55d1e292ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbae4792740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1aed80 con 0x55d1e28bd7e0 2024-03-27T06:46:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbae4792740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1ae5f0 con 0x55d1e28b40f0 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbae4792740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc1ab7a0 con 0x55d1e28b4a10 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbae4792740 10 client.0 _ll_register_callbacks cb 0x7fbacc0ba580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbadbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28bd7e0 0x55d1e292ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbadbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28bd7e0 0x55d1e292ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39450/0 (socket says 172.21.15.16:39450) 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbadbfff640 1 -- 172.21.15.16:0/738532578 learned_addr learned my addr 172.21.15.16:0/738532578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbadaffd640 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b4a10 0x55d1e28c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28b40f0 0x55d1e28c06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbadbfff640 1 -- 172.21.15.16:0/738532578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b4a10 msgr2=0x55d1e28c0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbadbfff640 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b4a10 0x55d1e28c0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbadbfff640 1 -- 172.21.15.16:0/738532578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28b40f0 msgr2=0x55d1e28c06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbadbfff640 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28b40f0 0x55d1e28c06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.001+0000 7fbadbfff640 1 -- 172.21.15.16:0/738532578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e292c060 con 0x55d1e28bd7e0 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbadbfff640 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28bd7e0 0x55d1e292ba20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbad4003dc0 tx=0x7fbad4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5391219d6f7b49 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbabaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:46:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbabaffd640 1 -- 172.21.15.16:0/738532578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4004550 con 0x55d1e28bd7e0 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbabaffd640 1 -- 172.21.15.16:0/738532578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fbad40046f0 con 0x55d1e28bd7e0 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbae4792740 1 -- 172.21.15.16:0/738532578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad003b070 con 0x55d1e28bd7e0 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbabaffd640 1 -- 172.21.15.16:0/738532578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbad4004b20 con 0x55d1e28bd7e0 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbae4792740 10 client.16389 fetch_fsmap learned FSMap version 38 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbae4792740 1 -- 172.21.15.16:0/738532578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d1e2933460 con 0x55d1e28bd7e0 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbabaffd640 1 -- 172.21.15.16:0/738532578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbad003b070 con 0x55d1e28bd7e0 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbabaffd640 1 -- 172.21.15.16:0/738532578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 38) ==== 8402+0+0 (secure 0 0 0) 0x7fbad401f040 con 0x55d1e28bd7e0 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbae4792740 10 client.16389 fetch_fsmap finished waiting for FSMap version 38 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbae4792740 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbae4792740 20 client.16389 populate_metadata read hostname 'smithi016' 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbae4792740 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55d1e28cd760 0x55d1e2935c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbae4792740 4 client.16389 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.002+0000 7fbae4792740 1 -- 172.21.15.16:0/738532578 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1e281f6e0 con 0x55d1e28cd760 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.003+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55d1e28cd760 0x55d1e2935c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.003+0000 7fbabaffd640 10 client.16389 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.003+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55d1e28cd760 0x55d1e2935c90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=681362f3013ea63e server_cookie=b4ebb9d66c3f4246 in_seq=0 out_seq=0 2024-03-27T06:46:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbabaffd640 1 -- 172.21.15.16:0/738532578 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1e281f6e0 con 0x55d1e28cd760 2024-03-27T06:46:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbabaffd640 10 client.16389 handle_command_reply: tid=0 2024-03-27T06:46:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 1 client.16389 shutdown 2024-03-27T06:46:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 1 -- 172.21.15.16:0/738532578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28bd7e0 msgr2=0x55d1e292ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28bd7e0 0x55d1e292ba20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbad4003dc0 tx=0x7fbad4004a40 comp rx=0 tx=0).stop 2024-03-27T06:46:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 1 -- 172.21.15.16:0/738532578 shutdown_connections 2024-03-27T06:46:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55d1e28cd760 0x55d1e2935c90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1e28bd7e0 0x55d1e292ba20 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbabaffd640 0 client.16389 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1e28b4a10 0x55d1e28c0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 1 --2- 172.21.15.16:0/738532578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1e28b40f0 0x55d1e28c06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 1 -- 172.21.15.16:0/738532578 >> 172.21.15.16:0/738532578 conn(0x55d1e28aea10 msgr2=0x55d1e28ada70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 1 -- 172.21.15.16:0/738532578 shutdown_connections 2024-03-27T06:46:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 1 -- 172.21.15.16:0/738532578 wait complete. 2024-03-27T06:46:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.031+0000 7fbae4792740 20 client.16389 trim_cache size 0 max 16384 2024-03-27T06:46:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:46:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:46:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:46:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:46:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 -- 172.21.15.16:0/1019322304 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbac40a3820 msgr2=0x7fbac40a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbac40a3820 0x7fbac40a5ce0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009fd0 tx=0x7fbad4004ab0 comp rx=0 tx=0).stop 2024-03-27T06:46:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 -- 172.21.15.16:0/1019322304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc100ca0 msgr2=0x7fbadc1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc100ca0 0x7fbadc1984a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbacc005940 tx=0x7fbacc02dc80 comp rx=0 tx=0).stop 2024-03-27T06:46:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 -- 172.21.15.16:0/1019322304 shutdown_connections 2024-03-27T06:46:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbac40a3820 0x7fbac40a5ce0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbadc1059f0 0x7fbadc19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc101660 0x7fbadc1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 --2- 172.21.15.16:0/1019322304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbadc100ca0 0x7fbadc1984a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 -- 172.21.15.16:0/1019322304 >> 172.21.15.16:0/1019322304 conn(0x7fbadc0778a0 msgr2=0x7fbadc069370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 -- 172.21.15.16:0/1019322304 shutdown_connections 2024-03-27T06:46:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:32.032+0000 7fbae1bac640 1 -- 172.21.15.16:0/1019322304 wait complete. 2024-03-27T06:46:32.394 INFO:tasks.workunit.client.0.smithi016.stderr:+ tar c linux-4.0.5 2024-03-27T06:46:32.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:32 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:32.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:32 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:32.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:32 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:33.066 INFO:tasks.workunit.client.0.smithi016.stdout:Ok 2024-03-27T06:46:33.067 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo Ok 2024-03-27T06:46:33.067 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-27T06:46:33.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-27T06:46:33.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:33 smithi156 ceph-mon[36261]: pgmap v93: 97 pgs: 97 active+clean; 137 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-03-27T06:46:33.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:33 smithi170 ceph-mon[36438]: pgmap v93: 97 pgs: 97 active+clean; 137 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-03-27T06:46:33.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:33 smithi016 ceph-mon[31908]: pgmap v93: 97 pgs: 97 active+clean; 137 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-03-27T06:46:35.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:35 smithi170 ceph-mon[36438]: pgmap v94: 97 pgs: 97 active+clean; 137 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 1021 KiB/s wr, 158 op/s 2024-03-27T06:46:35.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:35 smithi156 ceph-mon[36261]: pgmap v94: 97 pgs: 97 active+clean; 137 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 1021 KiB/s wr, 158 op/s 2024-03-27T06:46:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:35 smithi016 ceph-mon[31908]: pgmap v94: 97 pgs: 97 active+clean; 137 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 1021 KiB/s wr, 158 op/s 2024-03-27T06:46:37.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:37 smithi156 ceph-mon[36261]: pgmap v95: 97 pgs: 97 active+clean; 161 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 393 op/s 2024-03-27T06:46:37.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:37 smithi170 ceph-mon[36438]: pgmap v95: 97 pgs: 97 active+clean; 161 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 393 op/s 2024-03-27T06:46:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:37 smithi016 ceph-mon[31908]: pgmap v95: 97 pgs: 97 active+clean; 161 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 393 op/s 2024-03-27T06:46:39.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:39 smithi156 ceph-mon[36261]: pgmap v96: 97 pgs: 97 active+clean; 161 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 249 op/s 2024-03-27T06:46:39.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:39 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:39.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:39 smithi170 ceph-mon[36438]: pgmap v96: 97 pgs: 97 active+clean; 161 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 249 op/s 2024-03-27T06:46:39.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:39 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:39 smithi016 ceph-mon[31908]: pgmap v96: 97 pgs: 97 active+clean; 161 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 249 op/s 2024-03-27T06:46:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:39 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:41.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:41 smithi156 ceph-mon[36261]: pgmap v97: 97 pgs: 97 active+clean; 164 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-27T06:46:41.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:41 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:41 smithi170 ceph-mon[36438]: pgmap v97: 97 pgs: 97 active+clean; 164 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-27T06:46:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:41 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:41.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:41 smithi016 ceph-mon[31908]: pgmap v97: 97 pgs: 97 active+clean; 164 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 314 op/s 2024-03-27T06:46:41.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:41 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:42.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:42 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:46:42.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:42 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:46:42.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:42 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:46:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:43 smithi156 ceph-mon[36261]: pgmap v98: 97 pgs: 97 active+clean; 166 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-03-27T06:46:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:43 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:46:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:43 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:46:43.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:43 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:46:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:43 smithi170 ceph-mon[36438]: pgmap v98: 97 pgs: 97 active+clean; 166 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-03-27T06:46:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:43 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:46:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:43 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:46:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:43 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:46:43.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:43 smithi016 ceph-mon[31908]: pgmap v98: 97 pgs: 97 active+clean; 166 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-03-27T06:46:43.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:43 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:46:43.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:43 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:46:43.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:43 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:46:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.122+0000 7fa36b80e640 1 -- 172.21.15.16:0/1448857480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364101200 msgr2=0x7fa364101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.122+0000 7fa36b80e640 1 --2- 172.21.15.16:0/1448857480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364101200 0x7fa364101680 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa354009900 tx=0x7fa35402bc30 comp rx=0 tx=0).stop 2024-03-27T06:46:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.122+0000 7fa36b80e640 1 -- 172.21.15.16:0/1448857480 shutdown_connections 2024-03-27T06:46:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.122+0000 7fa36b80e640 1 --2- 172.21.15.16:0/1448857480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa364101bc0 0x7fa36410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.122+0000 7fa36b80e640 1 --2- 172.21.15.16:0/1448857480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364101200 0x7fa364101680 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.122+0000 7fa36b80e640 1 --2- 172.21.15.16:0/1448857480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa364105a50 0x7fa364105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.122+0000 7fa36b80e640 1 -- 172.21.15.16:0/1448857480 >> 172.21.15.16:0/1448857480 conn(0x7fa364076f70 msgr2=0x7fa3640fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.122+0000 7fa36b80e640 1 -- 172.21.15.16:0/1448857480 shutdown_connections 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.122+0000 7fa36b80e640 1 -- 172.21.15.16:0/1448857480 wait complete. 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa36b80e640 1 Processor -- start 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa36b80e640 1 -- start start 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa36b80e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa364101200 0x7fa364197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa36b80e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364101bc0 0x7fa3641977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa36b80e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa364105a50 0x7fa36419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa36b80e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa364110f20 con 0x7fa364101bc0 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa36b80e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fa364110da0 con 0x7fa364101200 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa36b80e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641110a0 con 0x7fa364105a50 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa368da9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364101bc0 0x7fa3641977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa368da9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364101bc0 0x7fa3641977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46482/0 (socket says 172.21.15.16:46482) 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa368da9640 1 -- 172.21.15.16:0/2057338178 learned_addr learned my addr 172.21.15.16:0/2057338178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa3695aa640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa364101200 0x7fa364197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa369dab640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa364105a50 0x7fa36419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa368da9640 1 -- 172.21.15.16:0/2057338178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa364105a50 msgr2=0x7fa36419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa368da9640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa364105a50 0x7fa36419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa368da9640 1 -- 172.21.15.16:0/2057338178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa364101200 msgr2=0x7fa364197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa368da9640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa364101200 0x7fa364197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.123+0000 7fa368da9640 1 -- 172.21.15.16:0/2057338178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36419c270 con 0x7fa364101bc0 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.124+0000 7fa368da9640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364101bc0 0x7fa3641977c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa354005ab0 tx=0x7fa35402dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.124+0000 7fa35a7fc640 1 -- 172.21.15.16:0/2057338178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354043020 con 0x7fa364101bc0 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.124+0000 7fa36b80e640 1 -- 172.21.15.16:0/2057338178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36419c500 con 0x7fa364101bc0 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.124+0000 7fa35a7fc640 1 -- 172.21.15.16:0/2057338178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa354004e50 con 0x7fa364101bc0 2024-03-27T06:46:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.124+0000 7fa35a7fc640 1 -- 172.21.15.16:0/2057338178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354003970 con 0x7fa364101bc0 2024-03-27T06:46:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.124+0000 7fa36b80e640 1 -- 172.21.15.16:0/2057338178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3641a3df0 con 0x7fa364101bc0 2024-03-27T06:46:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa35a7fc640 1 -- 172.21.15.16:0/2057338178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fa354046020 con 0x7fa364101bc0 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa36e3f6740 1 Processor -- start 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa36e3f6740 1 -- start start 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5c12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e6fd57ecd0 con 0x55e6fd5b80f0 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e6fd4c3950 con 0x55e6fd5b8a10 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e6fd51be50 con 0x55e6fd5c17e0 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa369dab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa369dab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5b84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46498/0 (socket says 172.21.15.16:46498) 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa369dab640 1 -- 172.21.15.16:0/3230068164 learned_addr learned my addr 172.21.15.16:0/3230068164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa368da9640 1 --2- 172.21.15.16:0/3230068164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa3695aa640 1 --2- 172.21.15.16:0/3230068164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5c12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa35a7fc640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa3500a3820 0x7fa3500a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.127+0000 7fa369dab640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa3500a3820 0x7fa3500a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa369dab640 1 -- 172.21.15.16:0/3230068164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5c17e0 msgr2=0x55e6fd5c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa369dab640 1 --2- 172.21.15.16:0/3230068164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa35a7fc640 1 -- 172.21.15.16:0/2057338178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fa3540f1fd0 con 0x7fa364101bc0 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa369dab640 1 -- 172.21.15.16:0/3230068164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5b8a10 msgr2=0x55e6fd5c12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa369dab640 1 --2- 172.21.15.16:0/3230068164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5c12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa369dab640 1 -- 172.21.15.16:0/3230068164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6fd230210 con 0x55e6fd5b80f0 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa368da9640 1 --2- 172.21.15.16:0/3230068164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa369dab640 1 --2- 172.21.15.16:0/3230068164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5b84d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa35c005c40 tx=0x7fa35c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a748be959490507f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa34f7fe640 1 -- 172.21.15.16:0/3230068164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c022020 con 0x55e6fd5b80f0 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa34f7fe640 1 -- 172.21.15.16:0/3230068164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa35c013170 con 0x55e6fd5b80f0 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa34f7fe640 1 -- 172.21.15.16:0/3230068164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c0107b0 con 0x55e6fd5b80f0 2024-03-27T06:46:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.128+0000 7fa369dab640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa3500a3820 0x7fa3500a5ce0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa35c023000 tx=0x7fa35c021000 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.129+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3230068164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 msgr2=0x55e6fd5b84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.129+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/3230068164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5b84d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa35c005c40 tx=0x7fa35c00f3c0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.129+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3230068164 shutdown_connections 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.129+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/3230068164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.129+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/3230068164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5c12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.129+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/3230068164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5b84d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.129+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3230068164 >> 172.21.15.16:0/3230068164 conn(0x55e6fd5b2a10 msgr2=0x55e6fd5b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.129+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3230068164 shutdown_connections 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.129+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3230068164 wait complete. 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.129+0000 7fa36e3f6740 1 -- start start 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.129+0000 7fa36e3f6740 1 Processor -- start 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5bf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e6fd517e00 con 0x55e6fd5b8a10 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x55e6fd4c4520 con 0x55e6fd5c17e0 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e6fd4d78e0 con 0x55e6fd5b80f0 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa36e3f6740 10 client.0 _ll_register_callbacks cb 0x55e6fd512a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa368da9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa368da9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46502/0 (socket says 172.21.15.16:46502) 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa368da9640 1 -- 172.21.15.16:0/461894108 learned_addr learned my addr 172.21.15.16:0/461894108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa3695aa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa369dab640 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5bf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa368da9640 1 -- 172.21.15.16:0/461894108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5b80f0 msgr2=0x55e6fd5be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa368da9640 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5be830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa368da9640 1 -- 172.21.15.16:0/461894108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5c17e0 msgr2=0x55e6fd5bf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa368da9640 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5bf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa368da9640 1 -- 172.21.15.16:0/461894108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6fd2fc620 con 0x55e6fd5b8a10 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa368da9640 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bed70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa354006010 tx=0x7fa354009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137e2fbf34e9219 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa34cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa34cff9640 1 -- 172.21.15.16:0/461894108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3540be9a0 con 0x55e6fd5b8a10 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa34cff9640 1 -- 172.21.15.16:0/461894108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fa3540f4110 con 0x55e6fd5b8a10 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.130+0000 7fa36e3f6740 1 -- 172.21.15.16:0/461894108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa35c013170 con 0x55e6fd5b8a10 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa34cff9640 1 -- 172.21.15.16:0/461894108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa3540c2650 con 0x55e6fd5b8a10 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa36e3f6740 10 client.16401 fetch_fsmap learned FSMap version 40 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa36e3f6740 1 -- 172.21.15.16:0/461894108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e6fd5d0390 con 0x55e6fd5b8a10 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa34cff9640 1 -- 172.21.15.16:0/461894108 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa35c013170 con 0x55e6fd5b8a10 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa34cff9640 1 -- 172.21.15.16:0/461894108 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 40) ==== 8410+0+0 (secure 0 0 0) 0x7fa3540be500 con 0x55e6fd5b8a10 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa36e3f6740 10 client.16401 fetch_fsmap finished waiting for FSMap version 40 2024-03-27T06:46:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa36e3f6740 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa36e3f6740 20 client.16401 populate_metadata read hostname 'smithi016' 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e6fd5d1920 0x55e6fd5d3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa36e3f6740 4 client.16401 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa36e3f6740 1 -- 172.21.15.16:0/461894108 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6fd529390 con 0x55e6fd5d1920 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.131+0000 7fa3695aa640 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e6fd5d1920 0x55e6fd5d3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa3695aa640 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e6fd5d1920 0x55e6fd5d3d80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd6ecbb2f44750d6 server_cookie=509540ec2b5ee179 in_seq=0 out_seq=0 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa34cff9640 10 client.16401 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa34cff9640 1 -- 172.21.15.16:0/461894108 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e6fd529390 con 0x55e6fd5d1920 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa34cff9640 10 client.16401 handle_command_reply: tid=0 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa36e3f6740 1 client.16401 shutdown 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa36e3f6740 1 -- 172.21.15.16:0/461894108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 msgr2=0x55e6fd5bed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bed70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa354006010 tx=0x7fa354009560 comp rx=0 tx=0).stop 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa36e3f6740 1 -- 172.21.15.16:0/461894108 shutdown_connections 2024-03-27T06:46:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e6fd5d1920 0x55e6fd5d3d80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5bf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bed70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/461894108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa34cff9640 0 client.16401 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.132+0000 7fa36e3f6740 1 -- 172.21.15.16:0/461894108 >> 172.21.15.16:0/461894108 conn(0x55e6fd5b2a10 msgr2=0x55e6fd5b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.133+0000 7fa36e3f6740 1 -- 172.21.15.16:0/461894108 shutdown_connections 2024-03-27T06:46:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.133+0000 7fa36e3f6740 1 -- 172.21.15.16:0/461894108 wait complete. 2024-03-27T06:46:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.133+0000 7fa36e3f6740 20 client.16401 trim_cache size 0 max 16384 2024-03-27T06:46:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.147+0000 7fa36e3f6740 1 Processor -- start 2024-03-27T06:46:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.147+0000 7fa36e3f6740 1 -- start start 2024-03-27T06:46:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5c0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641ac5f0 con 0x55e6fd5b80f0 2024-03-27T06:46:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641ae2e0 con 0x55e6fd5b8a10 2024-03-27T06:46:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641acd20 con 0x55e6fd5c17e0 2024-03-27T06:46:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa3695aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5c0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa3695aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5c0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46510/0 (socket says 172.21.15.16:46510) 2024-03-27T06:46:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa3695aa640 1 -- 172.21.15.16:0/1683878077 learned_addr learned my addr 172.21.15.16:0/1683878077 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa369dab640 1 --2- 172.21.15.16:0/1683878077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa368da9640 1 --2- 172.21.15.16:0/1683878077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa3695aa640 1 -- 172.21.15.16:0/1683878077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5c17e0 msgr2=0x55e6fd5c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa3695aa640 1 --2- 172.21.15.16:0/1683878077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa3695aa640 1 -- 172.21.15.16:0/1683878077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5b8a10 msgr2=0x55e6fd5cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa3695aa640 1 --2- 172.21.15.16:0/1683878077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa3695aa640 1 -- 172.21.15.16:0/1683878077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6fd5ce150 con 0x55e6fd5b80f0 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa3695aa640 1 --2- 172.21.15.16:0/1683878077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5c0f60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa360009e50 tx=0x7fa360010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3db140cb8998a80 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa34effd640 1 -- 172.21.15.16:0/1683878077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360004230 con 0x55e6fd5b80f0 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa34effd640 1 -- 172.21.15.16:0/1683878077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa3600043d0 con 0x55e6fd5b80f0 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.148+0000 7fa34effd640 1 -- 172.21.15.16:0/1683878077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360003d10 con 0x55e6fd5b80f0 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.149+0000 7fa36e3f6740 1 -- 172.21.15.16:0/1683878077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 msgr2=0x55e6fd5c0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.149+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/1683878077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5c0f60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa360009e50 tx=0x7fa360010970 comp rx=0 tx=0).stop 2024-03-27T06:46:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.149+0000 7fa36e3f6740 1 -- 172.21.15.16:0/1683878077 shutdown_connections 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.149+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/1683878077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.149+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/1683878077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.149+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/1683878077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5c0f60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.149+0000 7fa36e3f6740 1 -- 172.21.15.16:0/1683878077 >> 172.21.15.16:0/1683878077 conn(0x55e6fd5b2a10 msgr2=0x55e6fd5be7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.149+0000 7fa36e3f6740 1 -- 172.21.15.16:0/1683878077 shutdown_connections 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.149+0000 7fa36e3f6740 1 -- 172.21.15.16:0/1683878077 wait complete. 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.149+0000 7fa36e3f6740 1 -- start start 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.149+0000 7fa36e3f6740 1 Processor -- start 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5bac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa36e3f6740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641adc10 con 0x55e6fd5b8a10 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641ad480 con 0x55e6fd5c17e0 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa36e3f6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641aa5d0 con 0x55e6fd5b80f0 2024-03-27T06:46:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa36e3f6740 10 client.0 _ll_register_callbacks cb 0x7fa3540057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-27T06:46:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa368da9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa3695aa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5bac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa368da9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46524/0 (socket says 172.21.15.16:46524) 2024-03-27T06:46:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa368da9640 1 -- 172.21.15.16:0/3205839412 learned_addr learned my addr 172.21.15.16:0/3205839412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:46:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa369dab640 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa368da9640 1 -- 172.21.15.16:0/3205839412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5b80f0 msgr2=0x55e6fd5bac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa368da9640 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5bac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa368da9640 1 -- 172.21.15.16:0/3205839412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5c17e0 msgr2=0x55e6fd5bb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-27T06:46:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa368da9640 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5bb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa368da9640 1 -- 172.21.15.16:0/3205839412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6fd5bc000 con 0x55e6fd5b8a10 2024-03-27T06:46:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa368da9640 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bb1d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa354005250 tx=0x7fa3540d2e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261401923e96d48f server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:46:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa34cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-27T06:46:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa34cff9640 1 -- 172.21.15.16:0/3205839412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3540be9a0 con 0x55e6fd5b8a10 2024-03-27T06:46:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa34cff9640 1 -- 172.21.15.16:0/3205839412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8469+0+0 (secure 0 0 0) 0x7fa3540bd6f0 con 0x55e6fd5b8a10 2024-03-27T06:46:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.150+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3205839412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3600043d0 con 0x55e6fd5b8a10 2024-03-27T06:46:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa34cff9640 1 -- 172.21.15.16:0/3205839412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa3540c6f10 con 0x55e6fd5b8a10 2024-03-27T06:46:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa36e3f6740 10 client.16407 fetch_fsmap learned FSMap version 40 2024-03-27T06:46:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3205839412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e6fd63a4d0 con 0x55e6fd5b8a10 2024-03-27T06:46:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa34cff9640 1 -- 172.21.15.16:0/3205839412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa3600043d0 con 0x55e6fd5b8a10 2024-03-27T06:46:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa34cff9640 1 -- 172.21.15.16:0/3205839412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 40) ==== 8410+0+0 (secure 0 0 0) 0x7fa3540b9ff0 con 0x55e6fd5b8a10 2024-03-27T06:46:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa36e3f6740 10 client.16407 fetch_fsmap finished waiting for FSMap version 40 2024-03-27T06:46:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa36e3f6740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-27T06:46:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa36e3f6740 20 client.16407 populate_metadata read hostname 'smithi016' 2024-03-27T06:46:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e6fd63ab30 0x55e6fd63cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:46:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa36e3f6740 4 client.16407 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-27T06:46:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3205839412 --> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6fd5236e0 con 0x55e6fd63ab30 2024-03-27T06:46:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.151+0000 7fa3695aa640 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e6fd63ab30 0x55e6fd63cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:46:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.152+0000 7fa3695aa640 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e6fd63ab30 0x55e6fd63cf50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0afad90d89bcf48 server_cookie=bd29849f6ce0c3d4 in_seq=0 out_seq=0 2024-03-27T06:46:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.152+0000 7fa34cff9640 10 client.16407 ms_handle_connect on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.167+0000 7fa34cff9640 1 -- 172.21.15.16:0/3205839412 <== mds.0 v2:172.21.15.156:6838/479168397 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6fd5236e0 con 0x55e6fd63ab30 2024-03-27T06:46:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.167+0000 7fa34cff9640 10 client.16407 handle_command_reply: tid=0 2024-03-27T06:46:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.167+0000 7fa36e3f6740 1 client.16407 shutdown 2024-03-27T06:46:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.167+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3205839412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 msgr2=0x55e6fd5bb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.167+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bb1d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa354005250 tx=0x7fa3540d2e00 comp rx=0 tx=0).stop 2024-03-27T06:46:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3205839412 shutdown_connections 2024-03-27T06:46:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.156:6838/479168397,v1:172.21.15.156:6839/479168397] conn(0x55e6fd63ab30 0x55e6fd63cf50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6fd5c17e0 0x55e6fd5bb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa34cff9640 0 client.16407 ms_handle_reset on v2:172.21.15.156:6838/479168397 2024-03-27T06:46:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6fd5b8a10 0x55e6fd5bb1d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36e3f6740 1 --2- 172.21.15.16:0/3205839412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6fd5b80f0 0x55e6fd5bac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3205839412 >> 172.21.15.16:0/3205839412 conn(0x55e6fd5b2a10 msgr2=0x55e6fd5b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3205839412 shutdown_connections 2024-03-27T06:46:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36e3f6740 1 -- 172.21.15.16:0/3205839412 wait complete. 2024-03-27T06:46:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36e3f6740 20 client.16407 trim_cache size 0 max 16384 2024-03-27T06:46:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-03-27T06:46:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-03-27T06:46:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-03-27T06:46:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-03-27T06:46:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 -- 172.21.15.16:0/2057338178 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa3500a3820 msgr2=0x7fa3500a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa3500a3820 0x7fa3500a5ce0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa35c023000 tx=0x7fa35c021000 comp rx=0 tx=0).stop 2024-03-27T06:46:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 -- 172.21.15.16:0/2057338178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364101bc0 msgr2=0x7fa3641977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:46:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364101bc0 0x7fa3641977c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa354005ab0 tx=0x7fa35402dc00 comp rx=0 tx=0).stop 2024-03-27T06:46:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 -- 172.21.15.16:0/2057338178 shutdown_connections 2024-03-27T06:46:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa3500a3820 0x7fa3500a5ce0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa364105a50 0x7fa36419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364101bc0 0x7fa3641977c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 --2- 172.21.15.16:0/2057338178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa364101200 0x7fa364197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:46:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 -- 172.21.15.16:0/2057338178 >> 172.21.15.16:0/2057338178 conn(0x7fa364076f70 msgr2=0x7fa364103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:46:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 -- 172.21.15.16:0/2057338178 shutdown_connections 2024-03-27T06:46:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-03-27T06:46:45.168+0000 7fa36b80e640 1 -- 172.21.15.16:0/2057338178 wait complete. 2024-03-27T06:46:45.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:45 smithi156 ceph-mon[36261]: pgmap v99: 97 pgs: 97 active+clean; 166 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 368 op/s 2024-03-27T06:46:45.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:45 smithi170 ceph-mon[36438]: pgmap v99: 97 pgs: 97 active+clean; 166 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 368 op/s 2024-03-27T06:46:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:45 smithi016 ceph-mon[31908]: pgmap v99: 97 pgs: 97 active+clean; 166 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 368 op/s 2024-03-27T06:46:46.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:46 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:46.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:46 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:46.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:46 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:47.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:47 smithi156 ceph-mon[36261]: pgmap v100: 97 pgs: 97 active+clean; 166 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 499 op/s 2024-03-27T06:46:47.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:47 smithi170 ceph-mon[36438]: pgmap v100: 97 pgs: 97 active+clean; 166 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 499 op/s 2024-03-27T06:46:47.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:47 smithi016 ceph-mon[31908]: pgmap v100: 97 pgs: 97 active+clean; 166 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 499 op/s 2024-03-27T06:46:48.028 INFO:tasks.workunit:Stopping ['fs/norstats'] on client.0... 2024-03-27T06:46:48.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-27T06:46:48.300 DEBUG:teuthology.parallel:result is None 2024-03-27T06:46:48.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-27T06:46:48.339 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-27T06:46:48.340 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-27T06:46:48.351 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi016.front.sepia.ceph.com 2024-03-27T06:46:48.388 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-27T06:46:48.398 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-27T06:46:48.398 INFO:teuthology.task.exec:Executing custom commands... 2024-03-27T06:46:48.398 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-03-27T06:46:48.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-27T06:46:48.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-27T06:46:49.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:49 smithi156 ceph-mon[36261]: pgmap v101: 97 pgs: 97 active+clean; 166 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 264 op/s 2024-03-27T06:46:49.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:49 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:49.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:49 smithi016 ceph-mon[31908]: pgmap v101: 97 pgs: 97 active+clean; 166 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 264 op/s 2024-03-27T06:46:49.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:49 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:49.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:49 smithi170 ceph-mon[36438]: pgmap v101: 97 pgs: 97 active+clean; 166 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 264 op/s 2024-03-27T06:46:49.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:49 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:51.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:51 smithi156 ceph-mon[36261]: pgmap v102: 97 pgs: 97 active+clean; 162 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 326 op/s 2024-03-27T06:46:51.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:51 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:51.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:51 smithi170 ceph-mon[36438]: pgmap v102: 97 pgs: 97 active+clean; 162 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 326 op/s 2024-03-27T06:46:51.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:51 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:51.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:51 smithi016 ceph-mon[31908]: pgmap v102: 97 pgs: 97 active+clean; 162 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 326 op/s 2024-03-27T06:46:51.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:51 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:46:53.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:53 smithi156 ceph-mon[36261]: pgmap v103: 97 pgs: 97 active+clean; 160 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 340 op/s 2024-03-27T06:46:53.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:53 smithi170 ceph-mon[36438]: pgmap v103: 97 pgs: 97 active+clean; 160 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 340 op/s 2024-03-27T06:46:53.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:53 smithi016 ceph-mon[31908]: pgmap v103: 97 pgs: 97 active+clean; 160 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 340 op/s 2024-03-27T06:46:55.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:55 smithi156 ceph-mon[36261]: pgmap v104: 97 pgs: 97 active+clean; 160 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 936 KiB/s wr, 271 op/s 2024-03-27T06:46:55.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:55 smithi170 ceph-mon[36438]: pgmap v104: 97 pgs: 97 active+clean; 160 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 936 KiB/s wr, 271 op/s 2024-03-27T06:46:55.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:55 smithi016 ceph-mon[31908]: pgmap v104: 97 pgs: 97 active+clean; 160 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 936 KiB/s wr, 271 op/s 2024-03-27T06:46:57.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:57 smithi156 ceph-mon[36261]: pgmap v105: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 940 KiB/s wr, 277 op/s 2024-03-27T06:46:57.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:57 smithi170 ceph-mon[36438]: pgmap v105: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 940 KiB/s wr, 277 op/s 2024-03-27T06:46:57.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:57 smithi016 ceph-mon[31908]: pgmap v105: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 940 KiB/s wr, 277 op/s 2024-03-27T06:46:59.478 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:46:59 smithi156 ceph-mon[36261]: pgmap v106: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 407 KiB/s wr, 147 op/s 2024-03-27T06:46:59.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:46:59 smithi170 ceph-mon[36438]: pgmap v106: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 407 KiB/s wr, 147 op/s 2024-03-27T06:46:59.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:46:59 smithi016 ceph-mon[31908]: pgmap v106: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 407 KiB/s wr, 147 op/s 2024-03-27T06:47:01.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:01 smithi156 ceph-mon[36261]: pgmap v107: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 407 KiB/s wr, 150 op/s 2024-03-27T06:47:01.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:01 smithi170 ceph-mon[36438]: pgmap v107: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 407 KiB/s wr, 150 op/s 2024-03-27T06:47:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:01 smithi016 ceph-mon[31908]: pgmap v107: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 407 KiB/s wr, 150 op/s 2024-03-27T06:47:03.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:03 smithi156 ceph-mon[36261]: pgmap v108: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 322 KiB/s wr, 89 op/s 2024-03-27T06:47:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:03 smithi170 ceph-mon[36438]: pgmap v108: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 322 KiB/s wr, 89 op/s 2024-03-27T06:47:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:03 smithi016 ceph-mon[31908]: pgmap v108: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 322 KiB/s wr, 89 op/s 2024-03-27T06:47:05.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:05 smithi156 ceph-mon[36261]: pgmap v109: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-03-27T06:47:05.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:05 smithi170 ceph-mon[36438]: pgmap v109: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-03-27T06:47:05.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:05 smithi016 ceph-mon[31908]: pgmap v109: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-03-27T06:47:07.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:07 smithi156 ceph-mon[36261]: pgmap v110: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 30 op/s 2024-03-27T06:47:07.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:07 smithi170 ceph-mon[36438]: pgmap v110: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 30 op/s 2024-03-27T06:47:07.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:07 smithi016 ceph-mon[31908]: pgmap v110: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 30 op/s 2024-03-27T06:47:09.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:09 smithi156 ceph-mon[36261]: pgmap v111: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-27T06:47:09.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:09 smithi170 ceph-mon[36438]: pgmap v111: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-27T06:47:09.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:09 smithi016 ceph-mon[31908]: pgmap v111: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-27T06:47:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:11 smithi156 ceph-mon[36261]: pgmap v112: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-27T06:47:11.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:11 smithi170 ceph-mon[36438]: pgmap v112: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-27T06:47:11.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:11 smithi016 ceph-mon[31908]: pgmap v112: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-27T06:47:13.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:13 smithi156 ceph-mon[36261]: pgmap v113: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-27T06:47:13.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:13 smithi170 ceph-mon[36438]: pgmap v113: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-27T06:47:13.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:13 smithi016 ceph-mon[31908]: pgmap v113: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-27T06:47:15.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:15 smithi156 ceph-mon[36261]: pgmap v114: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-27T06:47:15.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:15 smithi170 ceph-mon[36438]: pgmap v114: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-27T06:47:15.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:15 smithi016 ceph-mon[31908]: pgmap v114: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-27T06:47:17.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:17 smithi156 ceph-mon[36261]: pgmap v115: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-27T06:47:17.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:17 smithi170 ceph-mon[36438]: pgmap v115: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-27T06:47:17.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:17 smithi016 ceph-mon[31908]: pgmap v115: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-27T06:47:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:19 smithi156 ceph-mon[36261]: pgmap v116: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-27T06:47:19.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:19 smithi170 ceph-mon[36438]: pgmap v116: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-27T06:47:19.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:19 smithi016 ceph-mon[31908]: pgmap v116: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-27T06:47:21.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:21 smithi156 ceph-mon[36261]: pgmap v117: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-27T06:47:21.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:21 smithi170 ceph-mon[36438]: pgmap v117: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-27T06:47:21.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:21 smithi016 ceph-mon[31908]: pgmap v117: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-27T06:47:23.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:23 smithi156 ceph-mon[36261]: pgmap v118: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:47:23.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:23 smithi170 ceph-mon[36438]: pgmap v118: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:47:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:23 smithi016 ceph-mon[31908]: pgmap v118: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:47:25.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:25 smithi156 ceph-mon[36261]: pgmap v119: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:25.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:25 smithi170 ceph-mon[36438]: pgmap v119: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:25 smithi016 ceph-mon[31908]: pgmap v119: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:27.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:27 smithi156 ceph-mon[36261]: pgmap v120: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:27.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:27 smithi170 ceph-mon[36438]: pgmap v120: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:27 smithi016 ceph-mon[31908]: pgmap v120: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:29.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:29 smithi156 ceph-mon[36261]: pgmap v121: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:29.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:29 smithi170 ceph-mon[36438]: pgmap v121: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:29.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:29 smithi016 ceph-mon[31908]: pgmap v121: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:31.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:31 smithi170 ceph-mon[36438]: pgmap v122: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-27T06:47:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:31 smithi016 ceph-mon[31908]: pgmap v122: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-27T06:47:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:31 smithi156 ceph-mon[36261]: pgmap v122: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-27T06:47:33.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:33 smithi170 ceph-mon[36438]: pgmap v123: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:47:33.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:33 smithi016 ceph-mon[31908]: pgmap v123: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:47:33.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:33 smithi156 ceph-mon[36261]: pgmap v123: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:47:35.523 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:35 smithi156 ceph-mon[36261]: pgmap v124: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:35.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:35 smithi170 ceph-mon[36438]: pgmap v124: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:35 smithi016 ceph-mon[31908]: pgmap v124: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:37.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:37 smithi170 ceph-mon[36438]: pgmap v125: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:37 smithi016 ceph-mon[31908]: pgmap v125: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:37.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:37 smithi156 ceph-mon[36261]: pgmap v125: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:39.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:39 smithi170 ceph-mon[36438]: pgmap v126: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:39 smithi016 ceph-mon[31908]: pgmap v126: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:39.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:39 smithi156 ceph-mon[36261]: pgmap v126: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:41 smithi170 ceph-mon[36438]: pgmap v127: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:47:41.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:41 smithi016 ceph-mon[31908]: pgmap v127: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:47:41.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:41 smithi156 ceph-mon[36261]: pgmap v127: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:47:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:43 smithi170 ceph-mon[36438]: pgmap v128: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:47:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:43 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:47:43.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:43 smithi016 ceph-mon[31908]: pgmap v128: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:47:43.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:43 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:47:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:43 smithi156 ceph-mon[36261]: pgmap v128: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:47:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:43 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:47:44.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:44 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:47:44.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:44 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:47:44.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:44 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:47:44.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:44 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:47:44.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:44 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:47:44.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:44 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:47:44.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:44 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:47:44.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:44 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:47:44.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:44 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:47:45.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:45 smithi170 ceph-mon[36438]: pgmap v129: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:45.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:45 smithi016 ceph-mon[31908]: pgmap v129: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:45.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:45 smithi156 ceph-mon[36261]: pgmap v129: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:47.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:47 smithi170 ceph-mon[36438]: pgmap v130: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:47.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:47 smithi016 ceph-mon[31908]: pgmap v130: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:47 smithi156 ceph-mon[36261]: pgmap v130: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:49.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:49 smithi170 ceph-mon[36438]: pgmap v131: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:49 smithi016 ceph-mon[31908]: pgmap v131: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:49 smithi156 ceph-mon[36261]: pgmap v131: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:51.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:51 smithi170 ceph-mon[36438]: pgmap v132: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:47:51.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:51 smithi016 ceph-mon[31908]: pgmap v132: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:47:51.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:51 smithi156 ceph-mon[36261]: pgmap v132: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:47:53.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:53 smithi170 ceph-mon[36438]: pgmap v133: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:47:53.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:53 smithi016 ceph-mon[31908]: pgmap v133: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:47:53.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:53 smithi156 ceph-mon[36261]: pgmap v133: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:47:55.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:55 smithi170 ceph-mon[36438]: pgmap v134: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:55.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:55 smithi016 ceph-mon[31908]: pgmap v134: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:55.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:55 smithi156 ceph-mon[36261]: pgmap v134: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:57.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:57 smithi170 ceph-mon[36438]: pgmap v135: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:57.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:57 smithi016 ceph-mon[31908]: pgmap v135: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:57.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:57 smithi156 ceph-mon[36261]: pgmap v135: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:47:59.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:47:59 smithi170 ceph-mon[36438]: pgmap v136: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:59.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:47:59 smithi016 ceph-mon[31908]: pgmap v136: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:47:59.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:47:59 smithi156 ceph-mon[36261]: pgmap v136: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:01.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:01 smithi170 ceph-mon[36438]: pgmap v137: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:01.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:01 smithi016 ceph-mon[31908]: pgmap v137: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:01.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:01 smithi156 ceph-mon[36261]: pgmap v137: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:03.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:03 smithi170 ceph-mon[36438]: pgmap v138: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:03.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:03 smithi016 ceph-mon[31908]: pgmap v138: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:03 smithi156 ceph-mon[36261]: pgmap v138: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:05.523 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:05 smithi156 ceph-mon[36261]: pgmap v139: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:05.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:05 smithi170 ceph-mon[36438]: pgmap v139: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:05.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:05 smithi016 ceph-mon[31908]: pgmap v139: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:07.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:07 smithi170 ceph-mon[36438]: pgmap v140: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:48:07.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:07 smithi016 ceph-mon[31908]: pgmap v140: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:48:07.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:07 smithi156 ceph-mon[36261]: pgmap v140: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:48:09.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:09 smithi170 ceph-mon[36438]: pgmap v141: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:48:09.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:09 smithi016 ceph-mon[31908]: pgmap v141: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:48:09.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:09 smithi156 ceph-mon[36261]: pgmap v141: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:48:11.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:11 smithi170 ceph-mon[36438]: pgmap v142: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-27T06:48:11.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:11 smithi016 ceph-mon[31908]: pgmap v142: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-27T06:48:11.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:11 smithi156 ceph-mon[36261]: pgmap v142: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-27T06:48:13.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:13 smithi170 ceph-mon[36438]: pgmap v143: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-27T06:48:13.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:13 smithi016 ceph-mon[31908]: pgmap v143: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-27T06:48:13.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:13 smithi156 ceph-mon[36261]: pgmap v143: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-27T06:48:15.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:15 smithi170 ceph-mon[36438]: pgmap v144: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:48:15.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:15 smithi016 ceph-mon[31908]: pgmap v144: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:48:15.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:15 smithi156 ceph-mon[36261]: pgmap v144: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:48:17.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:17 smithi170 ceph-mon[36438]: pgmap v145: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:48:17.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:17 smithi016 ceph-mon[31908]: pgmap v145: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:48:17.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:17 smithi156 ceph-mon[36261]: pgmap v145: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:48:19.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:19 smithi170 ceph-mon[36438]: pgmap v146: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-27T06:48:19.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:19 smithi016 ceph-mon[31908]: pgmap v146: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-27T06:48:19.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:19 smithi156 ceph-mon[36261]: pgmap v146: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-27T06:48:21.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:21 smithi170 ceph-mon[36438]: pgmap v147: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-27T06:48:21.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:21 smithi016 ceph-mon[31908]: pgmap v147: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-27T06:48:21.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:21 smithi156 ceph-mon[36261]: pgmap v147: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-27T06:48:23.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:23 smithi170 ceph-mon[36438]: pgmap v148: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:23.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:23 smithi016 ceph-mon[31908]: pgmap v148: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:23.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:23 smithi156 ceph-mon[36261]: pgmap v148: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:25.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:25 smithi170 ceph-mon[36438]: pgmap v149: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:25.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:25 smithi016 ceph-mon[31908]: pgmap v149: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:25 smithi156 ceph-mon[36261]: pgmap v149: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:26.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:26 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:26.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:26 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:26.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:26 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:27.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:27 smithi170 ceph-mon[36438]: pgmap v150: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:27.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:27 smithi016 ceph-mon[31908]: pgmap v150: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:27 smithi156 ceph-mon[36261]: pgmap v150: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:29.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:29 smithi170 ceph-mon[36438]: pgmap v151: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:29.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:29 smithi016 ceph-mon[31908]: pgmap v151: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:29.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:29 smithi156 ceph-mon[36261]: pgmap v151: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:31.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:31 smithi170 ceph-mon[36438]: pgmap v152: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:31.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:31 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:31.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:31 smithi016 ceph-mon[31908]: pgmap v152: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:31.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:31 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:31 smithi156 ceph-mon[36261]: pgmap v152: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:31 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:33.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:33 smithi170 ceph-mon[36438]: pgmap v153: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:33.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:33 smithi016 ceph-mon[31908]: pgmap v153: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:33.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:33 smithi156 ceph-mon[36261]: pgmap v153: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:35.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:35 smithi170 ceph-mon[36438]: pgmap v154: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:35.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:35 smithi016 ceph-mon[31908]: pgmap v154: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:35.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:35 smithi156 ceph-mon[36261]: pgmap v154: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:37.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:37 smithi170 ceph-mon[36438]: pgmap v155: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:37.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:37 smithi016 ceph-mon[31908]: pgmap v155: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:37.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:37 smithi156 ceph-mon[36261]: pgmap v155: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:39.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:39 smithi170 ceph-mon[36438]: pgmap v156: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:39.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:39 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:39 smithi016 ceph-mon[31908]: pgmap v156: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:39.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:39 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:39.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:39 smithi156 ceph-mon[36261]: pgmap v156: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:39.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:39 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:40.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:40 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:40.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:40 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:40.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:40 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:41 smithi170 ceph-mon[36438]: pgmap v157: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:41 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:41.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:41 smithi016 ceph-mon[31908]: pgmap v157: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:41.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:41 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:41.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:41 smithi156 ceph-mon[36261]: pgmap v157: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:41.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:41 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:43.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:43 smithi170 ceph-mon[36438]: pgmap v158: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:43.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:43 smithi016 ceph-mon[31908]: pgmap v158: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:43.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:43 smithi156 ceph-mon[36261]: pgmap v158: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:44.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:44 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:48:44.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:44 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:48:44.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:44 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:48:45.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:45 smithi170 ceph-mon[36438]: pgmap v159: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:45.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:45 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:45.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:45 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:48:45.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:45 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:48:45.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:45 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:48:45.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:45 smithi016 ceph-mon[31908]: pgmap v159: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:45.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:45 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:45.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:45 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:48:45.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:45 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:48:45.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:45 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:48:45.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:45 smithi156 ceph-mon[36261]: pgmap v159: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:45.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:45 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:45.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:45 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:48:45.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:45 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:48:45.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:45 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:48:47.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:47 smithi016 ceph-mon[31908]: pgmap v160: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:47.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:47 smithi156 ceph-mon[36261]: pgmap v160: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:47 smithi170 ceph-mon[36438]: pgmap v160: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:49.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:49 smithi016 ceph-mon[31908]: pgmap v161: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:49.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:49 smithi156 ceph-mon[36261]: pgmap v161: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:49 smithi170 ceph-mon[36438]: pgmap v161: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:51.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:51 smithi016 ceph-mon[31908]: pgmap v162: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:51.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:51 smithi156 ceph-mon[36261]: pgmap v162: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:51.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:51 smithi170 ceph-mon[36438]: pgmap v162: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:48:53.611 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:53 smithi016 ceph-mon[31908]: pgmap v163: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:53.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:53 smithi156 ceph-mon[36261]: pgmap v163: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:53.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:53 smithi170 ceph-mon[36438]: pgmap v163: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:48:55.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:55 smithi016 ceph-mon[31908]: pgmap v164: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:55.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:55 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:55.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:55 smithi156 ceph-mon[36261]: pgmap v164: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:55.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:55 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:55.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:55 smithi170 ceph-mon[36438]: pgmap v164: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:55.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:55 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:57.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:57 smithi016 ceph-mon[31908]: pgmap v165: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:57.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:57 smithi156 ceph-mon[36261]: pgmap v165: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:57.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:57 smithi170 ceph-mon[36438]: pgmap v165: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:48:59.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:59 smithi156 ceph-mon[36261]: pgmap v166: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:59.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:48:59 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:59.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:59 smithi170 ceph-mon[36438]: pgmap v166: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:59.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:48:59 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:48:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:59 smithi016 ceph-mon[31908]: pgmap v166: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:48:59.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:48:59 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:01.723 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:01 smithi156 ceph-mon[36261]: pgmap v167: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:49:01.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:01 smithi170 ceph-mon[36438]: pgmap v167: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:49:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:01 smithi016 ceph-mon[31908]: pgmap v167: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-27T06:49:02.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:02 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:02.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:02 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:02.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:02 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:03.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:03 smithi156 ceph-mon[36261]: pgmap v168: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:49:03.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:03 smithi170 ceph-mon[36438]: pgmap v168: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:49:03.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:03 smithi016 ceph-mon[31908]: pgmap v168: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-27T06:49:05.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:05 smithi170 ceph-mon[36438]: pgmap v169: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:05.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:05 smithi156 ceph-mon[36261]: pgmap v169: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:05.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:05 smithi016 ceph-mon[31908]: pgmap v169: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:07.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:07 smithi156 ceph-mon[36261]: pgmap v170: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:49:07.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:07 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:07.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:07 smithi170 ceph-mon[36438]: pgmap v170: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:49:07.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:07 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:07.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:07 smithi016 ceph-mon[31908]: pgmap v170: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:49:07.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:07 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:09.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:09 smithi156 ceph-mon[36261]: pgmap v171: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:49:09.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:09 smithi170 ceph-mon[36438]: pgmap v171: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:49:09.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:09 smithi016 ceph-mon[31908]: pgmap v171: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:49:11.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:11 smithi156 ceph-mon[36261]: pgmap v172: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:49:11.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:11 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:11.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:11 smithi170 ceph-mon[36438]: pgmap v172: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:49:11.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:11 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:11.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:11 smithi016 ceph-mon[31908]: pgmap v172: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:49:11.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:11 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:12.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:12 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:12.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:12 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:12.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:12 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:13.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:13 smithi156 ceph-mon[36261]: pgmap v173: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-27T06:49:13.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:13 smithi170 ceph-mon[36438]: pgmap v173: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-27T06:49:13.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:13 smithi016 ceph-mon[31908]: pgmap v173: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-27T06:49:14.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:14 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:14.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:14 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:14.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:14 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:15.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:15 smithi156 ceph-mon[36261]: pgmap v174: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:49:15.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:15 smithi170 ceph-mon[36438]: pgmap v174: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:49:15.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:15 smithi016 ceph-mon[31908]: pgmap v174: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-27T06:49:16.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:16 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:16.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:16 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:16.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:16 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:17.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:17 smithi156 ceph-mon[36261]: pgmap v175: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:49:17.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:17 smithi170 ceph-mon[36438]: pgmap v175: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:49:17.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:17 smithi016 ceph-mon[31908]: pgmap v175: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-27T06:49:19.725 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:19 smithi156 ceph-mon[36261]: pgmap v176: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-27T06:49:19.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:19 smithi170 ceph-mon[36438]: pgmap v176: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-27T06:49:19.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:19 smithi016 ceph-mon[31908]: pgmap v176: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-27T06:49:21.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:21 smithi156 ceph-mon[36261]: pgmap v177: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-27T06:49:21.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:21 smithi170 ceph-mon[36438]: pgmap v177: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-27T06:49:21.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:21 smithi016 ceph-mon[31908]: pgmap v177: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-27T06:49:23.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:23 smithi156 ceph-mon[36261]: pgmap v178: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:23 smithi170 ceph-mon[36438]: pgmap v178: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:23.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:23 smithi016 ceph-mon[31908]: pgmap v178: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:25.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:25 smithi156 ceph-mon[36261]: pgmap v179: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:25 smithi170 ceph-mon[36438]: pgmap v179: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:25.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:25 smithi016 ceph-mon[31908]: pgmap v179: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:27.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:27 smithi156 ceph-mon[36261]: pgmap v180: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:27 smithi170 ceph-mon[36438]: pgmap v180: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:27.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:27 smithi016 ceph-mon[31908]: pgmap v180: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:29.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:29 smithi156 ceph-mon[36261]: pgmap v181: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:29 smithi170 ceph-mon[36438]: pgmap v181: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:29 smithi016 ceph-mon[31908]: pgmap v181: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:30.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:30 smithi156 ceph-mon[36261]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:30.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:30 smithi170 ceph-mon[36438]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:30.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:30 smithi016 ceph-mon[31908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-27T06:49:31.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:31 smithi170 ceph-mon[36438]: pgmap v182: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:31 smithi016 ceph-mon[31908]: pgmap v182: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:31.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:31 smithi156 ceph-mon[36261]: pgmap v182: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:33.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:33 smithi170 ceph-mon[36438]: pgmap v183: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:33.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:33 smithi016 ceph-mon[31908]: pgmap v183: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:33 smithi156 ceph-mon[36261]: pgmap v183: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:35 smithi170 ceph-mon[36438]: pgmap v184: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:35.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:35 smithi016 ceph-mon[31908]: pgmap v184: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:35 smithi156 ceph-mon[36261]: pgmap v184: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:37.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:37 smithi170 ceph-mon[36438]: pgmap v185: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:37.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:37 smithi016 ceph-mon[31908]: pgmap v185: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:37.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:37 smithi156 ceph-mon[36261]: pgmap v185: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:39.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:39 smithi170 ceph-mon[36438]: pgmap v186: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:39.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:39 smithi016 ceph-mon[31908]: pgmap v186: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:39.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:39 smithi156 ceph-mon[36261]: pgmap v186: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:41.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:41 smithi170 ceph-mon[36438]: pgmap v187: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:41.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:41 smithi016 ceph-mon[31908]: pgmap v187: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:41.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:41 smithi156 ceph-mon[36261]: pgmap v187: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:43.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:43 smithi170 ceph-mon[36438]: pgmap v188: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:43 smithi016 ceph-mon[31908]: pgmap v188: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:43.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:43 smithi156 ceph-mon[36261]: pgmap v188: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:45.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:45 smithi170 ceph-mon[36438]: pgmap v189: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:45.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:45 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:49:45.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:45 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:49:45.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:45 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:49:45.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:45 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:49:45.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:45 smithi016 ceph-mon[31908]: pgmap v189: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:45.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:45 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:49:45.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:45 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:49:45.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:45 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:49:45.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:45 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:49:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:45 smithi156 ceph-mon[36261]: pgmap v189: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:45 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:49:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:45 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:49:45.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:45 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:49:45.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:45 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:49:47.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:47 smithi170 ceph-mon[36438]: pgmap v190: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:47 smithi016 ceph-mon[31908]: pgmap v190: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:47.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:47 smithi156 ceph-mon[36261]: pgmap v190: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:49.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:49 smithi170 ceph-mon[36438]: pgmap v191: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:49.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:49 smithi016 ceph-mon[31908]: pgmap v191: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:49.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:49 smithi156 ceph-mon[36261]: pgmap v191: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:51.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:51 smithi170 ceph-mon[36438]: pgmap v192: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:51.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:51 smithi016 ceph-mon[31908]: pgmap v192: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:51.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:51 smithi156 ceph-mon[36261]: pgmap v192: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:53.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:53 smithi170 ceph-mon[36438]: pgmap v193: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:53.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:53 smithi016 ceph-mon[31908]: pgmap v193: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:53.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:53 smithi156 ceph-mon[36261]: pgmap v193: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:55.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:55 smithi170 ceph-mon[36438]: pgmap v194: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:55.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:55 smithi016 ceph-mon[31908]: pgmap v194: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:55.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:55 smithi156 ceph-mon[36261]: pgmap v194: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:57.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:57 smithi170 ceph-mon[36438]: pgmap v195: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:57.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:57 smithi016 ceph-mon[31908]: pgmap v195: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:57.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:57 smithi156 ceph-mon[36261]: pgmap v195: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:49:59.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:49:59 smithi170 ceph-mon[36438]: pgmap v196: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:59.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:49:59 smithi016 ceph-mon[31908]: pgmap v196: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:49:59.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:49:59 smithi156 ceph-mon[36261]: pgmap v196: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:00.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31904]: 2024-03-27T06:49:59.999+0000 7f6c4124d700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 4 MDSs report damaged metadata 2024-03-27T06:50:00.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31904]: 2024-03-27T06:49:59.999+0000 7f6c4124d700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 4 MDSs report damaged metadata 2024-03-27T06:50:00.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31904]: 2024-03-27T06:49:59.999+0000 7f6c4124d700 -1 log_channel(cluster) log [ERR] : mds.e(mds.4): Metadata damage detected 2024-03-27T06:50:00.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31904]: 2024-03-27T06:49:59.999+0000 7f6c4124d700 -1 log_channel(cluster) log [ERR] : mds.c(mds.3): Metadata damage detected 2024-03-27T06:50:00.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31904]: 2024-03-27T06:49:59.999+0000 7f6c4124d700 -1 log_channel(cluster) log [ERR] : mds.i(mds.2): Metadata damage detected 2024-03-27T06:50:00.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31904]: 2024-03-27T06:49:59.999+0000 7f6c4124d700 -1 log_channel(cluster) log [ERR] : mds.h(mds.1): Metadata damage detected 2024-03-27T06:50:00.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-mon[31908]: Health detail: HEALTH_ERR 4 MDSs report damaged metadata 2024-03-27T06:50:00.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-mon[31908]: [ERR] MDS_DAMAGE: 4 MDSs report damaged metadata 2024-03-27T06:50:00.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-mon[31908]: mds.e(mds.4): Metadata damage detected 2024-03-27T06:50:00.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-mon[31908]: mds.c(mds.3): Metadata damage detected 2024-03-27T06:50:00.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-mon[31908]: mds.i(mds.2): Metadata damage detected 2024-03-27T06:50:00.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:00 smithi016 ceph-mon[31908]: mds.h(mds.1): Metadata damage detected 2024-03-27T06:50:00.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:00 smithi156 ceph-mon[36261]: Health detail: HEALTH_ERR 4 MDSs report damaged metadata 2024-03-27T06:50:00.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:00 smithi156 ceph-mon[36261]: [ERR] MDS_DAMAGE: 4 MDSs report damaged metadata 2024-03-27T06:50:00.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:00 smithi156 ceph-mon[36261]: mds.e(mds.4): Metadata damage detected 2024-03-27T06:50:00.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:00 smithi156 ceph-mon[36261]: mds.c(mds.3): Metadata damage detected 2024-03-27T06:50:00.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:00 smithi156 ceph-mon[36261]: mds.i(mds.2): Metadata damage detected 2024-03-27T06:50:00.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:00 smithi156 ceph-mon[36261]: mds.h(mds.1): Metadata damage detected 2024-03-27T06:50:01.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:00 smithi170 ceph-mon[36438]: Health detail: HEALTH_ERR 4 MDSs report damaged metadata 2024-03-27T06:50:01.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:00 smithi170 ceph-mon[36438]: [ERR] MDS_DAMAGE: 4 MDSs report damaged metadata 2024-03-27T06:50:01.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:00 smithi170 ceph-mon[36438]: mds.e(mds.4): Metadata damage detected 2024-03-27T06:50:01.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:00 smithi170 ceph-mon[36438]: mds.c(mds.3): Metadata damage detected 2024-03-27T06:50:01.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:00 smithi170 ceph-mon[36438]: mds.i(mds.2): Metadata damage detected 2024-03-27T06:50:01.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:00 smithi170 ceph-mon[36438]: mds.h(mds.1): Metadata damage detected 2024-03-27T06:50:01.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:01 smithi016 ceph-mon[31908]: pgmap v197: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:01.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:01 smithi156 ceph-mon[36261]: pgmap v197: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:02.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:01 smithi170 ceph-mon[36438]: pgmap v197: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:03.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:03 smithi016 ceph-mon[31908]: pgmap v198: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:03.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:03 smithi156 ceph-mon[36261]: pgmap v198: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:04.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:03 smithi170 ceph-mon[36438]: pgmap v198: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:05.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:05 smithi016 ceph-mon[31908]: pgmap v199: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:05.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:05 smithi170 ceph-mon[36438]: pgmap v199: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:05.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:05 smithi156 ceph-mon[36261]: pgmap v199: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:07.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:07 smithi016 ceph-mon[31908]: pgmap v200: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-03-27T06:50:07.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:07 smithi156 ceph-mon[36261]: pgmap v200: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-03-27T06:50:08.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:07 smithi170 ceph-mon[36438]: pgmap v200: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-03-27T06:50:09.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:09 smithi016 ceph-mon[31908]: pgmap v201: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-03-27T06:50:09.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:09 smithi156 ceph-mon[36261]: pgmap v201: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-03-27T06:50:10.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:09 smithi170 ceph-mon[36438]: pgmap v201: 97 pgs: 97 active+clean; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-03-27T06:50:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:10 smithi156 ceph-mon[36261]: osdmap e91: 12 total, 12 up, 12 in 2024-03-27T06:50:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:10 smithi170 ceph-mon[36438]: osdmap e91: 12 total, 12 up, 12 in 2024-03-27T06:50:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:10 smithi016 ceph-mon[31908]: osdmap e91: 12 total, 12 up, 12 in 2024-03-27T06:50:11.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:11 smithi156 ceph-mon[36261]: pgmap v203: 97 pgs: 19 active+clean+snaptrim, 48 active+clean, 30 active+clean+snaptrim_wait; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-27T06:50:12.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:11 smithi170 ceph-mon[36438]: pgmap v203: 97 pgs: 19 active+clean+snaptrim, 48 active+clean, 30 active+clean+snaptrim_wait; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-27T06:50:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:11 smithi016 ceph-mon[31908]: pgmap v203: 97 pgs: 19 active+clean+snaptrim, 48 active+clean, 30 active+clean+snaptrim_wait; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-27T06:50:13.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:13 smithi156 ceph-mon[36261]: pgmap v204: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-27T06:50:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:13 smithi170 ceph-mon[36438]: pgmap v204: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-27T06:50:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:13 smithi016 ceph-mon[31908]: pgmap v204: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-27T06:50:15.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:15 smithi156 ceph-mon[36261]: pgmap v205: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-27T06:50:16.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:15 smithi170 ceph-mon[36438]: pgmap v205: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-27T06:50:16.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:15 smithi016 ceph-mon[31908]: pgmap v205: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 160 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-27T06:50:17.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:17 smithi156 ceph-mon[36261]: pgmap v206: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:50:18.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:17 smithi170 ceph-mon[36438]: pgmap v206: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:50:18.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:17 smithi016 ceph-mon[31908]: pgmap v206: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:50:19.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:19 smithi016 ceph-mon[31908]: pgmap v207: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:50:19.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:19 smithi156 ceph-mon[36261]: pgmap v207: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:50:20.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:19 smithi170 ceph-mon[36438]: pgmap v207: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:50:21.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:21 smithi170 ceph-mon[36438]: pgmap v208: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 199 B/s wr, 14 op/s 2024-03-27T06:50:21.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:21 smithi170 ceph-mon[36438]: osdmap e92: 12 total, 12 up, 12 in 2024-03-27T06:50:21.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:21 smithi016 ceph-mon[31908]: pgmap v208: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 199 B/s wr, 14 op/s 2024-03-27T06:50:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:21 smithi016 ceph-mon[31908]: osdmap e92: 12 total, 12 up, 12 in 2024-03-27T06:50:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:21 smithi156 ceph-mon[36261]: pgmap v208: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 199 B/s wr, 14 op/s 2024-03-27T06:50:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:21 smithi156 ceph-mon[36261]: osdmap e92: 12 total, 12 up, 12 in 2024-03-27T06:50:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:23 smithi170 ceph-mon[36438]: pgmap v210: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:50:23.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:23 smithi016 ceph-mon[31908]: pgmap v210: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:50:23.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:23 smithi156 ceph-mon[36261]: pgmap v210: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:50:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:25 smithi170 ceph-mon[36438]: pgmap v211: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:50:25.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:25 smithi016 ceph-mon[31908]: pgmap v211: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:50:25.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:25 smithi156 ceph-mon[36261]: pgmap v211: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:50:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:27 smithi170 ceph-mon[36438]: pgmap v212: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-27T06:50:27.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:27 smithi016 ceph-mon[31908]: pgmap v212: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-27T06:50:27.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:27 smithi156 ceph-mon[36261]: pgmap v212: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-27T06:50:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:29 smithi170 ceph-mon[36438]: pgmap v213: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-27T06:50:29.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:29 smithi016 ceph-mon[31908]: pgmap v213: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-27T06:50:29.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:29 smithi156 ceph-mon[36261]: pgmap v213: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-27T06:50:31.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:31 smithi170 ceph-mon[36438]: pgmap v214: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-27T06:50:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:31 smithi016 ceph-mon[31908]: pgmap v214: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-27T06:50:31.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:31 smithi156 ceph-mon[36261]: pgmap v214: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-27T06:50:33.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:33 smithi170 ceph-mon[36438]: pgmap v215: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-27T06:50:33.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:33 smithi016 ceph-mon[31908]: pgmap v215: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-27T06:50:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:33 smithi156 ceph-mon[36261]: pgmap v215: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-27T06:50:35.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:35 smithi170 ceph-mon[36438]: pgmap v216: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T06:50:35.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:35 smithi016 ceph-mon[31908]: pgmap v216: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T06:50:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:35 smithi156 ceph-mon[36261]: pgmap v216: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T06:50:37.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:37 smithi170 ceph-mon[36438]: pgmap v217: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:37.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:37 smithi016 ceph-mon[31908]: pgmap v217: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:37.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:37 smithi156 ceph-mon[36261]: pgmap v217: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:39.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:39 smithi170 ceph-mon[36438]: pgmap v218: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:39.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:39 smithi016 ceph-mon[31908]: pgmap v218: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:39.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:39 smithi156 ceph-mon[36261]: pgmap v218: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:41.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:41 smithi170 ceph-mon[36438]: pgmap v219: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:41.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:41 smithi016 ceph-mon[31908]: pgmap v219: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:41.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:41 smithi156 ceph-mon[36261]: pgmap v219: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:43 smithi016 ceph-mon[31908]: pgmap v220: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:43.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:43 smithi156 ceph-mon[36261]: pgmap v220: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:44.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:43 smithi170 ceph-mon[36438]: pgmap v220: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:45.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:45 smithi016 ceph-mon[31908]: pgmap v221: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:45.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:45 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:50:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:45 smithi156 ceph-mon[36261]: pgmap v221: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:45 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:50:46.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:45 smithi170 ceph-mon[36438]: pgmap v221: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:46.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:45 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:50:46.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:46 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:50:46.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:46 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:50:46.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:46 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:50:46.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:46 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:50:46.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:46 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:50:46.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:46 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:50:47.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:46 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:50:47.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:46 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:50:47.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:46 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:50:47.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:47 smithi016 ceph-mon[31908]: pgmap v222: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:47.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:47 smithi156 ceph-mon[36261]: pgmap v222: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:48.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:47 smithi170 ceph-mon[36438]: pgmap v222: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:49.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:49 smithi016 ceph-mon[31908]: pgmap v223: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:49.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:49 smithi156 ceph-mon[36261]: pgmap v223: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:50.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:49 smithi170 ceph-mon[36438]: pgmap v223: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:51.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:51 smithi016 ceph-mon[31908]: pgmap v224: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:51.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:51 smithi156 ceph-mon[36261]: pgmap v224: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:52.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:51 smithi170 ceph-mon[36438]: pgmap v224: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:53.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:53 smithi016 ceph-mon[31908]: pgmap v225: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:53.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:53 smithi156 ceph-mon[36261]: pgmap v225: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:53 smithi170 ceph-mon[36438]: pgmap v225: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:55.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:55 smithi156 ceph-mon[36261]: pgmap v226: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:56.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:55 smithi170 ceph-mon[36438]: pgmap v226: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:56.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:55 smithi016 ceph-mon[31908]: pgmap v226: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:50:57.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:57 smithi156 ceph-mon[36261]: pgmap v227: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:58.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:57 smithi170 ceph-mon[36438]: pgmap v227: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:58.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:57 smithi016 ceph-mon[31908]: pgmap v227: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:50:59.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:50:59 smithi156 ceph-mon[36261]: pgmap v228: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:00.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:50:59 smithi170 ceph-mon[36438]: pgmap v228: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:00.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:50:59 smithi016 ceph-mon[31908]: pgmap v228: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:01.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:01 smithi156 ceph-mon[36261]: pgmap v229: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:02.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:01 smithi170 ceph-mon[36438]: pgmap v229: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:02.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:01 smithi016 ceph-mon[31908]: pgmap v229: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:03.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:03 smithi156 ceph-mon[36261]: pgmap v230: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:04.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:03 smithi170 ceph-mon[36438]: pgmap v230: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:04.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:03 smithi016 ceph-mon[31908]: pgmap v230: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:05.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:05 smithi170 ceph-mon[36438]: pgmap v231: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:05.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:05 smithi156 ceph-mon[36261]: pgmap v231: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:06.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:05 smithi016 ceph-mon[31908]: pgmap v231: 97 pgs: 97 active+clean; 160 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:07.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:07 smithi156 ceph-mon[36261]: pgmap v232: 97 pgs: 97 active+clean; 160 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:51:08.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:07 smithi170 ceph-mon[36438]: pgmap v232: 97 pgs: 97 active+clean; 160 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:51:08.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:07 smithi016 ceph-mon[31908]: pgmap v232: 97 pgs: 97 active+clean; 160 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:51:09.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:09 smithi156 ceph-mon[36261]: pgmap v233: 97 pgs: 97 active+clean; 160 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:51:10.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:09 smithi170 ceph-mon[36438]: pgmap v233: 97 pgs: 97 active+clean; 160 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:51:10.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:09 smithi016 ceph-mon[31908]: pgmap v233: 97 pgs: 97 active+clean; 160 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:51:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:10 smithi156 ceph-mon[36261]: osdmap e93: 12 total, 12 up, 12 in 2024-03-27T06:51:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:10 smithi170 ceph-mon[36438]: osdmap e93: 12 total, 12 up, 12 in 2024-03-27T06:51:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:10 smithi016 ceph-mon[31908]: osdmap e93: 12 total, 12 up, 12 in 2024-03-27T06:51:11.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:11 smithi156 ceph-mon[36261]: pgmap v235: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 160 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-03-27T06:51:12.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:11 smithi170 ceph-mon[36438]: pgmap v235: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 160 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-03-27T06:51:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:11 smithi016 ceph-mon[31908]: pgmap v235: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 160 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-03-27T06:51:13.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:13 smithi156 ceph-mon[36261]: pgmap v236: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 160 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:51:14.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:13 smithi170 ceph-mon[36438]: pgmap v236: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 160 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:51:14.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:13 smithi016 ceph-mon[31908]: pgmap v236: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 160 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:51:14.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:14 smithi156 ceph-mon[36261]: pgmap v237: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 160 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:51:15.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:14 smithi170 ceph-mon[36438]: pgmap v237: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 160 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:51:15.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:14 smithi016 ceph-mon[31908]: pgmap v237: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 160 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:51:17.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:16 smithi156 ceph-mon[36261]: pgmap v238: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:51:17.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:16 smithi170 ceph-mon[36438]: pgmap v238: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:51:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:16 smithi016 ceph-mon[31908]: pgmap v238: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:51:19.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:18 smithi170 ceph-mon[36438]: pgmap v239: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:51:19.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:18 smithi016 ceph-mon[31908]: pgmap v239: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:51:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:18 smithi156 ceph-mon[36261]: pgmap v239: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:51:21.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:21 smithi170 ceph-mon[36438]: pgmap v240: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 199 B/s wr, 13 op/s 2024-03-27T06:51:21.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:21 smithi170 ceph-mon[36438]: osdmap e94: 12 total, 12 up, 12 in 2024-03-27T06:51:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:21 smithi016 ceph-mon[31908]: pgmap v240: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 199 B/s wr, 13 op/s 2024-03-27T06:51:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:21 smithi016 ceph-mon[31908]: osdmap e94: 12 total, 12 up, 12 in 2024-03-27T06:51:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:21 smithi156 ceph-mon[36261]: pgmap v240: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 199 B/s wr, 13 op/s 2024-03-27T06:51:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:21 smithi156 ceph-mon[36261]: osdmap e94: 12 total, 12 up, 12 in 2024-03-27T06:51:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:23 smithi170 ceph-mon[36438]: pgmap v242: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:51:23.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:23 smithi016 ceph-mon[31908]: pgmap v242: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:51:23.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:23 smithi156 ceph-mon[36261]: pgmap v242: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:51:25.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:25 smithi170 ceph-mon[36438]: pgmap v243: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:51:25.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:25 smithi016 ceph-mon[31908]: pgmap v243: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:51:25.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:25 smithi156 ceph-mon[36261]: pgmap v243: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:51:27.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:27 smithi170 ceph-mon[36438]: pgmap v244: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:51:27.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:27 smithi016 ceph-mon[31908]: pgmap v244: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:51:27.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:27 smithi156 ceph-mon[36261]: pgmap v244: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:51:29.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:29 smithi170 ceph-mon[36438]: pgmap v245: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:51:29.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:29 smithi016 ceph-mon[31908]: pgmap v245: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:51:29.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:29 smithi156 ceph-mon[36261]: pgmap v245: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:51:31.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:31 smithi170 ceph-mon[36438]: pgmap v246: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:51:31.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:31 smithi016 ceph-mon[31908]: pgmap v246: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:51:31.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:31 smithi156 ceph-mon[36261]: pgmap v246: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:51:33.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:33 smithi016 ceph-mon[31908]: pgmap v247: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T06:51:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:33 smithi156 ceph-mon[36261]: pgmap v247: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T06:51:34.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:33 smithi170 ceph-mon[36438]: pgmap v247: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T06:51:35.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:35 smithi016 ceph-mon[31908]: pgmap v248: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:35.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:35 smithi170 ceph-mon[36438]: pgmap v248: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:35 smithi156 ceph-mon[36261]: pgmap v248: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:37.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:37 smithi016 ceph-mon[31908]: pgmap v249: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:37.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:37 smithi156 ceph-mon[36261]: pgmap v249: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:38.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:37 smithi170 ceph-mon[36438]: pgmap v249: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:39.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:39 smithi016 ceph-mon[31908]: pgmap v250: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:39.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:39 smithi156 ceph-mon[36261]: pgmap v250: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:40.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:39 smithi170 ceph-mon[36438]: pgmap v250: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:41.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:41 smithi016 ceph-mon[31908]: pgmap v251: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:51:41.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:41 smithi156 ceph-mon[36261]: pgmap v251: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:51:42.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:41 smithi170 ceph-mon[36438]: pgmap v251: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:51:43.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:43 smithi016 ceph-mon[31908]: pgmap v252: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:51:43.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:43 smithi156 ceph-mon[36261]: pgmap v252: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:51:44.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:43 smithi170 ceph-mon[36438]: pgmap v252: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:51:45.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:45 smithi016 ceph-mon[31908]: pgmap v253: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:45 smithi156 ceph-mon[36261]: pgmap v253: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:46.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:45 smithi170 ceph-mon[36438]: pgmap v253: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:46.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:46 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:51:47.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:46 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:51:47.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:46 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:51:47.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:47 smithi156 ceph-mon[36261]: pgmap v254: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:47.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:47 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:51:47.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:47 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:51:47.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:47 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:51:48.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:47 smithi170 ceph-mon[36438]: pgmap v254: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:48.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:47 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:51:48.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:47 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:51:48.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:47 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:51:48.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:47 smithi016 ceph-mon[31908]: pgmap v254: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:48.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:47 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:51:48.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:47 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:51:48.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:47 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:51:49.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:49 smithi016 ceph-mon[31908]: pgmap v255: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:49.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:49 smithi156 ceph-mon[36261]: pgmap v255: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:50.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:49 smithi170 ceph-mon[36438]: pgmap v255: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:51.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:51 smithi156 ceph-mon[36261]: pgmap v256: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:51:52.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:51 smithi170 ceph-mon[36438]: pgmap v256: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:51:52.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:51 smithi016 ceph-mon[31908]: pgmap v256: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:51:53.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:53 smithi156 ceph-mon[36261]: pgmap v257: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:51:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:53 smithi170 ceph-mon[36438]: pgmap v257: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:51:54.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:53 smithi016 ceph-mon[31908]: pgmap v257: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:51:55.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:55 smithi156 ceph-mon[36261]: pgmap v258: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:56.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:55 smithi170 ceph-mon[36438]: pgmap v258: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:56.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:55 smithi016 ceph-mon[31908]: pgmap v258: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:51:57.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:57 smithi156 ceph-mon[36261]: pgmap v259: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:58.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:57 smithi170 ceph-mon[36438]: pgmap v259: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:58.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:57 smithi016 ceph-mon[31908]: pgmap v259: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:51:59.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:51:59 smithi156 ceph-mon[36261]: pgmap v260: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:00.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:51:59 smithi170 ceph-mon[36438]: pgmap v260: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:51:59 smithi016 ceph-mon[31908]: pgmap v260: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:01.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:01 smithi156 ceph-mon[36261]: pgmap v261: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:52:02.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:01 smithi170 ceph-mon[36438]: pgmap v261: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:52:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:01 smithi016 ceph-mon[31908]: pgmap v261: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:52:03.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:03 smithi156 ceph-mon[36261]: pgmap v262: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:52:04.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:03 smithi170 ceph-mon[36438]: pgmap v262: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:52:04.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:03 smithi016 ceph-mon[31908]: pgmap v262: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:52:05.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:05 smithi170 ceph-mon[36438]: pgmap v263: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:05.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:05 smithi156 ceph-mon[36261]: pgmap v263: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:06.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:05 smithi016 ceph-mon[31908]: pgmap v263: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:07.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:07 smithi156 ceph-mon[36261]: pgmap v264: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:52:08.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:07 smithi170 ceph-mon[36438]: pgmap v264: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:52:08.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:07 smithi016 ceph-mon[31908]: pgmap v264: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:52:09.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:09 smithi156 ceph-mon[36261]: pgmap v265: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:52:10.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:09 smithi170 ceph-mon[36438]: pgmap v265: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:52:10.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:09 smithi016 ceph-mon[31908]: pgmap v265: 97 pgs: 97 active+clean; 160 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:52:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:10 smithi156 ceph-mon[36261]: osdmap e95: 12 total, 12 up, 12 in 2024-03-27T06:52:10.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:10 smithi156 ceph-mon[36261]: pgmap v267: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 128 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:52:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:10 smithi170 ceph-mon[36438]: osdmap e95: 12 total, 12 up, 12 in 2024-03-27T06:52:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:10 smithi170 ceph-mon[36438]: pgmap v267: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 128 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:52:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:10 smithi016 ceph-mon[31908]: osdmap e95: 12 total, 12 up, 12 in 2024-03-27T06:52:11.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:10 smithi016 ceph-mon[31908]: pgmap v267: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 128 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:52:13.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:12 smithi170 ceph-mon[36438]: pgmap v268: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 123 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:52:13.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:12 smithi016 ceph-mon[31908]: pgmap v268: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 123 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:52:13.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:12 smithi156 ceph-mon[36261]: pgmap v268: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 123 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:52:15.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:14 smithi170 ceph-mon[36438]: pgmap v269: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 123 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:52:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:14 smithi016 ceph-mon[31908]: pgmap v269: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 123 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:52:15.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:14 smithi156 ceph-mon[36261]: pgmap v269: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 123 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:52:17.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:16 smithi170 ceph-mon[36438]: pgmap v270: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:52:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:16 smithi016 ceph-mon[31908]: pgmap v270: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:52:17.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:16 smithi156 ceph-mon[36261]: pgmap v270: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:52:19.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:19 smithi170 ceph-mon[36438]: pgmap v271: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:52:19.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:19 smithi016 ceph-mon[31908]: pgmap v271: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:52:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:19 smithi156 ceph-mon[36261]: pgmap v271: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:52:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:21 smithi016 ceph-mon[31908]: pgmap v272: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 199 B/s wr, 13 op/s 2024-03-27T06:52:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:21 smithi016 ceph-mon[31908]: osdmap e96: 12 total, 12 up, 12 in 2024-03-27T06:52:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:21 smithi156 ceph-mon[36261]: pgmap v272: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 199 B/s wr, 13 op/s 2024-03-27T06:52:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:21 smithi156 ceph-mon[36261]: osdmap e96: 12 total, 12 up, 12 in 2024-03-27T06:52:22.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:21 smithi170 ceph-mon[36438]: pgmap v272: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 199 B/s wr, 13 op/s 2024-03-27T06:52:22.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:21 smithi170 ceph-mon[36438]: osdmap e96: 12 total, 12 up, 12 in 2024-03-27T06:52:23.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:23 smithi016 ceph-mon[31908]: pgmap v274: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:52:23.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:23 smithi156 ceph-mon[36261]: pgmap v274: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:52:24.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:23 smithi170 ceph-mon[36438]: pgmap v274: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:52:25.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:25 smithi016 ceph-mon[31908]: pgmap v275: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:52:25.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:25 smithi156 ceph-mon[36261]: pgmap v275: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:52:26.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:25 smithi170 ceph-mon[36438]: pgmap v275: 97 pgs: 97 active+clean; 42 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:52:27.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:27 smithi016 ceph-mon[31908]: pgmap v276: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:52:27.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:27 smithi156 ceph-mon[36261]: pgmap v276: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:52:28.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:27 smithi170 ceph-mon[36438]: pgmap v276: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:52:29.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:29 smithi016 ceph-mon[31908]: pgmap v277: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:52:29.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:29 smithi156 ceph-mon[36261]: pgmap v277: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:52:30.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:29 smithi170 ceph-mon[36438]: pgmap v277: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:52:31.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:31 smithi016 ceph-mon[31908]: pgmap v278: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:52:31.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:31 smithi156 ceph-mon[36261]: pgmap v278: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:52:32.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:31 smithi170 ceph-mon[36438]: pgmap v278: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:52:33.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:33 smithi016 ceph-mon[31908]: pgmap v279: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T06:52:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:33 smithi156 ceph-mon[36261]: pgmap v279: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T06:52:34.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:33 smithi170 ceph-mon[36438]: pgmap v279: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T06:52:35.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:35 smithi170 ceph-mon[36438]: pgmap v280: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:35 smithi156 ceph-mon[36261]: pgmap v280: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:36.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:35 smithi016 ceph-mon[31908]: pgmap v280: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:37.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:37 smithi156 ceph-mon[36261]: pgmap v281: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:52:38.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:37 smithi170 ceph-mon[36438]: pgmap v281: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:52:38.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:37 smithi016 ceph-mon[31908]: pgmap v281: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:52:39.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:39 smithi156 ceph-mon[36261]: pgmap v282: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:40.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:39 smithi170 ceph-mon[36438]: pgmap v282: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:40.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:39 smithi016 ceph-mon[31908]: pgmap v282: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:41.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:41 smithi156 ceph-mon[36261]: pgmap v283: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:52:42.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:41 smithi170 ceph-mon[36438]: pgmap v283: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:52:42.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:41 smithi016 ceph-mon[31908]: pgmap v283: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:52:43.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:43 smithi156 ceph-mon[36261]: pgmap v284: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:52:44.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:43 smithi170 ceph-mon[36438]: pgmap v284: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:52:44.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:43 smithi016 ceph-mon[31908]: pgmap v284: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-27T06:52:45.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:45 smithi156 ceph-mon[36261]: pgmap v285: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:46.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:45 smithi170 ceph-mon[36438]: pgmap v285: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:46.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:45 smithi016 ceph-mon[31908]: pgmap v285: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:47.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:47 smithi156 ceph-mon[36261]: pgmap v286: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:52:47.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:47 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:52:48.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:47 smithi170 ceph-mon[36438]: pgmap v286: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:52:48.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:47 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:52:48.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:47 smithi016 ceph-mon[31908]: pgmap v286: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:52:48.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:47 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:52:48.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:48 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:52:48.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:48 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:52:48.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:48 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:52:49.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:48 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:52:49.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:48 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:52:49.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:48 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:52:49.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:48 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:52:49.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:48 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:52:49.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:48 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:52:49.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:49 smithi156 ceph-mon[36261]: pgmap v287: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:50.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:49 smithi170 ceph-mon[36438]: pgmap v287: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:50.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:49 smithi016 ceph-mon[31908]: pgmap v287: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:52:51.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:51 smithi156 ceph-mon[36261]: pgmap v288: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:52:52.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:51 smithi170 ceph-mon[36438]: pgmap v288: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:52:52.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:51 smithi016 ceph-mon[31908]: pgmap v288: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-03-27T06:52:53.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:53 smithi156 ceph-mon[36261]: pgmap v289: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-27T06:52:54.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:53 smithi170 ceph-mon[36438]: pgmap v289: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-27T06:52:54.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:53 smithi016 ceph-mon[31908]: pgmap v289: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-27T06:52:55.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:55 smithi156 ceph-mon[36261]: pgmap v290: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T06:52:56.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:55 smithi170 ceph-mon[36438]: pgmap v290: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T06:52:56.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:55 smithi016 ceph-mon[31908]: pgmap v290: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T06:52:57.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:57 smithi156 ceph-mon[36261]: pgmap v291: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:52:58.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:57 smithi170 ceph-mon[36438]: pgmap v291: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:52:58.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:57 smithi016 ceph-mon[31908]: pgmap v291: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:52:59.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:52:59 smithi156 ceph-mon[36261]: pgmap v292: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T06:53:00.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:52:59 smithi170 ceph-mon[36438]: pgmap v292: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T06:53:00.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:52:59 smithi016 ceph-mon[31908]: pgmap v292: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-27T06:53:01.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:01 smithi156 ceph-mon[36261]: pgmap v293: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:53:02.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:01 smithi170 ceph-mon[36438]: pgmap v293: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:53:02.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:01 smithi016 ceph-mon[31908]: pgmap v293: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:53:03.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:03 smithi156 ceph-mon[36261]: pgmap v294: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-27T06:53:04.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:03 smithi170 ceph-mon[36438]: pgmap v294: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-27T06:53:04.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:03 smithi016 ceph-mon[31908]: pgmap v294: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-27T06:53:05.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:05 smithi156 ceph-mon[36261]: pgmap v295: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:06.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:05 smithi170 ceph-mon[36438]: pgmap v295: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:06.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:05 smithi016 ceph-mon[31908]: pgmap v295: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:07.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:07 smithi156 ceph-mon[36261]: pgmap v296: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:53:08.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:07 smithi170 ceph-mon[36438]: pgmap v296: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:53:08.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:07 smithi016 ceph-mon[31908]: pgmap v296: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:53:09.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:09 smithi156 ceph-mon[36261]: pgmap v297: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:53:10.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:09 smithi170 ceph-mon[36438]: pgmap v297: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:53:10.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:09 smithi016 ceph-mon[31908]: pgmap v297: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:53:10.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:10 smithi156 ceph-mon[36261]: osdmap e97: 12 total, 12 up, 12 in 2024-03-27T06:53:11.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:10 smithi170 ceph-mon[36438]: osdmap e97: 12 total, 12 up, 12 in 2024-03-27T06:53:11.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:10 smithi016 ceph-mon[31908]: osdmap e97: 12 total, 12 up, 12 in 2024-03-27T06:53:11.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:11 smithi156 ceph-mon[36261]: pgmap v299: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:53:12.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:11 smithi170 ceph-mon[36438]: pgmap v299: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:53:12.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:11 smithi016 ceph-mon[31908]: pgmap v299: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:53:12.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:12 smithi156 ceph-mon[36261]: pgmap v300: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:53:13.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:12 smithi170 ceph-mon[36438]: pgmap v300: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:53:13.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:12 smithi016 ceph-mon[31908]: pgmap v300: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:53:15.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:15 smithi170 ceph-mon[36438]: pgmap v301: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:53:15.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:15 smithi016 ceph-mon[31908]: pgmap v301: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:53:15.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:15 smithi156 ceph-mon[36261]: pgmap v301: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:53:17.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:17 smithi170 ceph-mon[36438]: pgmap v302: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:53:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:17 smithi016 ceph-mon[31908]: pgmap v302: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:53:17.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:17 smithi156 ceph-mon[36261]: pgmap v302: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:53:19.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:19 smithi170 ceph-mon[36438]: pgmap v303: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:53:19.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:19 smithi016 ceph-mon[31908]: pgmap v303: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:53:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:19 smithi156 ceph-mon[36261]: pgmap v303: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:53:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:21 smithi156 ceph-mon[36261]: pgmap v304: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 198 B/s wr, 15 op/s 2024-03-27T06:53:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:21 smithi156 ceph-mon[36261]: osdmap e98: 12 total, 12 up, 12 in 2024-03-27T06:53:22.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:21 smithi170 ceph-mon[36438]: pgmap v304: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 198 B/s wr, 15 op/s 2024-03-27T06:53:22.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:21 smithi170 ceph-mon[36438]: osdmap e98: 12 total, 12 up, 12 in 2024-03-27T06:53:22.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:21 smithi016 ceph-mon[31908]: pgmap v304: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 198 B/s wr, 15 op/s 2024-03-27T06:53:22.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:21 smithi016 ceph-mon[31908]: osdmap e98: 12 total, 12 up, 12 in 2024-03-27T06:53:23.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:23 smithi156 ceph-mon[36261]: pgmap v306: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:53:24.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:23 smithi170 ceph-mon[36438]: pgmap v306: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:53:24.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:23 smithi016 ceph-mon[31908]: pgmap v306: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:53:25.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:25 smithi156 ceph-mon[36261]: pgmap v307: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:53:26.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:25 smithi170 ceph-mon[36438]: pgmap v307: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:53:26.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:25 smithi016 ceph-mon[31908]: pgmap v307: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:53:27.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:27 smithi156 ceph-mon[36261]: pgmap v308: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:53:28.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:27 smithi170 ceph-mon[36438]: pgmap v308: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:53:28.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:27 smithi016 ceph-mon[31908]: pgmap v308: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:53:29.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:29 smithi156 ceph-mon[36261]: pgmap v309: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:53:30.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:29 smithi170 ceph-mon[36438]: pgmap v309: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:53:30.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:29 smithi016 ceph-mon[31908]: pgmap v309: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-27T06:53:31.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:31 smithi156 ceph-mon[36261]: pgmap v310: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:53:32.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:31 smithi170 ceph-mon[36438]: pgmap v310: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:53:32.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:31 smithi016 ceph-mon[31908]: pgmap v310: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-27T06:53:33.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:33 smithi156 ceph-mon[36261]: pgmap v311: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T06:53:34.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:33 smithi170 ceph-mon[36438]: pgmap v311: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T06:53:34.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:33 smithi016 ceph-mon[31908]: pgmap v311: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-27T06:53:35.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:35 smithi170 ceph-mon[36438]: pgmap v312: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:35.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:35 smithi156 ceph-mon[36261]: pgmap v312: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:36.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:35 smithi016 ceph-mon[31908]: pgmap v312: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:37.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:37 smithi156 ceph-mon[36261]: pgmap v313: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:53:38.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:37 smithi170 ceph-mon[36438]: pgmap v313: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:53:38.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:37 smithi016 ceph-mon[31908]: pgmap v313: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:53:39.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:39 smithi156 ceph-mon[36261]: pgmap v314: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:40.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:39 smithi170 ceph-mon[36438]: pgmap v314: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:40.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:39 smithi016 ceph-mon[31908]: pgmap v314: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:41.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:41 smithi156 ceph-mon[36261]: pgmap v315: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:53:42.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:41 smithi170 ceph-mon[36438]: pgmap v315: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:53:42.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:41 smithi016 ceph-mon[31908]: pgmap v315: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:53:42.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-27T06:53:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.647+0000 7f5150fb5640 1 -- 172.21.15.16:0/2401961282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f514c107c00 msgr2=0x7f514c107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.647+0000 7f5150fb5640 1 --2- 172.21.15.16:0/2401961282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f514c107c00 0x7f514c107fe0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f513c009900 tx=0x7f513c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:53:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.647+0000 7f5150fb5640 1 -- 172.21.15.16:0/2401961282 shutdown_connections 2024-03-27T06:53:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.647+0000 7f5150fb5640 1 --2- 172.21.15.16:0/2401961282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f514c102500 0x7f514c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.647+0000 7f5150fb5640 1 --2- 172.21.15.16:0/2401961282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514c101b40 0x7f514c101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.647+0000 7f5150fb5640 1 --2- 172.21.15.16:0/2401961282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f514c107c00 0x7f514c107fe0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.647+0000 7f5150fb5640 1 -- 172.21.15.16:0/2401961282 >> 172.21.15.16:0/2401961282 conn(0x7f514c0fd630 msgr2=0x7f514c0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.647+0000 7f5150fb5640 1 -- 172.21.15.16:0/2401961282 shutdown_connections 2024-03-27T06:53:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.647+0000 7f5150fb5640 1 -- 172.21.15.16:0/2401961282 wait complete. 2024-03-27T06:53:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f5150fb5640 1 Processor -- start 2024-03-27T06:53:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f5150fb5640 1 -- start start 2024-03-27T06:53:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f5150fb5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514c101b40 0x7f514c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f5150fb5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f514c102500 0x7f514c074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f5150fb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f514c107c00 0x7f514c0748d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f5150fb5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f514c112060 con 0x7f514c107c00 2024-03-27T06:53:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f5150fb5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f514c111ee0 con 0x7f514c102500 2024-03-27T06:53:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f5150fb5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f514c1121e0 con 0x7f514c101b40 2024-03-27T06:53:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f514ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f514c107c00 0x7f514c0748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:42.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f514ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f514c107c00 0x7f514c0748d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47968/0 (socket says 172.21.15.16:47968) 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f514ad9d640 1 -- 172.21.15.16:0/206948296 learned_addr learned my addr 172.21.15.16:0/206948296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f5143fff640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f514c102500 0x7f514c074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.648+0000 7f514a59c640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514c101b40 0x7f514c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f514ad9d640 1 -- 172.21.15.16:0/206948296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514c101b40 msgr2=0x7f514c073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f514ad9d640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514c101b40 0x7f514c073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f514ad9d640 1 -- 172.21.15.16:0/206948296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f514c102500 msgr2=0x7f514c074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f514ad9d640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f514c102500 0x7f514c074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f514ad9d640 1 -- 172.21.15.16:0/206948296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c1a68a0 con 0x7f514c107c00 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f514a59c640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514c101b40 0x7f514c073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f5143fff640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f514c102500 0x7f514c074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f514ad9d640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f514c107c00 0x7f514c0748d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f514400eb10 tx=0x7f514400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f51437fe640 1 -- 172.21.15.16:0/206948296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144004b20 con 0x7f514c107c00 2024-03-27T06:53:42.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f51437fe640 1 -- 172.21.15.16:0/206948296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5144004cc0 con 0x7f514c107c00 2024-03-27T06:53:42.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f51437fe640 1 -- 172.21.15.16:0/206948296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51440107f0 con 0x7f514c107c00 2024-03-27T06:53:42.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f5150fb5640 1 -- 172.21.15.16:0/206948296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514c1a6b90 con 0x7f514c107c00 2024-03-27T06:53:42.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.649+0000 7f5150fb5640 1 -- 172.21.15.16:0/206948296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514c1a7090 con 0x7f514c107c00 2024-03-27T06:53:42.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.650+0000 7f5150fb5640 1 -- 172.21.15.16:0/206948296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514c04f8b0 con 0x7f514c107c00 2024-03-27T06:53:42.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.651+0000 7f51437fe640 1 -- 172.21.15.16:0/206948296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f5144005020 con 0x7f514c107c00 2024-03-27T06:53:42.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.652+0000 7f51437fe640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51380a3820 0x7f51380a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:42.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.652+0000 7f514a59c640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51380a3820 0x7f51380a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:42.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.652+0000 7f51437fe640 1 -- 172.21.15.16:0/206948296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f51440d0690 con 0x7f514c107c00 2024-03-27T06:53:42.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.653+0000 7f514a59c640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51380a3820 0x7f51380a5ce0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f513c006010 tx=0x7f513c005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:42.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.657+0000 7f51437fe640 1 -- 172.21.15.16:0/206948296 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f514409cec0 con 0x7f514c107c00 2024-03-27T06:53:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.865+0000 7f5150fb5640 1 -- 172.21.15.16:0/206948296 --> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f514c0ff250 con 0x7f51380a3820 2024-03-27T06:53:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.872+0000 7f51437fe640 1 -- 172.21.15.16:0/206948296 <== mgr.15354 v2:172.21.15.16:6800/3715976676 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f514c0ff250 con 0x7f51380a3820 2024-03-27T06:53:42.868 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-27T00:00:00", "created": "2024-03-27T06:43:41", "first": "2024-03-27T06:44:00", "last": "2024-03-27T06:53:00", "last_pruned": "2024-03-27T06:53:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-27T06:53:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.874+0000 7f5150fb5640 1 -- 172.21.15.16:0/206948296 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51380a3820 msgr2=0x7f51380a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.874+0000 7f5150fb5640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51380a3820 0x7f51380a5ce0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f513c006010 tx=0x7f513c005e40 comp rx=0 tx=0).stop 2024-03-27T06:53:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.874+0000 7f5150fb5640 1 -- 172.21.15.16:0/206948296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f514c107c00 msgr2=0x7f514c0748d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.874+0000 7f5150fb5640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f514c107c00 0x7f514c0748d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f514400eb10 tx=0x7f514400c2d0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.874+0000 7f5150fb5640 1 -- 172.21.15.16:0/206948296 shutdown_connections 2024-03-27T06:53:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.874+0000 7f5150fb5640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f514c107c00 0x7f514c0748d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.874+0000 7f5150fb5640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f514c102500 0x7f514c074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.874+0000 7f5150fb5640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f51380a3820 0x7f51380a5ce0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.874+0000 7f5150fb5640 1 --2- 172.21.15.16:0/206948296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514c101b40 0x7f514c073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.874+0000 7f5150fb5640 1 -- 172.21.15.16:0/206948296 >> 172.21.15.16:0/206948296 conn(0x7f514c0fd630 msgr2=0x7f514c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.875+0000 7f5150fb5640 1 -- 172.21.15.16:0/206948296 shutdown_connections 2024-03-27T06:53:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.875+0000 7f5150fb5640 1 -- 172.21.15.16:0/206948296 wait complete. 2024-03-27T06:53:42.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-27T06:53:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.989+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1393622521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f4107c40 msgr2=0x7f53f4108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.989+0000 7f53fc7b6640 1 --2- 172.21.15.16:0/1393622521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f4107c40 0x7f53f4108020 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009900 tx=0x7f53ec02bcd0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.989+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1393622521 shutdown_connections 2024-03-27T06:53:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.989+0000 7f53fc7b6640 1 --2- 172.21.15.16:0/1393622521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53f4069b70 0x7f53f410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.989+0000 7f53fc7b6640 1 --2- 172.21.15.16:0/1393622521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53f40691d0 0x7f53f4069630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.989+0000 7f53fc7b6640 1 --2- 172.21.15.16:0/1393622521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f4107c40 0x7f53f4108020 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.989+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1393622521 >> 172.21.15.16:0/1393622521 conn(0x7f53f40778a0 msgr2=0x7f53f4077cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1393622521 shutdown_connections 2024-03-27T06:53:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1393622521 wait complete. 2024-03-27T06:53:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fc7b6640 1 Processor -- start 2024-03-27T06:53:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fc7b6640 1 -- start start 2024-03-27T06:53:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fc7b6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53f40691d0 0x7f53f4100d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fc7b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f4069b70 0x7f53f41012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fc7b6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53f4107c40 0x7f53f41017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fc7b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53f4111fe0 con 0x7f53f4069b70 2024-03-27T06:53:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fc7b6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f53f4111e60 con 0x7f53f40691d0 2024-03-27T06:53:42.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fc7b6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f53f4112160 con 0x7f53f4107c40 2024-03-27T06:53:42.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53f9d51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f4069b70 0x7f53f41012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:42.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53f9d51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f4069b70 0x7f53f41012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48000/0 (socket says 172.21.15.16:48000) 2024-03-27T06:53:42.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53f9d51640 1 -- 172.21.15.16:0/1532546666 learned_addr learned my addr 172.21.15.16:0/1532546666 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:53:42.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fa552640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53f40691d0 0x7f53f4100d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:42.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.990+0000 7f53fad53640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53f4107c40 0x7f53f41017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.991+0000 7f53f9d51640 1 -- 172.21.15.16:0/1532546666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53f4107c40 msgr2=0x7f53f41017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.991+0000 7f53f9d51640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53f4107c40 0x7f53f41017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.991+0000 7f53f9d51640 1 -- 172.21.15.16:0/1532546666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53f40691d0 msgr2=0x7f53f4100d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.991+0000 7f53f9d51640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53f40691d0 0x7f53f4100d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.991+0000 7f53f9d51640 1 -- 172.21.15.16:0/1532546666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f41a69a0 con 0x7f53f4069b70 2024-03-27T06:53:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.991+0000 7f53f9d51640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f4069b70 0x7f53f41012b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f53e400e9f0 tx=0x7f53e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.991+0000 7f53eb7fe640 1 -- 172.21.15.16:0/1532546666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e400cd80 con 0x7f53f4069b70 2024-03-27T06:53:42.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.991+0000 7f53eb7fe640 1 -- 172.21.15.16:0/1532546666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53e4010930 con 0x7f53f4069b70 2024-03-27T06:53:42.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.991+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1532546666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f41a6c90 con 0x7f53f4069b70 2024-03-27T06:53:42.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.991+0000 7f53eb7fe640 1 -- 172.21.15.16:0/1532546666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4018b00 con 0x7f53f4069b70 2024-03-27T06:53:42.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.991+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1532546666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f41a7190 con 0x7f53f4069b70 2024-03-27T06:53:42.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.992+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1532546666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f404f8b0 con 0x7f53f4069b70 2024-03-27T06:53:42.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.993+0000 7f53eb7fe640 1 -- 172.21.15.16:0/1532546666 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f53e4018ca0 con 0x7f53f4069b70 2024-03-27T06:53:42.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.994+0000 7f53eb7fe640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f53e00a3820 0x7f53e00a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:42.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.994+0000 7f53fa552640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f53e00a3820 0x7f53e00a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:42.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.994+0000 7f53eb7fe640 1 -- 172.21.15.16:0/1532546666 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f53e40d0490 con 0x7f53f4069b70 2024-03-27T06:53:42.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.994+0000 7f53fa552640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f53e00a3820 0x7f53e00a5ce0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f53ec006010 tx=0x7f53ec005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:42.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:42.998+0000 7f53eb7fe640 1 -- 172.21.15.16:0/1532546666 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f53e409c6d0 con 0x7f53f4069b70 2024-03-27T06:53:43.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.208+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1532546666 --> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f53f410a0a0 con 0x7f53e00a3820 2024-03-27T06:53:43.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.214+0000 7f53eb7fe640 1 -- 172.21.15.16:0/1532546666 <== mgr.15354 v2:172.21.15.16:6800/3715976676 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f53ec02ca70 con 0x7f53e00a3820 2024-03-27T06:53:43.210 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-03-27T06:53:43.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1532546666 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f53e00a3820 msgr2=0x7f53e00a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f53e00a3820 0x7f53e00a5ce0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f53ec006010 tx=0x7f53ec005e40 comp rx=0 tx=0).stop 2024-03-27T06:53:43.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1532546666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f4069b70 msgr2=0x7f53f41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f4069b70 0x7f53f41012b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f53e400e9f0 tx=0x7f53e400eeb0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1532546666 shutdown_connections 2024-03-27T06:53:43.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f53e00a3820 0x7f53e00a5ce0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53f4107c40 0x7f53f41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f4069b70 0x7f53f41012b0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 --2- 172.21.15.16:0/1532546666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53f40691d0 0x7f53f4100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1532546666 >> 172.21.15.16:0/1532546666 conn(0x7f53f40778a0 msgr2=0x7f53f40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1532546666 shutdown_connections 2024-03-27T06:53:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.216+0000 7f53fc7b6640 1 -- 172.21.15.16:0/1532546666 wait complete. 2024-03-27T06:53:43.223 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-27T06:53:43.238 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-27T06:53:43.238 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-27T06:53:43.239 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-27T06:53:43.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:53:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.365+0000 7fa1c3439640 1 -- 172.21.15.16:0/429461639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc06a450 msgr2=0x7fa1bc06a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.365+0000 7fa1c3439640 1 --2- 172.21.15.16:0/429461639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc06a450 0x7fa1bc06a830 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009930 tx=0x7fa1b402bc80 comp rx=0 tx=0).stop 2024-03-27T06:53:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.365+0000 7fa1c3439640 1 -- 172.21.15.16:0/429461639 shutdown_connections 2024-03-27T06:53:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.365+0000 7fa1c3439640 1 --2- 172.21.15.16:0/429461639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1bc106ee0 0x7fa1bc10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.365+0000 7fa1c3439640 1 --2- 172.21.15.16:0/429461639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1bc06ae00 0x7fa1bc1067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.365+0000 7fa1c3439640 1 --2- 172.21.15.16:0/429461639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc06a450 0x7fa1bc06a830 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.365+0000 7fa1c3439640 1 -- 172.21.15.16:0/429461639 >> 172.21.15.16:0/429461639 conn(0x7fa1bc077af0 msgr2=0x7fa1bc077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.365+0000 7fa1c3439640 1 -- 172.21.15.16:0/429461639 shutdown_connections 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.365+0000 7fa1c3439640 1 -- 172.21.15.16:0/429461639 wait complete. 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c3439640 1 Processor -- start 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c3439640 1 -- start start 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c3439640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc06a450 0x7fa1bc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c3439640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1bc106ee0 0x7fa1bc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c3439640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1bc19cd40 0x7fa1bc19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c3439640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc112050 con 0x7fa1bc06a450 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c3439640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc111ed0 con 0x7fa1bc19cd40 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c3439640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc1121d0 con 0x7fa1bc106ee0 2024-03-27T06:53:43.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c11d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc06a450 0x7fa1bc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c11d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc06a450 0x7fa1bc1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48012/0 (socket says 172.21.15.16:48012) 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c11d5640 1 -- 172.21.15.16:0/3807321471 learned_addr learned my addr 172.21.15.16:0/3807321471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c19d6640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1bc19cd40 0x7fa1bc19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c09d4640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1bc106ee0 0x7fa1bc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c11d5640 1 -- 172.21.15.16:0/3807321471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1bc106ee0 msgr2=0x7fa1bc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c11d5640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1bc106ee0 0x7fa1bc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c11d5640 1 -- 172.21.15.16:0/3807321471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1bc19cd40 msgr2=0x7fa1bc19d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c11d5640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1bc19cd40 0x7fa1bc19d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c11d5640 1 -- 172.21.15.16:0/3807321471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc19d830 con 0x7fa1bc06a450 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.366+0000 7fa1c11d5640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc06a450 0x7fa1bc1984d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009230 tx=0x7fa1b402dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.367+0000 7fa1ba7fc640 1 -- 172.21.15.16:0/3807321471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4043020 con 0x7fa1bc06a450 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.367+0000 7fa1c3439640 1 -- 172.21.15.16:0/3807321471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1bc19dac0 con 0x7fa1bc06a450 2024-03-27T06:53:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.367+0000 7fa1ba7fc640 1 -- 172.21.15.16:0/3807321471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa1b4004250 con 0x7fa1bc06a450 2024-03-27T06:53:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.367+0000 7fa1ba7fc640 1 -- 172.21.15.16:0/3807321471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4003970 con 0x7fa1bc06a450 2024-03-27T06:53:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.367+0000 7fa1c3439640 1 -- 172.21.15.16:0/3807321471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1bc1a5280 con 0x7fa1bc06a450 2024-03-27T06:53:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.368+0000 7fa1c3439640 1 -- 172.21.15.16:0/3807321471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1bc068d90 con 0x7fa1bc06a450 2024-03-27T06:53:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.369+0000 7fa1ba7fc640 1 -- 172.21.15.16:0/3807321471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fa1b4046020 con 0x7fa1bc06a450 2024-03-27T06:53:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.370+0000 7fa1ba7fc640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa1a40a3780 0x7fa1a40a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.370+0000 7fa1c09d4640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa1a40a3780 0x7fa1a40a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.370+0000 7fa1ba7fc640 1 -- 172.21.15.16:0/3807321471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fa1b40f22c0 con 0x7fa1bc06a450 2024-03-27T06:53:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.371+0000 7fa1c09d4640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa1a40a3780 0x7fa1a40a5c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc108300 tx=0x7fa1b0005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.374+0000 7fa1ba7fc640 1 -- 172.21.15.16:0/3807321471 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fa1b404d020 con 0x7fa1bc06a450 2024-03-27T06:53:43.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.559+0000 7fa1c3439640 1 -- 172.21.15.16:0/3807321471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1bc1997d0 con 0x7fa1bc06a450 2024-03-27T06:53:43.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.561+0000 7fa1ba7fc640 1 -- 172.21.15.16:0/3807321471 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23072 (secure 0 0 0) 0x7fa1bc1997d0 con 0x7fa1bc06a450 2024-03-27T06:53:43.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:53:43.558 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":98,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:53:20.617038+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3655179049":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3519122217":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6800/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/212881673":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/361067788":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:53:43.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 -- 172.21.15.16:0/3807321471 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa1a40a3780 msgr2=0x7fa1a40a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa1a40a3780 0x7fa1a40a5c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc108300 tx=0x7fa1b0005f90 comp rx=0 tx=0).stop 2024-03-27T06:53:43.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 -- 172.21.15.16:0/3807321471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc06a450 msgr2=0x7fa1bc1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc06a450 0x7fa1bc1984d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009230 tx=0x7fa1b402dc80 comp rx=0 tx=0).stop 2024-03-27T06:53:43.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 -- 172.21.15.16:0/3807321471 shutdown_connections 2024-03-27T06:53:43.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1bc19cd40 0x7fa1bc19d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa1a40a3780 0x7fa1a40a5c40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1bc106ee0 0x7fa1bc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 --2- 172.21.15.16:0/3807321471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1bc06a450 0x7fa1bc1984d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 -- 172.21.15.16:0/3807321471 >> 172.21.15.16:0/3807321471 conn(0x7fa1bc077af0 msgr2=0x7fa1bc100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:43.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 -- 172.21.15.16:0/3807321471 shutdown_connections 2024-03-27T06:53:43.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.563+0000 7fa1c3439640 1 -- 172.21.15.16:0/3807321471 wait complete. 2024-03-27T06:53:43.570 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T06:41:26.393978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T06:41:27.707469+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:53:43.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T06:53:43.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.680+0000 7fd653e8d640 1 -- 172.21.15.16:0/1225854550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c107ad0 msgr2=0x7fd64c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.680+0000 7fd653e8d640 1 --2- 172.21.15.16:0/1225854550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c107ad0 0x7fd64c107eb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd648009930 tx=0x7fd64802bc30 comp rx=0 tx=0).stop 2024-03-27T06:53:43.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.680+0000 7fd653e8d640 1 -- 172.21.15.16:0/1225854550 shutdown_connections 2024-03-27T06:53:43.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.680+0000 7fd653e8d640 1 --2- 172.21.15.16:0/1225854550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd64c0696f0 0x7fd64c10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.680+0000 7fd653e8d640 1 --2- 172.21.15.16:0/1225854550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd64c068d30 0x7fd64c0691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 --2- 172.21.15.16:0/1225854550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c107ad0 0x7fd64c107eb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 -- 172.21.15.16:0/1225854550 >> 172.21.15.16:0/1225854550 conn(0x7fd64c077b50 msgr2=0x7fd64c077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:43.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 -- 172.21.15.16:0/1225854550 shutdown_connections 2024-03-27T06:53:43.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 -- 172.21.15.16:0/1225854550 wait complete. 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 Processor -- start 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 -- start start 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd64c068d30 0x7fd64c100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c0696f0 0x7fd64c100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd64c107ad0 0x7fd64c101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c111fb0 con 0x7fd64c0696f0 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c111e30 con 0x7fd64c068d30 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd653e8d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c112130 con 0x7fd64c107ad0 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd651428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c0696f0 0x7fd64c100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd651428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c0696f0 0x7fd64c100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48026/0 (socket says 172.21.15.16:48026) 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.681+0000 7fd651428640 1 -- 172.21.15.16:0/4043534848 learned_addr learned my addr 172.21.15.16:0/4043534848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd65242a640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd64c107ad0 0x7fd64c101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd651c29640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd64c068d30 0x7fd64c100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd651428640 1 -- 172.21.15.16:0/4043534848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd64c107ad0 msgr2=0x7fd64c101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd651428640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd64c107ad0 0x7fd64c101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd651428640 1 -- 172.21.15.16:0/4043534848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd64c068d30 msgr2=0x7fd64c100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd651428640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd64c068d30 0x7fd64c100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd651428640 1 -- 172.21.15.16:0/4043534848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64c1a69d0 con 0x7fd64c0696f0 2024-03-27T06:53:43.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd651428640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c0696f0 0x7fd64c100f50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00e9f0 tx=0x7fd63c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:43.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd642ffd640 1 -- 172.21.15.16:0/4043534848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00cd80 con 0x7fd64c0696f0 2024-03-27T06:53:43.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd642ffd640 1 -- 172.21.15.16:0/4043534848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd63c010930 con 0x7fd64c0696f0 2024-03-27T06:53:43.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd653e8d640 1 -- 172.21.15.16:0/4043534848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64c1a6cc0 con 0x7fd64c0696f0 2024-03-27T06:53:43.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd642ffd640 1 -- 172.21.15.16:0/4043534848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c018b00 con 0x7fd64c0696f0 2024-03-27T06:53:43.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.682+0000 7fd653e8d640 1 -- 172.21.15.16:0/4043534848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64c1a7200 con 0x7fd64c0696f0 2024-03-27T06:53:43.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.683+0000 7fd653e8d640 1 -- 172.21.15.16:0/4043534848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd64c068940 con 0x7fd64c0696f0 2024-03-27T06:53:43.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.684+0000 7fd642ffd640 1 -- 172.21.15.16:0/4043534848 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fd63c018ca0 con 0x7fd64c0696f0 2024-03-27T06:53:43.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.685+0000 7fd642ffd640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd6380a3800 0x7fd6380a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.685+0000 7fd651c29640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd6380a3800 0x7fd6380a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.685+0000 7fd642ffd640 1 -- 172.21.15.16:0/4043534848 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fd63c0d0790 con 0x7fd64c0696f0 2024-03-27T06:53:43.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.685+0000 7fd651c29640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd6380a3800 0x7fd6380a5cc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd648005ab0 tx=0x7fd648005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.688+0000 7fd642ffd640 1 -- 172.21.15.16:0/4043534848 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd63c09c970 con 0x7fd64c0696f0 2024-03-27T06:53:43.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:43 smithi016 ceph-mon[31908]: pgmap v316: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:43.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:43 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3807321471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:53:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.871+0000 7fd653e8d640 1 -- 172.21.15.16:0/4043534848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd64c109aa0 con 0x7fd64c0696f0 2024-03-27T06:53:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.872+0000 7fd642ffd640 1 -- 172.21.15.16:0/4043534848 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7fd63c0a0c70 con 0x7fd64c0696f0 2024-03-27T06:53:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:53:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 -- 172.21.15.16:0/4043534848 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd6380a3800 msgr2=0x7fd6380a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd6380a3800 0x7fd6380a5cc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd648005ab0 tx=0x7fd648005fd0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 -- 172.21.15.16:0/4043534848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c0696f0 msgr2=0x7fd64c100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c0696f0 0x7fd64c100f50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00e9f0 tx=0x7fd63c00eeb0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 -- 172.21.15.16:0/4043534848 shutdown_connections 2024-03-27T06:53:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd6380a3800 0x7fd6380a5cc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd64c107ad0 0x7fd64c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c0696f0 0x7fd64c100f50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 --2- 172.21.15.16:0/4043534848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd64c068d30 0x7fd64c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 -- 172.21.15.16:0/4043534848 >> 172.21.15.16:0/4043534848 conn(0x7fd64c077b50 msgr2=0x7fd64c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 -- 172.21.15.16:0/4043534848 shutdown_connections 2024-03-27T06:53:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.874+0000 7fd653e8d640 1 -- 172.21.15.16:0/4043534848 wait complete. 2024-03-27T06:53:43.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T06:53:43.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:43 smithi156 ceph-mon[36261]: pgmap v316: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:43.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:43 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3807321471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:53:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 -- 172.21.15.16:0/3134149847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0102dc0 msgr2=0x7faea01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 --2- 172.21.15.16:0/3134149847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0102dc0 0x7faea01031a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fae90009900 tx=0x7fae9002bcd0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 -- 172.21.15.16:0/3134149847 shutdown_connections 2024-03-27T06:53:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 --2- 172.21.15.16:0/3134149847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea010c190 0x7faea010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 --2- 172.21.15.16:0/3134149847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faea01036e0 0x7faea010bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 --2- 172.21.15.16:0/3134149847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea0102dc0 0x7faea01031a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 -- 172.21.15.16:0/3134149847 >> 172.21.15.16:0/3134149847 conn(0x7faea0077bb0 msgr2=0x7faea0077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 -- 172.21.15.16:0/3134149847 shutdown_connections 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 -- 172.21.15.16:0/3134149847 wait complete. 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 Processor -- start 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 -- start start 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faea0102dc0 0x7faea0100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea01036e0 0x7faea0100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea010c190 0x7faea0101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faea0112080 con 0x7faea01036e0 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7faea0111f00 con 0x7faea010c190 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.991+0000 7faea590c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faea0112200 con 0x7faea0102dc0 2024-03-27T06:53:43.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7fae97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea01036e0 0x7faea0100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7fae97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea01036e0 0x7faea0100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48042/0 (socket says 172.21.15.16:48042) 2024-03-27T06:53:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7fae97fff640 1 -- 172.21.15.16:0/4139334442 learned_addr learned my addr 172.21.15.16:0/4139334442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:53:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7fae9f7fe640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea010c190 0x7faea0101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7fae9effd640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faea0102dc0 0x7faea0100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7fae97fff640 1 -- 172.21.15.16:0/4139334442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faea0102dc0 msgr2=0x7faea0100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7fae97fff640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faea0102dc0 0x7faea0100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7fae97fff640 1 -- 172.21.15.16:0/4139334442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea010c190 msgr2=0x7faea0101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7fae97fff640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea010c190 0x7faea0101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7fae97fff640 1 -- 172.21.15.16:0/4139334442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea01a6a00 con 0x7faea01036e0 2024-03-27T06:53:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7fae97fff640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea01036e0 0x7faea0100f50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fae8800e9f0 tx=0x7fae8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:43.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7faea490a640 1 -- 172.21.15.16:0/4139334442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8800cd80 con 0x7faea01036e0 2024-03-27T06:53:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7faea590c640 1 -- 172.21.15.16:0/4139334442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea01a6cf0 con 0x7faea01036e0 2024-03-27T06:53:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7faea490a640 1 -- 172.21.15.16:0/4139334442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae88005520 con 0x7faea01036e0 2024-03-27T06:53:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7faea490a640 1 -- 172.21.15.16:0/4139334442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88010690 con 0x7faea01036e0 2024-03-27T06:53:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.992+0000 7faea590c640 1 -- 172.21.15.16:0/4139334442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea01a71e0 con 0x7faea01036e0 2024-03-27T06:53:43.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.993+0000 7faea590c640 1 -- 172.21.15.16:0/4139334442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faea0068d90 con 0x7faea01036e0 2024-03-27T06:53:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.994+0000 7faea490a640 1 -- 172.21.15.16:0/4139334442 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fae880046c0 con 0x7faea01036e0 2024-03-27T06:53:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.995+0000 7faea490a640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae8c0a37b0 0x7fae8c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.995+0000 7fae9effd640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae8c0a37b0 0x7fae8c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.995+0000 7faea490a640 1 -- 172.21.15.16:0/4139334442 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fae880d0740 con 0x7faea01036e0 2024-03-27T06:53:43.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.996+0000 7fae9effd640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae8c0a37b0 0x7fae8c0a5c70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fae90006010 tx=0x7fae90005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:43.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:43.999+0000 7faea490a640 1 -- 172.21.15.16:0/4139334442 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fae88053cb0 con 0x7faea01036e0 2024-03-27T06:53:44.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:43 smithi170 ceph-mon[36438]: pgmap v316: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:44.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:43 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3807321471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:53:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.183+0000 7faea590c640 1 -- 172.21.15.16:0/4139334442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faea0107280 con 0x7faea01036e0 2024-03-27T06:53:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.183+0000 7faea490a640 1 -- 172.21.15.16:0/4139334442 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fae8809cfb0 con 0x7faea01036e0 2024-03-27T06:53:44.180 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-03-27T06:53:44.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 -- 172.21.15.16:0/4139334442 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae8c0a37b0 msgr2=0x7fae8c0a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae8c0a37b0 0x7fae8c0a5c70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fae90006010 tx=0x7fae90005e40 comp rx=0 tx=0).stop 2024-03-27T06:53:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 -- 172.21.15.16:0/4139334442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea01036e0 msgr2=0x7faea0100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea01036e0 0x7faea0100f50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fae8800e9f0 tx=0x7fae8800edf0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 -- 172.21.15.16:0/4139334442 shutdown_connections 2024-03-27T06:53:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea010c190 0x7faea0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faea01036e0 0x7faea0100f50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fae8c0a37b0 0x7fae8c0a5c70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 --2- 172.21.15.16:0/4139334442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faea0102dc0 0x7faea0100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 -- 172.21.15.16:0/4139334442 >> 172.21.15.16:0/4139334442 conn(0x7faea0077bb0 msgr2=0x7faea00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 -- 172.21.15.16:0/4139334442 shutdown_connections 2024-03-27T06:53:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.185+0000 7faea590c640 1 -- 172.21.15.16:0/4139334442 wait complete. 2024-03-27T06:53:44.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T06:53:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.302+0000 7ff088639640 1 -- 172.21.15.16:0/3756637738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080100a70 msgr2=0x7ff080100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.302+0000 7ff088639640 1 --2- 172.21.15.16:0/3756637738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080100a70 0x7ff080100ef0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff070009930 tx=0x7ff07002bc30 comp rx=0 tx=0).stop 2024-03-27T06:53:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.302+0000 7ff088639640 1 -- 172.21.15.16:0/3756637738 shutdown_connections 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.302+0000 7ff088639640 1 --2- 172.21.15.16:0/3756637738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080101430 0x7ff08010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.302+0000 7ff088639640 1 --2- 172.21.15.16:0/3756637738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080100a70 0x7ff080100ef0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.302+0000 7ff088639640 1 --2- 172.21.15.16:0/3756637738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff080106a80 0x7ff080106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.302+0000 7ff088639640 1 -- 172.21.15.16:0/3756637738 >> 172.21.15.16:0/3756637738 conn(0x7ff080077bb0 msgr2=0x7ff080077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.302+0000 7ff088639640 1 -- 172.21.15.16:0/3756637738 shutdown_connections 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.302+0000 7ff088639640 1 -- 172.21.15.16:0/3756637738 wait complete. 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff088639640 1 Processor -- start 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff088639640 1 -- start start 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff088639640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080100a70 0x7ff080198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff088639640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080101430 0x7ff080198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff088639640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff080106a80 0x7ff08019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff088639640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff080110f60 con 0x7ff080101430 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff088639640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7ff080110de0 con 0x7ff080100a70 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff088639640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801110e0 con 0x7ff080106a80 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff085bd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080101430 0x7ff080198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff085bd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080101430 0x7ff080198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48058/0 (socket says 172.21.15.16:48058) 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff085bd4640 1 -- 172.21.15.16:0/2843798688 learned_addr learned my addr 172.21.15.16:0/2843798688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff0863d5640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080100a70 0x7ff080198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff086bd6640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff080106a80 0x7ff08019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff085bd4640 1 -- 172.21.15.16:0/2843798688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff080106a80 msgr2=0x7ff08019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff085bd4640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff080106a80 0x7ff08019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.303+0000 7ff085bd4640 1 -- 172.21.15.16:0/2843798688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080100a70 msgr2=0x7ff080198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.304+0000 7ff085bd4640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080100a70 0x7ff080198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.304+0000 7ff085bd4640 1 -- 172.21.15.16:0/2843798688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08019d4f0 con 0x7ff080101430 2024-03-27T06:53:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.304+0000 7ff086bd6640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff080106a80 0x7ff08019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:53:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.304+0000 7ff085bd4640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080101430 0x7ff080198a40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff070009230 tx=0x7ff07002dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.304+0000 7ff0777fe640 1 -- 172.21.15.16:0/2843798688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070043020 con 0x7ff080101430 2024-03-27T06:53:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.304+0000 7ff088639640 1 -- 172.21.15.16:0/2843798688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08019d780 con 0x7ff080101430 2024-03-27T06:53:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.304+0000 7ff0777fe640 1 -- 172.21.15.16:0/2843798688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff070004250 con 0x7ff080101430 2024-03-27T06:53:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.304+0000 7ff0777fe640 1 -- 172.21.15.16:0/2843798688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070003970 con 0x7ff080101430 2024-03-27T06:53:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.304+0000 7ff088639640 1 -- 172.21.15.16:0/2843798688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0801a5070 con 0x7ff080101430 2024-03-27T06:53:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.305+0000 7ff088639640 1 -- 172.21.15.16:0/2843798688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff080068d90 con 0x7ff080101430 2024-03-27T06:53:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.306+0000 7ff0777fe640 1 -- 172.21.15.16:0/2843798688 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7ff070046020 con 0x7ff080101430 2024-03-27T06:53:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.306+0000 7ff0777fe640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff06c0a3780 0x7ff06c0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.306+0000 7ff0863d5640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff06c0a3780 0x7ff06c0a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.307+0000 7ff0777fe640 1 -- 172.21.15.16:0/2843798688 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7ff0700f2060 con 0x7ff080101430 2024-03-27T06:53:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.307+0000 7ff0863d5640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff06c0a3780 0x7ff06c0a5c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009b50 tx=0x7ff07c005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.311+0000 7ff0777fe640 1 -- 172.21.15.16:0/2843798688 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7ff07004d020 con 0x7ff080101430 2024-03-27T06:53:44.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.497+0000 7ff088639640 1 -- 172.21.15.16:0/2843798688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff080109ff0 con 0x7ff080101430 2024-03-27T06:53:44.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.498+0000 7ff0777fe640 1 -- 172.21.15.16:0/2843798688 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7ff0700bea00 con 0x7ff080101430 2024-03-27T06:53:44.494 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-03-27T06:53:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 -- 172.21.15.16:0/2843798688 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff06c0a3780 msgr2=0x7ff06c0a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff06c0a3780 0x7ff06c0a5c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009b50 tx=0x7ff07c005f90 comp rx=0 tx=0).stop 2024-03-27T06:53:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 -- 172.21.15.16:0/2843798688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080101430 msgr2=0x7ff080198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080101430 0x7ff080198a40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff070009230 tx=0x7ff07002dc90 comp rx=0 tx=0).stop 2024-03-27T06:53:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 -- 172.21.15.16:0/2843798688 shutdown_connections 2024-03-27T06:53:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7ff06c0a3780 0x7ff06c0a5c40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff080106a80 0x7ff08019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080101430 0x7ff080198a40 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 --2- 172.21.15.16:0/2843798688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff080100a70 0x7ff080198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 -- 172.21.15.16:0/2843798688 >> 172.21.15.16:0/2843798688 conn(0x7ff080077bb0 msgr2=0x7ff08006a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 -- 172.21.15.16:0/2843798688 shutdown_connections 2024-03-27T06:53:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.500+0000 7ff088639640 1 -- 172.21.15.16:0/2843798688 wait complete. 2024-03-27T06:53:44.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-27T06:53:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.614+0000 7fcb0f11e640 1 -- 172.21.15.16:0/3568342878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb08106a20 msgr2=0x7fcb08106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.614+0000 7fcb0f11e640 1 --2- 172.21.15.16:0/3568342878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb08106a20 0x7fcb08106e00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009930 tx=0x7fcafc02bc30 comp rx=0 tx=0).stop 2024-03-27T06:53:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.614+0000 7fcb0f11e640 1 -- 172.21.15.16:0/3568342878 shutdown_connections 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.614+0000 7fcb0f11e640 1 --2- 172.21.15.16:0/3568342878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb081013d0 0x7fcb0810d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.614+0000 7fcb0f11e640 1 --2- 172.21.15.16:0/3568342878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb08100a10 0x7fcb08100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.614+0000 7fcb0f11e640 1 --2- 172.21.15.16:0/3568342878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb08106a20 0x7fcb08106e00 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.614+0000 7fcb0f11e640 1 -- 172.21.15.16:0/3568342878 >> 172.21.15.16:0/3568342878 conn(0x7fcb08077af0 msgr2=0x7fcb08077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.614+0000 7fcb0f11e640 1 -- 172.21.15.16:0/3568342878 shutdown_connections 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.614+0000 7fcb0f11e640 1 -- 172.21.15.16:0/3568342878 wait complete. 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0f11e640 1 Processor -- start 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0f11e640 1 -- start start 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0f11e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb08100a10 0x7fcb08198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0f11e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb081013d0 0x7fcb08198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0f11e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb08106a20 0x7fcb0819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0f11e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb08112000 con 0x7fcb08100a10 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0f11e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fcb08111e80 con 0x7fcb081013d0 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0f11e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcb08112180 con 0x7fcb08106a20 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0ceba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb08100a10 0x7fcb08198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0ceba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb08100a10 0x7fcb08198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48074/0 (socket says 172.21.15.16:48074) 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0ceba640 1 -- 172.21.15.16:0/2026917457 learned_addr learned my addr 172.21.15.16:0/2026917457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb07fff640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb081013d0 0x7fcb08198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0d6bb640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb08106a20 0x7fcb0819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0ceba640 1 -- 172.21.15.16:0/2026917457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb08106a20 msgr2=0x7fcb0819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0ceba640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb08106a20 0x7fcb0819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0ceba640 1 -- 172.21.15.16:0/2026917457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb081013d0 msgr2=0x7fcb08198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0ceba640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb081013d0 0x7fcb08198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0ceba640 1 -- 172.21.15.16:0/2026917457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0819d4f0 con 0x7fcb08100a10 2024-03-27T06:53:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.615+0000 7fcb0ceba640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb08100a10 0x7fcb08198500 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009230 tx=0x7fcafc02dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.616+0000 7fcb05ffb640 1 -- 172.21.15.16:0/2026917457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc043020 con 0x7fcb08100a10 2024-03-27T06:53:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.616+0000 7fcb05ffb640 1 -- 172.21.15.16:0/2026917457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcafc004250 con 0x7fcb08100a10 2024-03-27T06:53:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.616+0000 7fcb0f11e640 1 -- 172.21.15.16:0/2026917457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0819d780 con 0x7fcb08100a10 2024-03-27T06:53:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.616+0000 7fcb05ffb640 1 -- 172.21.15.16:0/2026917457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc003970 con 0x7fcb08100a10 2024-03-27T06:53:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.616+0000 7fcb0f11e640 1 -- 172.21.15.16:0/2026917457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb081a5070 con 0x7fcb08100a10 2024-03-27T06:53:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.617+0000 7fcb0f11e640 1 -- 172.21.15.16:0/2026917457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb08068d90 con 0x7fcb08100a10 2024-03-27T06:53:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.618+0000 7fcb05ffb640 1 -- 172.21.15.16:0/2026917457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fcafc046020 con 0x7fcb08100a10 2024-03-27T06:53:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.619+0000 7fcb05ffb640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcaf80a3780 0x7fcaf80a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:53:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.619+0000 7fcb07fff640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcaf80a3780 0x7fcaf80a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:53:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.619+0000 7fcb05ffb640 1 -- 172.21.15.16:0/2026917457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fcafc0f2350 con 0x7fcb08100a10 2024-03-27T06:53:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.619+0000 7fcb07fff640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcaf80a3780 0x7fcaf80a5c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcb08078210 tx=0x7fcaf4005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:53:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.624+0000 7fcb05ffb640 1 -- 172.21.15.16:0/2026917457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fcafc04d020 con 0x7fcb08100a10 2024-03-27T06:53:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.860+0000 7fcb0f11e640 1 -- 172.21.15.16:0/2026917457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fcb08199850 con 0x7fcb08100a10 2024-03-27T06:53:44.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:44 smithi016 ceph-mon[31908]: from='client.16413 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:53:44.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:44 smithi016 ceph-mon[31908]: from='client.16419 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:53:44.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:44 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/4043534848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:53:44.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:44 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/4139334442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:53:44.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:44 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2843798688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:53:44.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.869+0000 7fcb05ffb640 1 -- 172.21.15.16:0/2026917457 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+744337 (secure 0 0 0) 0x7fcb08199850 con 0x7fcb08100a10 2024-03-27T06:53:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:53:44.876 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:53:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-03-27T06:53:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.876 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 15354, 2024-03-27T06:53:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2024-03-27T06:53:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2024-03-27T06:53:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-03-27T06:53:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-03-27T06:53:44.877 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2024-03-27T06:53:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3715976676 2024-03-27T06:53:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-03-27T06:53:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2024-03-27T06:53:44.878 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3715976676 2024-03-27T06:53:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:53:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.16:6801/3715976676", 2024-03-27T06:53:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2024-03-27T06:43:33.781101+0000", 2024-03-27T06:53:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540701547738038271, 2024-03-27T06:53:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2024-03-27T06:53:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2024-03-27T06:53:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:44.879 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 25193, 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:44.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:44.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:44.884 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:44.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-03-27T06:53:44.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-03-27T06:53:44.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-03-27T06:53:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-27T06:53:44.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-03-27T06:53:44.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-03-27T06:53:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-03-27T06:53:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-03-27T06:53:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-03-27T06:53:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-27T06:53:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-03-27T06:53:44.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-03-27T06:53:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:44.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:44.904 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:44.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:44.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-27T06:53:44.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-03-27T06:53:44.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:44.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:44.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-03-27T06:53:44.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-03-27T06:53:44.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-03-27T06:53:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-03-27T06:53:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-03-27T06:53:44.922 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.926 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-03-27T06:53:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-03-27T06:53:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-03-27T06:53:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-27T06:53:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.939 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.941 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-03-27T06:53:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-03-27T06:53:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-03-27T06:53:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.945 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.948 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.949 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-27T06:53:44.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.954 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-03-27T06:53:44.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-03-27T06:53:44.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:44.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.960 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:44.961 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-03-27T06:53:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.975 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:44.983 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:44.984 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:44.986 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.991 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:44.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:44.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-03-27T06:53:45.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-03-27T06:53:45.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.009 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.010 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.017 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-27T06:53:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.022 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.028 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.029 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-27T06:53:45.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.032 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-03-27T06:53:45.036 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-03-27T06:53:45.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-03-27T06:53:45.038 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-03-27T06:53:45.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-03-27T06:53:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.044 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.045 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.049 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-03-27T06:53:45.053 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-03-27T06:53:45.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.055 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.056 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.060 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-03-27T06:53:45.062 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-03-27T06:53:45.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.066 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.072 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.080 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.083 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.085 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.086 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.088 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-03-27T06:53:45.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-03-27T06:53:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-03-27T06:53:45.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.094 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.097 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.102 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-27T06:53:45.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.107 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.116 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-03-27T06:53:45.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-03-27T06:53:45.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-03-27T06:53:45.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.124 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.128 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.129 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.137 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.138 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.140 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.142 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.144 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.145 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-03-27T06:53:45.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.151 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.153 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.160 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.161 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.162 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-03-27T06:53:45.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.171 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.172 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.175 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.178 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.182 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-27T06:53:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-03-27T06:53:45.184 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-03-27T06:53:45.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.186 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-03-27T06:53:45.187 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-03-27T06:53:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.190 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.193 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.194 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.196 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.197 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-03-27T06:53:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.202 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.205 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.206 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.209 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.214 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.215 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.216 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.219 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.220 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.224 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.225 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.228 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.230 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.238 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.239 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.241 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.246 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.249 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.251 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-03-27T06:53:45.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:45.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.262 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-03-27T06:53:45.264 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-27T06:53:45.266 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.278 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.281 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.283 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.284 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.285 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.289 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.292 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.297 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.298 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-03-27T06:53:45.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.304 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 25201, 2024-03-27T06:53:45.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "z", 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:45.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.317 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-03-27T06:53:45.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-03-27T06:53:45.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-03-27T06:53:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-03-27T06:53:45.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-03-27T06:53:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.327 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-03-27T06:53:45.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-27T06:53:45.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-27T06:53:45.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-27T06:53:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.336 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.339 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.348 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-27T06:53:45.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-27T06:53:45.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-27T06:53:45.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-27T06:53:45.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-27T06:53:45.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-27T06:53:45.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-27T06:53:45.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-27T06:53:45.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-27T06:53:45.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-27T06:53:45.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-27T06:53:45.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-03-27T06:53:45.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-27T06:53:45.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-27T06:53:45.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.391 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-03-27T06:53:45.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-03-27T06:53:45.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-03-27T06:53:45.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-03-27T06:53:45.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.414 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.416 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-03-27T06:53:45.420 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.424 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.426 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.427 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.428 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.429 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.430 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-27T06:53:45.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-27T06:53:45.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-27T06:53:45.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-27T06:53:45.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-27T06:53:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-27T06:53:45.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.446 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-27T06:53:45.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-03-27T06:53:45.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-03-27T06:53:45.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.458 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-03-27T06:53:45.481 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-03-27T06:53:45.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.486 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.504 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-03-27T06:53:45.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-03-27T06:53:45.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-03-27T06:53:45.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.513 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-03-27T06:53:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-03-27T06:53:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:45.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.522 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.524 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.527 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.537 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.539 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-27T06:53:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.541 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.547 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.548 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.549 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-03-27T06:53:45.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-03-27T06:53:45.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-03-27T06:53:45.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-03-27T06:53:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-27T06:53:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-27T06:53:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.554 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.556 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.558 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.564 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.575 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.587 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.590 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-03-27T06:53:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.602 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.603 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-03-27T06:53:45.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:45.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.606 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.607 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.608 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.609 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-03-27T06:53:45.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-27T06:53:45.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-03-27T06:53:45.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-03-27T06:53:45.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-03-27T06:53:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.618 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.619 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.624 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.625 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-03-27T06:53:45.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.635 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.639 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.645 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.650 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.667 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.673 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-03-27T06:53:45.675 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.679 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.682 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.683 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.684 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.686 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.687 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.689 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-27T06:53:45.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-27T06:53:45.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.705 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.713 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.714 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.716 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.717 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.718 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.721 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.724 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-03-27T06:53:45.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-03-27T06:53:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-03-27T06:53:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-03-27T06:53:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.732 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.734 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard", 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "restful", 2024-03-27T06:53:45.741 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_schedule" 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:45.742 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.750 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-03-27T06:53:45.756 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-03-27T06:53:45.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-27T06:53:45.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-03-27T06:53:45.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-03-27T06:53:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-03-27T06:53:45.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.766 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-03-27T06:53:45.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.769 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-03-27T06:53:45.774 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.775 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:45.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:45.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.779 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-03-27T06:53:45.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-03-27T06:53:45.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-03-27T06:53:45.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-03-27T06:53:45.783 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-03-27T06:53:45.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.786 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.790 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.792 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.796 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.797 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.798 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-03-27T06:53:45.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-03-27T06:53:45.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-03-27T06:53:45.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-03-27T06:53:45.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-03-27T06:53:45.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-03-27T06:53:45.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-03-27T06:53:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-03-27T06:53:45.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-03-27T06:53:45.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.810 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.811 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.813 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.814 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-27T06:53:45.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.820 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.827 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-03-27T06:53:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.831 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.834 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.836 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.842 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.843 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.845 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.847 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.856 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.860 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.865 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.867 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.868 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.870 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.877 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.885 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-03-27T06:53:45.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.906 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-03-27T06:53:45.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-03-27T06:53:45.910 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.913 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.915 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.917 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.918 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-27T06:53:45.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.926 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.930 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.932 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-03-27T06:53:45.935 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.939 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.940 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.941 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.943 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-03-27T06:53:45.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-03-27T06:53:45.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.946 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.947 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.949 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.950 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.952 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.953 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.956 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.957 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.961 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-03-27T06:53:45.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.968 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.970 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.972 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:45.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.981 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.982 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.985 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.989 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.992 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.995 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.996 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:45.997 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:45.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.004 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.007 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.008 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.009 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.011 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.017 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.019 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:46.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-03-27T06:53:46.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.026 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.031 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.032 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.034 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.035 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.038 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.040 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:46.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.042 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-03-27T06:53:46.047 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.048 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.052 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.055 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.058 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.061 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-03-27T06:53:46.065 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.067 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.067 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.067 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.067 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.067 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.067 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.067 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.068 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.069 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.071 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.072 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.080 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.081 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-03-27T06:53:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-03-27T06:53:46.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.088 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.095 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-03-27T06:53:46.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.098 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.102 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.106 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-03-27T06:53:46.108 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.113 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-03-27T06:53:46.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:46.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.119 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.120 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-27T06:53:46.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.135 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.144 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.146 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.149 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-03-27T06:53:46.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-03-27T06:53:46.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-03-27T06:53:46.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-03-27T06:53:46.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-27T06:53:46.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-03-27T06:53:46.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-03-27T06:53:46.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-03-27T06:53:46.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: "services": { 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard": "https://172.21.15.16:8443/" 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2024-03-27T06:53:46.171 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-03-27T06:53:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-03-27T06:53:46.173 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "reef": [ 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-03-27T06:53:46.174 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "squid": [ 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-03-27T06:53:46.175 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 89, 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 960688324 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.176 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1923373083 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2029963617 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1296194547 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.178 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3297833414 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-03-27T06:53:46.179 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-03-27T06:53:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-03-27T06:53:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-03-27T06:53:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 106600817 2024-03-27T06:53:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:53:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:46.180 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-27T06:53:46.180 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:53:46.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:44 smithi170 ceph-mon[36438]: from='client.16413 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:53:46.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:44 smithi170 ceph-mon[36438]: from='client.16419 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:53:46.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:44 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/4043534848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:53:46.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:44 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/4139334442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:53:46.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:44 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2843798688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:53:46.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:45 smithi170 ceph-mon[36438]: pgmap v317: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:46.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:45 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2026917457' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-27T06:53:46.182 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:44 smithi156 ceph-mon[36261]: from='client.16413 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:53:46.182 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:44 smithi156 ceph-mon[36261]: from='client.16419 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-27T06:53:46.182 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:44 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/4043534848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:53:46.182 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:44 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/4139334442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:53:46.183 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:44 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2843798688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:53:46.183 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:45 smithi156 ceph-mon[36261]: pgmap v317: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:46.183 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:45 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2026917457' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-27T06:53:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 -- 172.21.15.16:0/2026917457 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcaf80a3780 msgr2=0x7fcaf80a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcaf80a3780 0x7fcaf80a5c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcb08078210 tx=0x7fcaf4005f90 comp rx=0 tx=0).stop 2024-03-27T06:53:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 -- 172.21.15.16:0/2026917457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb08100a10 msgr2=0x7fcb08198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:53:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb08100a10 0x7fcb08198500 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009230 tx=0x7fcafc02dc90 comp rx=0 tx=0).stop 2024-03-27T06:53:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 -- 172.21.15.16:0/2026917457 shutdown_connections 2024-03-27T06:53:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fcaf80a3780 0x7fcaf80a5c40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb08106a20 0x7fcb0819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb081013d0 0x7fcb08198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 --2- 172.21.15.16:0/2026917457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb08100a10 0x7fcb08198500 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:53:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 -- 172.21.15.16:0/2026917457 >> 172.21.15.16:0/2026917457 conn(0x7fcb08077af0 msgr2=0x7fcb080feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:53:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 -- 172.21.15.16:0/2026917457 shutdown_connections 2024-03-27T06:53:46.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:53:44.875+0000 7fcb0f11e640 1 -- 172.21.15.16:0/2026917457 wait complete. 2024-03-27T06:53:46.190 DEBUG:tasks.check_counter:Getting stats from a 2024-03-27T06:53:46.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-27T06:53:46.191 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:45 smithi016 ceph-mon[31908]: pgmap v317: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:46.191 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:45 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2026917457' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-27T06:53:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:53:47.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:47 smithi016 ceph-mon[31908]: pgmap v318: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-27T06:53:47.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:47 smithi156 ceph-mon[36261]: pgmap v318: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-27T06:53:48.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:47 smithi170 ceph-mon[36438]: pgmap v318: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-27T06:53:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:53:48.442 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:53:48.442 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1513, 2024-03-27T06:53:48.442 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1513, 2024-03-27T06:53:48.442 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1867692, 2024-03-27T06:53:48.442 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 480311, 2024-03-27T06:53:48.443 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-03-27T06:53:48.443 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-03-27T06:53:48.443 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.140231024, 2024-03-27T06:53:48.443 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.047545701, 2024-03-27T06:53:48.443 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.037319610, 2024-03-27T06:53:48.443 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.042084022, 2024-03-27T06:53:48.443 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:48.443 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1512, 2024-03-27T06:53:48.443 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.056411284, 2024-03-27T06:53:48.444 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000037309 2024-03-27T06:53:48.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.444 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:48.444 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.444 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.444 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.444 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-27T06:53:48.444 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-03-27T06:53:48.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.445 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:53:48.445 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 616, 2024-03-27T06:53:48.445 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 644, 2024-03-27T06:53:48.445 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 152102, 2024-03-27T06:53:48.445 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 323174, 2024-03-27T06:53:48.445 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2024-03-27T06:53:48.445 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 5, 2024-03-27T06:53:48.445 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.063883442, 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.018675566, 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.016010555, 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.022735586, 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 641, 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.023576265, 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000036780 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.446 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.447 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-27T06:53:48.447 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 127536 2024-03-27T06:53:48.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.448 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:53:48.448 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 264, 2024-03-27T06:53:48.448 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 343, 2024-03-27T06:53:48.448 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1698444, 2024-03-27T06:53:48.448 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 434686, 2024-03-27T06:53:48.448 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-03-27T06:53:48.448 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-03-27T06:53:48.448 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.051542023, 2024-03-27T06:53:48.448 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.020755487, 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.034994642, 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.004137895, 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 342, 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.025667312, 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000075050 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1698176, 2024-03-27T06:53:48.449 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 434368 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3442, 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.099984173, 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000029048 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.451 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 187, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 187, 2024-03-27T06:53:48.452 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 187, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-03-27T06:53:48.453 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-03-27T06:53:48.454 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-03-27T06:53:48.455 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-03-27T06:53:48.456 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-03-27T06:53:48.457 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 632, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-03-27T06:53:48.458 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 5610189, 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 5608101, 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.460 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 632 2024-03-27T06:53:48.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.460 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-03-27T06:53:48.460 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 4, 2024-03-27T06:53:48.460 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 260, 2024-03-27T06:53:48.460 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 256, 2024-03-27T06:53:48.460 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 2, 2024-03-27T06:53:48.460 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 103, 2024-03-27T06:53:48.460 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 101, 2024-03-27T06:53:48.460 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 187, 2024-03-27T06:53:48.460 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 427, 2024-03-27T06:53:48.461 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 240, 2024-03-27T06:53:48.461 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-03-27T06:53:48.461 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-03-27T06:53:48.461 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-03-27T06:53:48.461 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 51604, 2024-03-27T06:53:48.461 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215416 2024-03-27T06:53:48.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.461 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-03-27T06:53:48.461 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.462 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.463 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.463 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-03-27T06:53:48.463 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.463 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.463 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.464 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-03-27T06:53:48.464 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.464 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.464 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.465 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-03-27T06:53:48.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.465 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.465 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-03-27T06:53:48.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.465 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.466 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-03-27T06:53:48.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.466 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.466 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-03-27T06:53:48.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.466 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-03-27T06:53:48.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.468 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.468 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.468 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-03-27T06:53:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-03-27T06:53:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-03-27T06:53:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-03-27T06:53:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-03-27T06:53:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-03-27T06:53:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.472 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.473 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.473 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-03-27T06:53:48.473 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.473 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.473 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.473 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-03-27T06:53:48.473 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.475 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-03-27T06:53:48.476 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 3, 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 2, 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 3, 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-03-27T06:53:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 3886, 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:53:48.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:53:48.479 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:53:48.480 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 197553, 2024-03-27T06:53:48.481 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 57, 2024-03-27T06:53:48.481 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-03-27T06:53:48.481 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-03-27T06:53:48.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-03-27T06:53:48.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-03-27T06:53:48.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:53:48.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-03-27T06:53:48.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:53:48.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-03-27T06:53:48.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8032, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 167, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 146808, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 783, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.482 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 2125, 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 2125, 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2125, 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.983969151, 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000933632 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-03-27T06:53:48.483 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2125, 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2126 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 2125, 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 2125, 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 1343, 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 780, 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-03-27T06:53:48.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-03-27T06:53:48.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-03-27T06:53:48.486 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 98, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 12, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 6, 2024-03-27T06:53:48.487 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 6, 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-03-27T06:53:48.488 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:48.489 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 2389, 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 3486866, 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 2389, 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2389, 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 3486866, 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:48.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2125, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 1415890, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2125, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1415890, 2024-03-27T06:53:48.491 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-27T06:53:48.492 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2125, 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2125, 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2125, 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2125, 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-03-27T06:53:48.493 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55fbb4fc06e0": { 2024-03-27T06:53:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-03-27T06:53:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-27T06:53:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-27T06:53:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-27T06:53:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-27T06:53:48.496 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-27T06:53:48.496 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-03-27T06:53:48.496 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-03-27T06:53:48.496 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:48.496 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:48.496 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:48.496 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:48.496 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:48.496 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:48.496 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:53:48.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:48 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:53:48.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:48 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:53:48.993 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:48 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-27T06:53:49.207 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-27T06:53:49.207 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-27T06:53:49.207 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-03-27T06:53:49.207 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-03-27T06:53:49.207 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-03-27T06:53:49.207 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-27T06:53:49.207 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-27T06:53:49.207 DEBUG:tasks.check_counter:Getting stats from d 2024-03-27T06:53:49.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-27T06:53:49.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:49 smithi016 ceph-mon[31908]: pgmap v319: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T06:53:49.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:49 smithi156 ceph-mon[36261]: pgmap v319: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T06:53:50.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:49 smithi170 ceph-mon[36438]: pgmap v319: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T06:53:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:53:51.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: pgmap v320: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi170 to 174.1M 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: Unable to set osd_memory_target on smithi170 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.223 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-27T06:53:51.224 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:50 smithi156 ceph-mon[36261]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-27T06:53:51.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: pgmap v320: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-27T06:53:51.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi170 to 174.1M 2024-03-27T06:53:51.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: Unable to set osd_memory_target on smithi170 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-03-27T06:53:51.259 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.259 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.259 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.259 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.259 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-27T06:53:51.259 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:50 smithi016 ceph-mon[31908]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-27T06:53:51.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:51.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: pgmap v320: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi170 to 174.1M 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: Unable to set osd_memory_target on smithi170 to 182639206: error parsing value: Value '182639206' is below minimum 939524096 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi156 to 174.1M 2024-03-27T06:53:51.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:50 smithi170 ceph-mon[36438]: Unable to set osd_memory_target on smithi156 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-27T06:53:51.915 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:53:51.915 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:53:51.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-03-27T06:53:51.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-03-27T06:53:51.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-03-27T06:53:51.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-03-27T06:53:51.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-03-27T06:53:51.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-03-27T06:53:51.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001195930, 2024-03-27T06:53:51.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-03-27T06:53:51.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000239121, 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-03-27T06:53:51.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 217, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 207, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1447312, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 127264, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.040578403, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.013019972, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.031530742, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.004257323, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 206, 2024-03-27T06:53:51.917 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.009735423, 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000047259 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1447312, 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 127264 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 6, 2024-03-27T06:53:51.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 160, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3392, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 183984, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.013298700, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.009036910, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002075289, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 158, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.017546518, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000111053 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 183984 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:53:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:53:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:53:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148977, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-03-27T06:53:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 219, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1416699, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 219, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 219, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1416699, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:51.925 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:53:52.816 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-27T06:53:52.817 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-27T06:53:52.817 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-27T06:53:52.817 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-27T06:53:52.817 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-27T06:53:52.817 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-27T06:53:52.817 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-27T06:53:52.817 DEBUG:tasks.check_counter:Getting stats from g 2024-03-27T06:53:52.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-27T06:53:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:53:53.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:53 smithi016 ceph-mon[31908]: pgmap v321: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T06:53:53.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:53 smithi170 ceph-mon[36438]: pgmap v321: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T06:53:53.474 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:53 smithi156 ceph-mon[36261]: pgmap v321: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T06:53:54.249 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:54 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:54.249 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:54 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:54.249 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:54 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.250 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:54 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.250 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:54 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.250 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:54 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.250 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:54 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:53:54.250 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:54 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:53:54.250 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:54 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:54.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:54 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:54.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:54 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:54.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:54 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:54 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:54 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:54 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:54 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:53:54.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:54 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:53:54.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:54 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:54.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:54 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:54.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:54 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:54.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:54 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:54 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:54 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:54 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-27T06:53:54.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:54 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-27T06:53:54.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:54 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-27T06:53:54.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:54 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' 2024-03-27T06:53:54.798 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:53:54.798 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:53:54.798 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 738, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 740, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 247913, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 234685, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.080550665, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.034163236, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.016758303, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.020937751, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 738, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.038205527, 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000051769 2024-03-27T06:53:54.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1663, 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1617, 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2544172, 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 572085, 2024-03-27T06:53:54.800 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.130033247, 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.037367730, 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.059600742, 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.037278036, 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1617, 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.050410726, 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000031175 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1701904, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 128736 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 5, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 158, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1668, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 438469, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 5, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.012835139, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.008097612, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002766255, 2024-03-27T06:53:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000055531, 2024-03-27T06:53:54.803 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:54.803 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 155, 2024-03-27T06:53:54.803 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.015266990, 2024-03-27T06:53:54.803 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000098496 2024-03-27T06:53:54.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.803 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:54.803 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.803 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.803 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.803 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-27T06:53:54.804 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 437504 2024-03-27T06:53:54.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.804 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-03-27T06:53:54.804 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-03-27T06:53:54.804 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-03-27T06:53:54.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.804 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-03-27T06:53:54.804 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-03-27T06:53:54.804 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3625, 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.086762803, 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000023934 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:54.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-03-27T06:53:54.806 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 80, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 80, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 80, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-03-27T06:53:54.807 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-03-27T06:53:54.808 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-03-27T06:53:54.809 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-03-27T06:53:54.810 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T06:53:54.811 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 217, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-03-27T06:53:54.812 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4681792, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4679704, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 217 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 19, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 111, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 92, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 4, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 59, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 55, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 80, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 171, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 91, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-03-27T06:53:54.814 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 51272, 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215416 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.816 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-03-27T06:53:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.818 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-03-27T06:53:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-03-27T06:53:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-03-27T06:53:54.819 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.820 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.820 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.820 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-03-27T06:53:54.820 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.820 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.820 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.820 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.821 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 2, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 3, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 1, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 2, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 2802, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-03-27T06:53:54.827 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:53:54.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-03-27T06:53:54.829 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 198058, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 72, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:53:54.830 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8032, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 167, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 111169, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 1061, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-03-27T06:53:54.831 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 2135, 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 2135, 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2135, 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2.643958913, 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001238388 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2135, 2024-03-27T06:53:54.832 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2136 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 2135, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 2135, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 1355, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 778, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-03-27T06:53:54.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T06:53:54.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-03-27T06:53:54.835 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 98, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 12, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 6, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 6, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-03-27T06:53:54.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-03-27T06:53:54.837 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 2402, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 2561055, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 2402, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-27T06:53:54.838 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2402, 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2561055, 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-27T06:53:54.839 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2135, 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 487493, 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2135, 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 487493, 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:54.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2135, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2135, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2135, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2135, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.841 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-03-27T06:53:54.842 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55bbcfec06e0": { 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-27T06:53:54.843 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-27T06:53:54.844 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-03-27T06:53:54.844 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-03-27T06:53:54.844 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:54.844 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:54.844 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:54.844 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:54.844 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:54.844 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:54.844 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:53:55.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:55 smithi170 ceph-mon[36438]: pgmap v322: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T06:53:55.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:55 smithi170 ceph-mon[36438]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-03-27T06:53:55.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:55 smithi170 ceph-mon[36438]: Unable to set osd_memory_target on smithi016 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-03-27T06:53:55.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:55 smithi016 ceph-mon[31908]: pgmap v322: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T06:53:55.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:55 smithi016 ceph-mon[31908]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-03-27T06:53:55.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:55 smithi016 ceph-mon[31908]: Unable to set osd_memory_target on smithi016 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-03-27T06:53:55.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:55 smithi156 ceph-mon[36261]: pgmap v322: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-27T06:53:55.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:55 smithi156 ceph-mon[36261]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-03-27T06:53:55.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:55 smithi156 ceph-mon[36261]: Unable to set osd_memory_target on smithi016 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-03-27T06:53:55.586 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-27T06:53:55.586 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-27T06:53:55.587 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-03-27T06:53:55.587 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-03-27T06:53:55.587 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-03-27T06:53:55.587 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-27T06:53:55.587 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-27T06:53:55.587 DEBUG:tasks.check_counter:Getting stats from j 2024-03-27T06:53:55.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-27T06:53:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.a/config 2024-03-27T06:53:57.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:57 smithi016 ceph-mon[31908]: pgmap v323: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-27T06:53:57.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:57 smithi170 ceph-mon[36438]: pgmap v323: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-27T06:53:57.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:57 smithi156 ceph-mon[36261]: pgmap v323: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-27T06:53:57.568 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-27T06:53:57.568 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:53:57.568 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-03-27T06:53:57.568 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-03-27T06:53:57.568 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-03-27T06:53:57.568 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-03-27T06:53:57.568 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-03-27T06:53:57.568 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001330215, 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000440391, 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:57.569 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 223, 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 211, 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1450928, 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 128992, 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.041418750, 2024-03-27T06:53:57.570 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.013382783, 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.033272055, 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.004071836, 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 211, 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.010660804, 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000050525 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:57.571 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1450928, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 128992 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 160, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 183888, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.014388644, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.009773208, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002298037, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:53:57.572 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 157, 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.012263553, 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000078111 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 183888 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-03-27T06:53:57.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:53:57.574 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:53:57.575 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 150008, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 31, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-03-27T06:53:57.576 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-03-27T06:53:57.577 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 221, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1416959, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 221, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 221, 2024-03-27T06:53:57.578 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1416959, 2024-03-27T06:53:57.579 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-27T06:53:57.579 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-27T06:53:57.579 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-27T06:53:57.579 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-27T06:53:57.579 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:57.579 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-27T06:53:57.579 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-27T06:53:58.585 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-27T06:53:58.585 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-27T06:53:58.585 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-27T06:53:58.585 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-27T06:53:58.585 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-27T06:53:58.585 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-27T06:53:58.585 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-27T06:53:58.585 DEBUG:tasks.check_counter:Getting stats from b 2024-03-27T06:53:58.586 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-27T06:53:58.872 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:53:59.267 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:53:59 smithi156 ceph-mon[36261]: pgmap v324: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:59.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:53:59 smithi170 ceph-mon[36438]: pgmap v324: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:53:59.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:53:59 smithi016 ceph-mon[31908]: pgmap v324: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:54:00.766 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 9011, 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 8949, 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 4919483, 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 30640374, 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 70, 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 8, 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.883381660, 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.246415314, 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.220334048, 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.376848787, 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 8941, 2024-03-27T06:54:00.767 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 9.537447192, 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001066709 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 333, 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000039029, 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000117 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1824480, 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 487536 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 53003, 2024-03-27T06:54:00.768 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 92054, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 18536362, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 71739385, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 13, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 3.617520782, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 1.723589508, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 1.950733865, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.407057086, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 92059, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 15.576646612, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000169202 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.769 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 89287, 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.019142236, 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000214 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 45792, 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 134000 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 15981, 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 21887, 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 5064914, 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 41840651, 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 16, 2024-03-27T06:54:00.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 1.699627587, 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.806843667, 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.569275384, 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.317431505, 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 21885, 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 7.300864767, 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000333601 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 24694, 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.005460330, 2024-03-27T06:54:00.771 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000221 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 360352, 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 98512 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4041, 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 33.437274691, 2024-03-27T06:54:00.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.008274504 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 11793, 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 4.392069599, 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000372430 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 17839, 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 17815, 2024-03-27T06:54:00.773 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 17815, 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 58.766613858, 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003298715 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 24, 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 2, 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 1, 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 1487, 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 8, 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 1, 2024-03-27T06:54:00.774 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 5248, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 3530, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 1530, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 188, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 205, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 2671, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 3, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 3, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 9, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 19071, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 18332, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 1, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 322, 2024-03-27T06:54:00.775 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 93, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 1968, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 25, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 299, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 3904, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 4, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 234, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 6, 2024-03-27T06:54:00.776 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 3904, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 336, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 4, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 332, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 3568, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 42, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 1203, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 17392, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T06:54:00.777 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 10605, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 8739, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 140, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 8837 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 1278, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 868, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 591, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 548, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 2705, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 180, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-03-27T06:54:00.778 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 7461, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 7281, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_quiesce_path": 0, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_quiesce_inode": 0, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 4, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 94, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 9, 2024-03-27T06:54:00.779 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 38245, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 38166, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 38166, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 79, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 170, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 61, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 60, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 60, 2024-03-27T06:54:00.780 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 1, 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 1, 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 1, 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 87381963, 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 87531319, 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1433, 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 112.652226277, 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.078612858 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.781 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 2495, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 11976, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 9481, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 249, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 858, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 609, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 5248, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 16530, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 11282, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 3, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 9431, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 9428, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 105504, 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 231800 2024-03-27T06:54:00.782 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 20435, 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 487, 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 17839, 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 124, 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 1227, 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 8904, 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 20.165881880, 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002264811 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 129, 2024-03-27T06:54:00.783 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.057772709, 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000447850 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 139, 2024-03-27T06:54:00.784 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 3.334452993, 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.023988870 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000164836, 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000164836 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-03-27T06:54:00.785 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.786 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.786 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.786 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.786 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-03-27T06:54:00.786 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.786 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.786 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.786 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.787 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-03-27T06:54:00.787 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.787 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.787 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 11, 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.002009046, 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000182640 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 467, 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.946032108, 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002025764 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-03-27T06:54:00.788 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 11, 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.112168901, 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.010197172 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000303974, 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000303974 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.789 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 650, 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.918172211, 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002951034 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2, 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.002144878, 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001072439 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.790 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 308, 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 9.816313525, 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.031871147 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 5, 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.021527838, 2024-03-27T06:54:00.791 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004305567 2024-03-27T06:54:00.792 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.792 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-03-27T06:54:00.792 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.792 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.792 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.792 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.792 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-03-27T06:54:00.792 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 26, 2024-03-27T06:54:00.792 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 5.964217216, 2024-03-27T06:54:00.792 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.229392969 2024-03-27T06:54:00.792 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.793 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.794 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-03-27T06:54:00.794 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 8, 2024-03-27T06:54:00.794 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.017287835, 2024-03-27T06:54:00.794 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002160979 2024-03-27T06:54:00.794 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.794 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-03-27T06:54:00.794 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.794 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.794 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.794 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7153, 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 16.408163908, 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002293885 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.795 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 6, 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 4, 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 84, 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 42, 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 612, 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-03-27T06:54:00.796 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-03-27T06:54:00.797 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-03-27T06:54:00.798 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 1857952, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 2969, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 1760, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 20, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-03-27T06:54:00.799 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8032, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 167, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 10138642, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 154171, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-03-27T06:54:00.800 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 24149, 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 100519543, 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 37, 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 24149, 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 24149, 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1983.673808156, 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.082143103 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-03-27T06:54:00.801 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-03-27T06:54:00.802 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 24149, 2024-03-27T06:54:00.802 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 44804 2024-03-27T06:54:00.802 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.802 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 24149, 2024-03-27T06:54:00.802 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 4061, 2024-03-27T06:54:00.802 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 20088, 2024-03-27T06:54:00.802 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-03-27T06:54:00.802 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 1054, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 8880, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 150, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 1579, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 132, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 8, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 7877, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-03-27T06:54:00.803 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 3911, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 17760, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-03-27T06:54:00.804 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-03-27T06:54:00.805 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-03-27T06:54:00.805 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-03-27T06:54:00.805 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-03-27T06:54:00.805 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-03-27T06:54:00.805 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-03-27T06:54:00.805 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-03-27T06:54:00.805 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-03-27T06:54:00.805 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-03-27T06:54:00.805 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-03-27T06:54:00.805 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-03-27T06:54:00.806 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-03-27T06:54:00.806 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-03-27T06:54:00.806 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-03-27T06:54:00.806 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-03-27T06:54:00.806 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 98, 2024-03-27T06:54:00.806 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-03-27T06:54:00.806 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 13, 2024-03-27T06:54:00.806 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2024-03-27T06:54:00.806 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2024-03-27T06:54:00.806 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-03-27T06:54:00.806 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 3146, 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 6, 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 301 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-03-27T06:54:00.807 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 20, 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 6, 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 7319, 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 7281, 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-27T06:54:00.808 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 77991, 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 21889209, 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 77991, 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 77991, 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 21889209, 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:00.809 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 24149, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 101272128, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 24149, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 101272128, 2024-03-27T06:54:00.810 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:00.811 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 24149, 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 24149, 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 24149, 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 24149, 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:00.812 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 7281, 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 752621, 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 7281, 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 146, 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 752621, 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x5579b7a18500": { 2024-03-27T06:54:00.813 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 84, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 38245, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 83337015, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 38245, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 1433, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 83336931, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:00.814 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:00.815 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:00.815 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:00.815 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:00.815 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-27T06:54:01.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:01 smithi170 ceph-mon[36438]: pgmap v325: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:54:01.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:01 smithi016 ceph-mon[31908]: pgmap v325: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:54:01.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:01 smithi156 ceph-mon[36261]: pgmap v325: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-27T06:54:01.575 INFO:tasks.check_counter:Daemon mds.b mds.exported=93 2024-03-27T06:54:01.575 INFO:tasks.check_counter:Daemon mds.b mds.imported=25 2024-03-27T06:54:01.575 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1278 2024-03-27T06:54:01.575 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=868 2024-03-27T06:54:01.575 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-27T06:54:01.575 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-03-27T06:54:01.575 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-03-27T06:54:01.575 DEBUG:tasks.check_counter:Getting stats from e 2024-03-27T06:54:01.576 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-27T06:54:01.913 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:54:03.222 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:03 smithi156 ceph-mon[36261]: pgmap v326: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:54:03.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:03 smithi170 ceph-mon[36438]: pgmap v326: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:54:03.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:03 smithi016 ceph-mon[31908]: pgmap v326: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:54:03.868 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-27T06:54:03.868 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:54:03.868 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 14364, 2024-03-27T06:54:03.868 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 7323, 2024-03-27T06:54:03.868 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 6574344, 2024-03-27T06:54:03.868 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 2377875, 2024-03-27T06:54:03.868 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 18, 2024-03-27T06:54:03.868 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2024-03-27T06:54:03.868 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.687214861, 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.281363968, 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.536495348, 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.006702224, 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7322, 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.158996999, 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000021714 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 21124, 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.003116169, 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000147 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.869 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 630, 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 844, 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 326198, 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 842836, 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 17, 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.046379892, 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.019721153, 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.028673057, 2024-03-27T06:54:03.870 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.001106075, 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 841, 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.040295461, 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000047913 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 996, 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000171696, 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000172 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 444512 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.871 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 388, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 363, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1731611, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 194136, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 8, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.044359975, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.010973528, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.040156972, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.005930626, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 362, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.025563111, 2024-03-27T06:54:03.872 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000070616 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 34, 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000006769, 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000199 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1711632, 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 152560 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-03-27T06:54:03.873 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 244, 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.078534497, 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000321862 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 155, 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.006269077, 2024-03-27T06:54:03.874 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000040445 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 107, 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 106, 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 106, 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.231753979, 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002186358 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 1, 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-03-27T06:54:03.875 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 11, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 62, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 11, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 39, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 12, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 43, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 1237, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 6, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 1756, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 907, 2024-03-27T06:54:03.876 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 770, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 8, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 8, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 10, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 70, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 9, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 3713, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 28613248, 2024-03-27T06:54:03.877 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 3, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 9, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 2, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 2, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 7, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 2, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 19, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 77, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T06:54:03.878 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 30, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 15, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 9, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 106 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 50, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 2175, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 1462, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 1393, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 58, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 2, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-03-27T06:54:03.879 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 51, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 49, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_quiesce_path": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_quiesce_inode": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 8, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-03-27T06:54:03.880 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 217, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 217, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 2, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2024-03-27T06:54:03.881 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 2, 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 2, 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 4681792, 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 79, 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.670663721, 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.021147642 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-03-27T06:54:03.882 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 54, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 722, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 668, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 40, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 783, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 743, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 62, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 1387, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 1325, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 59, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 59, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 54228, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 223608 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-03-27T06:54:03.883 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 203, 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 96, 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 107, 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 21, 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 263, 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7, 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.012235075, 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001747867 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.884 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 34, 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.012872845, 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000378613 2024-03-27T06:54:03.885 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.886 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.887 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.888 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.888 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.888 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-03-27T06:54:03.888 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.888 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.889 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 13, 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.007106561, 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000546658 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.890 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2, 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.032345650, 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.016172825 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-03-27T06:54:03.891 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.049656667, 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.049656667 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-03-27T06:54:03.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.893 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 49, 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.117537181, 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002398717 2024-03-27T06:54:03.894 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 3, 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 1, 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-03-27T06:54:03.895 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 86, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 43, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 598, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:54:03.896 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:54:03.897 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:54:03.898 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 336496, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 135, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8032, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 167, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:54:03.899 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 244350, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 1538, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 256, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 517696, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 33, 2024-03-27T06:54:03.900 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 256, 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 256, 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2.468964982, 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.009644394 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 256, 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 311 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 256, 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 25, 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 231, 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-03-27T06:54:03.901 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 0, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 10, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 16, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 95, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 25, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 61, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 9, 2024-03-27T06:54:03.902 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 20, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-03-27T06:54:03.903 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 98, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 12, 2024-03-27T06:54:03.904 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 59, 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 0, 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 16 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-03-27T06:54:03.905 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 6, 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 6, 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 49, 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 49, 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:03.906 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 15378, 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 7231701, 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 15378, 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 15378, 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 7231701, 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-03-27T06:54:03.907 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 256, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 522669, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 256, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 522669, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.908 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 256, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 256, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 256, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 256, 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:03.909 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 49, 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 4973, 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 49, 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:03.910 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 16, 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 4973, 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x561e3bcac500": { 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 217, 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 487488, 2024-03-27T06:54:03.911 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 217, 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 79, 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 487488, 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:03.912 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-27T06:54:04.662 INFO:tasks.check_counter:Daemon mds.e mds.exported=8 2024-03-27T06:54:04.662 INFO:tasks.check_counter:Daemon mds.e mds.imported=10 2024-03-27T06:54:04.662 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=50 2024-03-27T06:54:04.663 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=2175 2024-03-27T06:54:04.663 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=3 2024-03-27T06:54:04.663 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-27T06:54:04.663 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-27T06:54:04.663 DEBUG:tasks.check_counter:Getting stats from h 2024-03-27T06:54:04.663 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-27T06:54:04.928 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:54:05.306 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:05 smithi156 ceph-mon[36261]: pgmap v327: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:54:05.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:05 smithi170 ceph-mon[36438]: pgmap v327: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:54:05.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:05 smithi016 ceph-mon[31908]: pgmap v327: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-27T06:54:06.724 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-27T06:54:06.724 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1788, 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2826, 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 2242541, 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 3877545, 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 23, 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 8, 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.164056245, 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.064060753, 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.110607688, 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.010604750, 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:06.725 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2820, 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.097918758, 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000034722 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2217, 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000587167, 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000264 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1727808, 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 176624 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:54:06.726 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 7767, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 4926, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 3351352, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1935878, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 12, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.445472968, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.200773803, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.311927800, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.006650634, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 4929, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.141373514, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000028681 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.727 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 11956, 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.002065047, 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000172 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 445216 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 6120, 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 3133, 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 3020731, 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 740842, 2024-03-27T06:54:06.728 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 9, 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.258469688, 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.096497964, 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.205329106, 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.007390987, 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3131, 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.079618618, 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000025429 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 8618, 2024-03-27T06:54:06.729 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.001265637, 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000146 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 5872 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 392, 2024-03-27T06:54:06.730 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.545842846, 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001392456 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 568, 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.043233831, 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000076115 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.731 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 588, 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 586, 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 586, 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.221249875, 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002084044 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 2, 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 11, 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-03-27T06:54:06.732 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 47, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 11, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 11, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 25, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 28, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 1376, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 6, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 2647, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 1470, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 736, 2024-03-27T06:54:06.733 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 14, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 236, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 14, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 167, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 125, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 3713, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 28613248, 2024-03-27T06:54:06.734 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 3, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 125, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 1, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 1, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 124, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 27, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 441, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T06:54:06.735 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 278, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 261, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 21, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 580 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 100, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 2086, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 1401, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 1338, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 107, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 1, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-03-27T06:54:06.736 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 291, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 290, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_quiesce_path": 0, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_quiesce_inode": 0, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 14, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-03-27T06:54:06.737 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 1336, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 1336, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 3, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2024-03-27T06:54:06.738 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 3, 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 2, 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 6557835, 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 112, 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.810715824, 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.007238534 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2024-03-27T06:54:06.739 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 47, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 1081, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 1034, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 39, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 782, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 743, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 47, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 1755, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 1708, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 311, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 311, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 58004, 2024-03-27T06:54:06.740 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 223608 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 1039, 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 57, 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 588, 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 25, 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 522, 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 124, 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.217579003, 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001754669 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-03-27T06:54:06.741 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000152443, 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000152443 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.742 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 146, 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.074295459, 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000508873 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.743 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.744 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-03-27T06:54:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.745 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.746 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 24, 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.020747252, 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000864468 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.747 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.748 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.021974538, 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.021974538 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.749 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-03-27T06:54:06.750 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.751 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 290, 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.886501180, 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003056900 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-03-27T06:54:06.752 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 4, 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 2, 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 332, 2024-03-27T06:54:06.753 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 166, 2024-03-27T06:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 602, 2024-03-27T06:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-03-27T06:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-03-27T06:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-03-27T06:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:54:06.754 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:54:06.755 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:54:06.756 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 293221, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 139, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 352, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 4, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8032, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 167, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:54:06.757 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 223297, 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 1076, 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-03-27T06:54:06.758 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 696, 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 2442552, 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 33, 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 696, 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 696, 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2.192968944, 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003150817 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 696, 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 764 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 696, 2024-03-27T06:54:06.759 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 157, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 539, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 0, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 10, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 32, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 144, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 33, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-03-27T06:54:06.760 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 302, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 125, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 20, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-03-27T06:54:06.761 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-03-27T06:54:06.762 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 98, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 12, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 75, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 0, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 23 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.763 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 6, 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-03-27T06:54:06.764 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 6, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 290, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 290, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 15671, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 7200930, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 15671, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:06.765 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 15671, 2024-03-27T06:54:06.766 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 7200930, 2024-03-27T06:54:06.766 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:06.766 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.766 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.766 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.766 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:06.766 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.766 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-03-27T06:54:06.766 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 696, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 2472834, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 696, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2472834, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.767 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 696, 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 696, 2024-03-27T06:54:06.768 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 696, 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 696, 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 290, 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 30282, 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:06.769 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 290, 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 32, 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 30282, 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55b3e5832500": { 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-27T06:54:06.770 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 1336, 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 2363531, 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 1336, 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 112, 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2363531, 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:06.771 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-27T06:54:07.345 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:07 smithi170 ceph-mon[36438]: pgmap v328: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:54:07.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:07 smithi016 ceph-mon[31908]: pgmap v328: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:54:07.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:07 smithi156 ceph-mon[36261]: pgmap v328: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:54:07.536 INFO:tasks.check_counter:Daemon mds.h mds.exported=14 2024-03-27T06:54:07.537 INFO:tasks.check_counter:Daemon mds.h mds.imported=14 2024-03-27T06:54:07.537 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=100 2024-03-27T06:54:07.537 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=2086 2024-03-27T06:54:07.537 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=3 2024-03-27T06:54:07.537 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-27T06:54:07.537 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-27T06:54:07.537 DEBUG:tasks.check_counter:Getting stats from k 2024-03-27T06:54:07.537 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-27T06:54:07.840 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.b/config 2024-03-27T06:54:09.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:09 smithi170 ceph-mon[36438]: pgmap v329: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:54:09.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:09 smithi016 ceph-mon[31908]: pgmap v329: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:54:09.367 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:09 smithi156 ceph-mon[36261]: pgmap v329: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-27T06:54:09.753 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2024-03-27T06:54:09.753 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:54:09.753 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2239, 2024-03-27T06:54:09.753 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2239, 2024-03-27T06:54:09.753 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 2996960, 2024-03-27T06:54:09.753 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 711185, 2024-03-27T06:54:09.753 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2024-03-27T06:54:09.754 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2024-03-27T06:54:09.754 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.130678416, 2024-03-27T06:54:09.754 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.039051673, 2024-03-27T06:54:09.754 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.034993886, 2024-03-27T06:54:09.754 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.047804408, 2024-03-27T06:54:09.754 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:09.754 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2238, 2024-03-27T06:54:09.754 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.040534804, 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000018112 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 4, 2024-03-27T06:54:09.755 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 165, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 876, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 446074, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.008273335, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.004809435, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.001994352, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000065536, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 162, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.012755201, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000078735 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.756 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 445376 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 276, 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 232, 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1701884, 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 134526, 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 3, 2024-03-27T06:54:09.757 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2024-03-27T06:54:09.758 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.037291775, 2024-03-27T06:54:09.758 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.008585715, 2024-03-27T06:54:09.758 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.036746804, 2024-03-27T06:54:09.758 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.003694654, 2024-03-27T06:54:09.758 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:09.758 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 231, 2024-03-27T06:54:09.758 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.006398826, 2024-03-27T06:54:09.758 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000027700 2024-03-27T06:54:09.758 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.758 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:09.758 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1701616, 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 134208 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-27T06:54:09.759 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3788, 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.074938259, 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000019783 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.760 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2024-03-27T06:54:09.761 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2024-03-27T06:54:09.761 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2024-03-27T06:54:09.761 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2024-03-27T06:54:09.761 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.761 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.761 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.761 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 307, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 0, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 307, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 0, 2024-03-27T06:54:09.762 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 307, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 1, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2024-03-27T06:54:09.763 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T06:54:09.764 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2024-03-27T06:54:09.765 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2024-03-27T06:54:09.766 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_quiesce_path": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_quiesce_inode": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2024-03-27T06:54:09.767 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 1336, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 3, 2024-03-27T06:54:09.768 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 2, 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 6557835, 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 6555747, 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 1336 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2024-03-27T06:54:09.769 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 13, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 490, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 477, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 4, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 100, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 96, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 307, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 656, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 349, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 52736, 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 215416 2024-03-27T06:54:09.770 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.771 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2024-03-27T06:54:09.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.773 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.774 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.775 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.775 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2024-03-27T06:54:09.775 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.775 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.775 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.775 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.775 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2024-03-27T06:54:09.776 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.776 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.776 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.776 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.776 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2024-03-27T06:54:09.776 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.776 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.776 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.776 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.776 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.777 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.778 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2024-03-27T06:54:09.779 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.780 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.781 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2024-03-27T06:54:09.782 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 4, 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 2, 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2024-03-27T06:54:09.783 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 3646, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:54:09.784 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:54:09.785 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:54:09.786 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 181663, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 57, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8032, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 167, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:54:09.787 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 256620, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 1608, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 2238, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 2238, 2024-03-27T06:54:09.788 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2238, 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.962986417, 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000430288 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2238, 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2239 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 2238, 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 2238, 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2024-03-27T06:54:09.789 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 1433, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 803, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2024-03-27T06:54:09.790 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2024-03-27T06:54:09.791 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2024-03-27T06:54:09.792 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 98, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 12, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 4, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 4, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2024-03-27T06:54:09.793 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed_ops": 0, 2024-03-27T06:54:09.794 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 2515, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 4456332, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 2515, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2515, 2024-03-27T06:54:09.795 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 4456332, 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-27T06:54:09.796 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 2238, 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 2363536, 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2238, 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2363536, 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:09.797 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 2238, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 2238, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 2238, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 2238, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.798 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.799 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x5651a05b86e0": { 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2024-03-27T06:54:09.800 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2024-03-27T06:54:09.801 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2024-03-27T06:54:09.801 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2024-03-27T06:54:09.801 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2024-03-27T06:54:09.801 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2024-03-27T06:54:09.801 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:09.801 INFO:teuthology.orchestra.run.smithi156.stdout: } 2024-03-27T06:54:09.801 INFO:teuthology.orchestra.run.smithi156.stdout:} 2024-03-27T06:54:10.547 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-27T06:54:10.547 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-27T06:54:10.547 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-03-27T06:54:10.547 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-03-27T06:54:10.547 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-03-27T06:54:10.547 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-27T06:54:10.547 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-27T06:54:10.548 DEBUG:tasks.check_counter:Getting stats from c 2024-03-27T06:54:10.548 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-27T06:54:10.803 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:54:11.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:11 smithi170 ceph-mon[36438]: pgmap v330: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:54:11.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:11 smithi170 ceph-mon[36438]: osdmap e99: 12 total, 12 up, 12 in 2024-03-27T06:54:11.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:11 smithi016 ceph-mon[31908]: pgmap v330: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:54:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:11 smithi016 ceph-mon[31908]: osdmap e99: 12 total, 12 up, 12 in 2024-03-27T06:54:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:11 smithi156 ceph-mon[36261]: pgmap v330: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-27T06:54:11.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:11 smithi156 ceph-mon[36261]: osdmap e99: 12 total, 12 up, 12 in 2024-03-27T06:54:12.716 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-03-27T06:54:12.716 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 14517, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 8025, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 6643143, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 3505482, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 14, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.603439433, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.250866613, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.459416771, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.011089651, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8027, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.139344242, 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000017359 2024-03-27T06:54:12.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 21667, 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.002804393, 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000129 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 397, 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 693, 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 186778, 2024-03-27T06:54:12.718 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 773795, 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 14, 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 8, 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.035159583, 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.015630211, 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.016524055, 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.002516704, 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 690, 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.032136054, 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000046573 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:12.719 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 708, 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000120115, 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000169 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 447952 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1204, 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1949, 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 2047058, 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 1085175, 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 15, 2024-03-27T06:54:12.720 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 9, 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.095607792, 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.035842083, 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.068808592, 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.005828548, 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1945, 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.045803619, 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000023549 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1661, 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000364940, 2024-03-27T06:54:12.721 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000219 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1722096, 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 151776 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-27T06:54:12.722 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 367, 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.270361061, 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000736678 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 419, 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.032821861, 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000078333 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-03-27T06:54:12.723 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 344, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 342, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 342, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.922995707, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002698817 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 2, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 11, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 24, 2024-03-27T06:54:12.724 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 7, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 7, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 10, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 13, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 1231, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 1, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 1, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 4, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 2110, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 1172, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 1, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 729, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-03-27T06:54:12.725 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 14, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 27, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 17, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 166, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 21, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 3713, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 28613248, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 3, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 21, 2024-03-27T06:54:12.726 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 1, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 1, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 20, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 2, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 51, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 260, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 103, 2024-03-27T06:54:12.727 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 49, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 15, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 337 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 120, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 2068, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 1389, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 1328, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 170, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 186, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 186, 2024-03-27T06:54:12.728 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 14, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-03-27T06:54:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 632, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 632, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 2, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 2, 2024-03-27T06:54:12.730 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 2, 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 5610189, 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 129, 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.317142659, 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002458470 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 31, 2024-03-27T06:54:12.731 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 836, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 805, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 28, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 777, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 749, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 24, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 1586, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 1562, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 1, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 203, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 202, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 58108, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 223608 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-03-27T06:54:12.732 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 645, 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 58, 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 344, 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 17, 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 303, 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 20, 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.062376095, 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003118804 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2, 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.003081867, 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001540933 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.733 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 98, 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.036073238, 2024-03-27T06:54:12.734 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000368094 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000137767, 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000137767 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.735 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.736 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.737 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.737 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-03-27T06:54:12.737 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.737 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-03-27T06:54:12.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 35, 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.030264032, 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000864686 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.739 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 15, 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.205911512, 2024-03-27T06:54:12.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.013727434 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.741 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.742 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.742 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-03-27T06:54:12.742 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.742 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.742 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.742 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-03-27T06:54:12.742 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.742 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-03-27T06:54:12.743 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 171, 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.585151196, 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003421936 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.744 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 3, 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 2, 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 268, 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 268, 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 627, 2024-03-27T06:54:12.745 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:54:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:54:12.747 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:54:12.748 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 326793, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 127, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 88, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 1, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 8032, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 167, 2024-03-27T06:54:12.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 140717, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 512, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-03-27T06:54:12.750 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 503, 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 1477675, 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 33, 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 503, 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 503, 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.292969228, 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002570515 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 503, 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 579 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.751 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 503, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 50, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 453, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 29, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 158, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 32, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-03-27T06:54:12.752 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 198, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 21, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T06:54:12.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-03-27T06:54:12.754 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 99, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 13, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-03-27T06:54:12.755 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 84, 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 0, 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 27 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-03-27T06:54:12.756 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 6, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 6, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 186, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 186, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 16114, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 7415451, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:12.757 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 16114, 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 16114, 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 7415451, 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-27T06:54:12.758 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 503, 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 1496509, 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 503, 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1496509, 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 503, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 503, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 503, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 503, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.760 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 186, 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 18834, 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 186, 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:12.761 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 29, 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 18834, 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x559a268dc500": { 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:12.762 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 632, 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1415885, 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 632, 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 129, 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1415885, 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:12.763 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-03-27T06:54:13.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:13 smithi170 ceph-mon[36438]: pgmap v332: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:54:13.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:13 smithi016 ceph-mon[31908]: pgmap v332: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:54:13.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:13 smithi156 ceph-mon[36261]: pgmap v332: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:54:13.554 INFO:tasks.check_counter:Daemon mds.c mds.exported=14 2024-03-27T06:54:13.554 INFO:tasks.check_counter:Daemon mds.c mds.imported=17 2024-03-27T06:54:13.554 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=120 2024-03-27T06:54:13.554 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=2068 2024-03-27T06:54:13.554 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=3 2024-03-27T06:54:13.554 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-27T06:54:13.554 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-27T06:54:13.554 DEBUG:tasks.check_counter:Getting stats from f 2024-03-27T06:54:13.554 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-27T06:54:13.816 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:54:15.280 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:15 smithi170 ceph-mon[36438]: pgmap v333: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:54:15.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:15 smithi016 ceph-mon[31908]: pgmap v333: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:54:15.382 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:15 smithi156 ceph-mon[36261]: pgmap v333: 97 pgs: 97 active+clean; 42 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-27T06:54:15.552 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-03-27T06:54:15.552 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:54:15.552 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 652, 2024-03-27T06:54:15.552 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 652, 2024-03-27T06:54:15.552 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 33665441, 2024-03-27T06:54:15.552 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 206522, 2024-03-27T06:54:15.552 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 7, 2024-03-27T06:54:15.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-03-27T06:54:15.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.062834833, 2024-03-27T06:54:15.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.013397215, 2024-03-27T06:54:15.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.028806921, 2024-03-27T06:54:15.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.015552373, 2024-03-27T06:54:15.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:15.553 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 651, 2024-03-27T06:54:15.553 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.019550351, 2024-03-27T06:54:15.553 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000030031 2024-03-27T06:54:15.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:15.554 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.554 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.554 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.554 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-27T06:54:15.554 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 448 2024-03-27T06:54:15.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.554 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:54:15.554 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1293, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1261, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 25319296, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 441936, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.083412841, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.022676757, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.034278481, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.024209667, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1260, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.032282184, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000025620 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 416112, 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 93024 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 76, 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 187, 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 25185487, 2024-03-27T06:54:15.556 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 306763, 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 6, 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.020184542, 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.004706380, 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.012926129, 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.001459401, 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 185, 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.007378276, 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000039882 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.557 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.558 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-27T06:54:15.558 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 282912 2024-03-27T06:54:15.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.558 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-03-27T06:54:15.558 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-03-27T06:54:15.558 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-03-27T06:54:15.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.558 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2990, 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.179304980, 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000059968 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.559 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2024-03-27T06:54:15.560 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 12479, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 8591, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 3888, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 205, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 2, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-03-27T06:54:15.561 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2024-03-27T06:54:15.562 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-03-27T06:54:15.562 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-03-27T06:54:15.562 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-03-27T06:54:15.562 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-03-27T06:54:15.562 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-03-27T06:54:15.562 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-03-27T06:54:15.562 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-03-27T06:54:15.562 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2024-03-27T06:54:15.562 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 4, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 234, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T06:54:15.563 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T06:54:15.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-03-27T06:54:15.564 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2024-03-27T06:54:15.564 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T06:54:15.564 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2024-03-27T06:54:15.564 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T06:54:15.564 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T06:54:15.564 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T06:54:15.564 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2024-03-27T06:54:15.564 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2024-03-27T06:54:15.565 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2024-03-27T06:54:15.565 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2024-03-27T06:54:15.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.565 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-03-27T06:54:15.565 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-03-27T06:54:15.565 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-03-27T06:54:15.565 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-03-27T06:54:15.565 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-03-27T06:54:15.566 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-03-27T06:54:15.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 38190, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 2, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 8, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 87381963, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 87531491, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 87531379, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.568 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 38190 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 2473, 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 11719, 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 9246, 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 231, 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 569, 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 338, 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 12479, 2024-03-27T06:54:15.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 15933, 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 3454, 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 71224, 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 215416 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2024-03-27T06:54:15.570 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.572 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-03-27T06:54:15.572 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.572 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.572 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.572 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-03-27T06:54:15.572 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.573 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.573 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-03-27T06:54:15.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.573 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.573 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-03-27T06:54:15.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.573 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.575 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-03-27T06:54:15.575 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.575 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.575 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.575 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-03-27T06:54:15.575 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.575 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.575 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.576 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-03-27T06:54:15.576 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.576 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.576 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.576 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-03-27T06:54:15.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.579 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-03-27T06:54:15.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.583 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-03-27T06:54:15.583 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.583 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.583 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.583 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-03-27T06:54:15.583 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-03-27T06:54:15.583 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 5, 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 3, 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 0, 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2024-03-27T06:54:15.584 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2024-03-27T06:54:15.585 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 4609, 2024-03-27T06:54:15.585 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-03-27T06:54:15.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.585 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-03-27T06:54:15.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:54:15.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-03-27T06:54:15.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:54:15.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:54:15.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:54:15.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:54:15.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:54:15.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:54:15.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:54:15.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:54:15.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:54:15.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:54:15.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:54:15.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-03-27T06:54:15.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:54:15.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:54:15.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:54:15.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-03-27T06:54:15.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-03-27T06:54:15.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-03-27T06:54:15.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:54:15.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:54:15.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:54:15.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:54:15.589 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 319309, 2024-03-27T06:54:15.589 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 2704, 2024-03-27T06:54:15.589 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-03-27T06:54:15.589 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-03-27T06:54:15.589 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-03-27T06:54:15.589 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-03-27T06:54:15.589 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 8152, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 170, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 14981077, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 163346, 2024-03-27T06:54:15.590 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 1820, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 26, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 4, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 1820, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1820, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.528963672, 2024-03-27T06:54:15.591 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000840089 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1820, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1821 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 1820, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 1820, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 1152, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 665, 2024-03-27T06:54:15.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2024-03-27T06:54:15.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2024-03-27T06:54:15.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-03-27T06:54:15.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-03-27T06:54:15.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-03-27T06:54:15.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-03-27T06:54:15.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2024-03-27T06:54:15.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-03-27T06:54:15.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-03-27T06:54:15.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-03-27T06:54:15.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 2, 2024-03-27T06:54:15.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2024-03-27T06:54:15.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-03-27T06:54:15.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-03-27T06:54:15.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-03-27T06:54:15.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-03-27T06:54:15.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-03-27T06:54:15.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-03-27T06:54:15.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T06:54:15.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-03-27T06:54:15.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-03-27T06:54:15.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-03-27T06:54:15.594 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-03-27T06:54:15.595 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-03-27T06:54:15.595 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-03-27T06:54:15.595 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-03-27T06:54:15.595 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-03-27T06:54:15.595 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-03-27T06:54:15.595 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-03-27T06:54:15.595 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-03-27T06:54:15.595 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-03-27T06:54:15.595 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-03-27T06:54:15.595 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-03-27T06:54:15.595 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-03-27T06:54:15.596 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-03-27T06:54:15.596 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-03-27T06:54:15.596 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-03-27T06:54:15.596 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-03-27T06:54:15.596 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 99, 2024-03-27T06:54:15.597 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-03-27T06:54:15.597 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 9, 2024-03-27T06:54:15.597 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 9, 2024-03-27T06:54:15.597 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 9, 2024-03-27T06:54:15.597 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-03-27T06:54:15.597 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-03-27T06:54:15.597 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2024-03-27T06:54:15.597 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2024-03-27T06:54:15.597 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2024-03-27T06:54:15.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.598 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-03-27T06:54:15.598 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2024-03-27T06:54:15.598 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-03-27T06:54:15.598 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-03-27T06:54:15.598 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-03-27T06:54:15.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-03-27T06:54:15.598 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-03-27T06:54:15.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.598 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-03-27T06:54:15.599 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-03-27T06:54:15.599 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T06:54:15.599 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-03-27T06:54:15.599 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2024-03-27T06:54:15.599 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 0, 2024-03-27T06:54:15.599 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2024-03-27T06:54:15.599 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-03-27T06:54:15.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.599 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:54:15.599 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:15.601 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-27T06:54:15.601 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:15.601 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 2017, 2024-03-27T06:54:15.601 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 83975702, 2024-03-27T06:54:15.601 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:15.601 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 2017, 2024-03-27T06:54:15.601 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:15.602 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:15.602 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2017, 2024-03-27T06:54:15.602 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 83975702, 2024-03-27T06:54:15.602 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:15.602 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.602 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.602 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.602 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:15.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.602 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 1820, 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 83240335, 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1820, 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 83240335, 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:15.603 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.604 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.604 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.604 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:15.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.604 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-03-27T06:54:15.604 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:15.604 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-03-27T06:54:15.604 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:15.605 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:15.605 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:15.605 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:15.605 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:15.605 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 1820, 2024-03-27T06:54:15.605 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 1820, 2024-03-27T06:54:15.605 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1820, 2024-03-27T06:54:15.605 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1820, 2024-03-27T06:54:15.605 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:15.605 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.605 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:15.606 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:15.607 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-03-27T06:54:15.607 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-03-27T06:54:15.607 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:15.607 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.607 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.607 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.607 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:15.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:15.607 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55b56485a6e0": { 2024-03-27T06:54:15.607 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:15.608 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-27T06:54:15.608 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:15.608 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:15.608 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:15.608 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:15.608 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:15.608 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:15.608 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:15.609 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-03-27T06:54:15.609 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-03-27T06:54:15.609 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:15.609 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:15.609 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:15.609 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:15.609 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:15.609 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:15.609 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-03-27T06:54:16.381 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-27T06:54:16.381 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-27T06:54:16.381 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-27T06:54:16.382 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-03-27T06:54:16.382 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-03-27T06:54:16.382 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-27T06:54:16.382 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-27T06:54:16.382 DEBUG:tasks.check_counter:Getting stats from i 2024-03-27T06:54:16.383 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-27T06:54:16.659 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:54:17.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:17 smithi170 ceph-mon[36438]: pgmap v334: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:54:17.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:17 smithi016 ceph-mon[31908]: pgmap v334: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:54:17.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:17 smithi156 ceph-mon[36261]: pgmap v334: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:54:18.553 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-03-27T06:54:18.553 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:54:18.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 4581, 2024-03-27T06:54:18.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 7502, 2024-03-27T06:54:18.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1095683, 2024-03-27T06:54:18.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 18457286, 2024-03-27T06:54:18.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 22, 2024-03-27T06:54:18.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-03-27T06:54:18.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.530498283, 2024-03-27T06:54:18.553 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.219834392, 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.197376550, 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.120468763, 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 7501, 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.216427849, 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000028853 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 9186, 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.001723758, 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000187 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 4160, 2024-03-27T06:54:18.554 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 320000 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 16141, 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 33639, 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 6207695, 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 23214195, 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 9, 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.035458714, 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.508676219, 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.651719317, 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.055661008, 2024-03-27T06:54:18.555 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 33638, 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.726696239, 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000021603 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 30436, 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.006245269, 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000205 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1724192, 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 153360 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 16239, 2024-03-27T06:54:18.556 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 18668, 2024-03-27T06:54:18.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 7531437, 2024-03-27T06:54:18.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 16733384, 2024-03-27T06:54:18.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 14, 2024-03-27T06:54:18.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 9, 2024-03-27T06:54:18.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.007779094, 2024-03-27T06:54:18.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.403911896, 2024-03-27T06:54:18.557 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.534660540, 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.139799018, 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 18665, 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.504024731, 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000027003 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 30425, 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.004533090, 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000148 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 5664, 2024-03-27T06:54:18.558 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 145184 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1077, 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 8.764452464, 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.008137838 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-03-27T06:54:18.559 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-27T06:54:18.560 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-27T06:54:18.560 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 6403, 2024-03-27T06:54:18.560 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.521538704, 2024-03-27T06:54:18.560 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000237629 2024-03-27T06:54:18.560 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:18.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.560 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-03-27T06:54:18.560 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 6441, 2024-03-27T06:54:18.560 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 6413, 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 6413, 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 18.208396254, 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002839294 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 28, 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 56, 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 5, 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 1, 2024-03-27T06:54:18.561 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 146, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 63, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 54, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 29, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 55, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 928, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 1, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 1, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 4, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 7653, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 6896, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 1, 2024-03-27T06:54:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 361, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 27, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 272, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 90, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 1809, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 12, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 3713, 2024-03-27T06:54:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 28613248, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 3, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 12, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 12, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 48, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 1033, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 6517, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T06:54:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 5045, 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 4024, 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 38, 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 4331 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 1165, 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 1016, 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 657, 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 775, 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 2342, 2024-03-27T06:54:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 42, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 3772, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 3730, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-03-27T06:54:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 27, 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 6, 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 15814, 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-03-27T06:54:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 15814, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 21, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 21, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 3, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 38296090, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-03-27T06:54:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 625, 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 22.775419446, 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.036440671 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 66, 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 4381, 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 4315, 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 75, 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 772, 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 697, 2024-03-27T06:54:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 146, 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 8500, 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 8354, 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 1, 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 4059, 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 4058, 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 82424, 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 223608 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 8482, 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 271, 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 6441, 2024-03-27T06:54:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 52, 2024-03-27T06:54:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 822, 2024-03-27T06:54:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-03-27T06:54:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2093, 2024-03-27T06:54:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.524083662, 2024-03-27T06:54:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002161530 2024-03-27T06:54:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-03-27T06:54:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 28, 2024-03-27T06:54:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.056559879, 2024-03-27T06:54:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002019995 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-03-27T06:54:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 44, 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.015582166, 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000354140 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-03-27T06:54:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.575 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-03-27T06:54:18.575 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.575 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.575 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 156, 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.157336388, 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001008566 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-03-27T06:54:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 320, 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.416188461, 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001300588 2024-03-27T06:54:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 291, 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 6.680968080, 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.022958653 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3481, 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 6.357677618, 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001826394 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-03-27T06:54:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 5, 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 3, 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-03-27T06:54:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 5545, 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 2772, 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 614, 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:54:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:54:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:54:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:54:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:54:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:54:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:54:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:54:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:54:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:54:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:54:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 310180, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 194, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 1496, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 17, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-03-27T06:54:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 8152, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 170, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 399581, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 9024, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-03-27T06:54:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 4879, 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 35717142, 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 29, 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 4879, 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4879, 2024-03-27T06:54:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 12.716697529, 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002606414 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4879, 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 5025 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 4879, 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 139, 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 4740, 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-03-27T06:54:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2024-03-27T06:54:18.591 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 127, 2024-03-27T06:54:18.591 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 752, 2024-03-27T06:54:18.591 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 76, 2024-03-27T06:54:18.591 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-03-27T06:54:18.591 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-03-27T06:54:18.591 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-03-27T06:54:18.591 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 5, 2024-03-27T06:54:18.591 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 3786, 2024-03-27T06:54:18.591 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-03-27T06:54:18.591 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 12, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-03-27T06:54:18.592 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-03-27T06:54:18.593 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 99, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 13, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 192, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 0, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 45 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-03-27T06:54:18.594 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 10, 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 6, 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 3762, 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 3730, 2024-03-27T06:54:18.595 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 36957, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 11671441, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 36957, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 36957, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 11671441, 2024-03-27T06:54:18.596 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:18.597 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 4879, 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 36093976, 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 4879, 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 36093976, 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 4879, 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 4879, 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 4879, 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 4879, 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.599 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 3730, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 376834, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 3730, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 127, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 376834, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.600 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55f984e8c500": { 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 15814, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 34101786, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 15814, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 625, 2024-03-27T06:54:18.601 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 34101786, 2024-03-27T06:54:18.602 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:18.602 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:18.602 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:18.602 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:18.602 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:18.602 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:18.602 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-03-27T06:54:19.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:19 smithi170 ceph-mon[36438]: pgmap v335: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:54:19.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:19 smithi016 ceph-mon[31908]: pgmap v335: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:54:19.392 INFO:tasks.check_counter:Daemon mds.i mds.exported=27 2024-03-27T06:54:19.392 INFO:tasks.check_counter:Daemon mds.i mds.imported=90 2024-03-27T06:54:19.392 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=1165 2024-03-27T06:54:19.392 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1016 2024-03-27T06:54:19.392 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=3 2024-03-27T06:54:19.392 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-27T06:54:19.393 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-27T06:54:19.393 DEBUG:tasks.check_counter:Getting stats from l 2024-03-27T06:54:19.393 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0 shell --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-27T06:54:19.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:19 smithi156 ceph-mon[36261]: pgmap v335: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-27T06:54:19.642 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/mon.c/config 2024-03-27T06:54:21.478 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 354, 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 356, 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 12677275, 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 112636, 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 7, 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.029841460, 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.006357903, 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.014146869, 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.006571234, 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:21.479 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 354, 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.010025462, 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000028320 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-03-27T06:54:21.480 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1002, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 958, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 15019526, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 366403, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.086418265, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.024519771, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.051175044, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.018338240, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 957, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.023535223, 2024-03-27T06:54:21.481 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000024592 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1704032, 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 137120 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1239, 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1401, 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 8764106, 2024-03-27T06:54:21.482 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 841678, 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 8, 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.098626758, 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.030460221, 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.028377014, 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.031350786, 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1399, 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.043609817, 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000031172 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.483 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 449584 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-27T06:54:21.484 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3847, 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.077188092, 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000020064 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-03-27T06:54:21.485 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2024-03-27T06:54:21.486 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2024-03-27T06:54:21.486 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-03-27T06:54:21.486 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.486 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.486 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.486 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-03-27T06:54:21.486 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2024-03-27T06:54:21.486 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-03-27T06:54:21.486 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-03-27T06:54:21.486 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 5001, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 0, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 5001, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 5001, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 1, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 1, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2024-03-27T06:54:21.487 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-03-27T06:54:21.488 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2024-03-27T06:54:21.489 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-03-27T06:54:21.490 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-03-27T06:54:21.491 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2024-03-27T06:54:21.492 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-03-27T06:54:21.493 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-03-27T06:54:21.493 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 15814, 2024-03-27T06:54:21.493 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-03-27T06:54:21.493 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-03-27T06:54:21.493 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2024-03-27T06:54:21.493 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-03-27T06:54:21.493 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-03-27T06:54:21.493 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 21, 2024-03-27T06:54:21.493 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 3, 2024-03-27T06:54:21.493 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 38296090, 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 38294002, 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 15814 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-03-27T06:54:21.494 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 54, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 4081, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 4027, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 63, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 410, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 347, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 5001, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 7871, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 2870, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 57420, 2024-03-27T06:54:21.495 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 215416 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.496 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-03-27T06:54:21.497 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.498 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-03-27T06:54:21.499 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.499 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.499 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.499 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-03-27T06:54:21.499 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.499 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.499 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.500 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-03-27T06:54:21.500 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.500 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.500 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.500 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-03-27T06:54:21.500 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.500 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.500 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.500 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-03-27T06:54:21.501 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.501 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.501 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.501 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-03-27T06:54:21.501 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.503 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-03-27T06:54:21.504 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.506 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-03-27T06:54:21.507 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-03-27T06:54:21.508 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 5, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 3, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 5227, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-03-27T06:54:21.509 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-03-27T06:54:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-03-27T06:54:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 224051, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 111, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-03-27T06:54:21.512 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 8152, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 170, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 3753642, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 33692, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-03-27T06:54:21.513 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 2308, 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 21, 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 3, 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 2308, 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2308, 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.269945991, 2024-03-27T06:54:21.514 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000983512 2024-03-27T06:54:21.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.515 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-03-27T06:54:21.515 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-03-27T06:54:21.515 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2308, 2024-03-27T06:54:21.515 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2309 2024-03-27T06:54:21.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.515 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 2308, 2024-03-27T06:54:21.515 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 2308, 2024-03-27T06:54:21.515 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 1479, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 827, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2024-03-27T06:54:21.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-03-27T06:54:21.517 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-03-27T06:54:21.517 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-03-27T06:54:21.517 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 1, 2024-03-27T06:54:21.517 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2024-03-27T06:54:21.517 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-03-27T06:54:21.517 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-03-27T06:54:21.517 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-03-27T06:54:21.517 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-03-27T06:54:21.517 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-03-27T06:54:21.517 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-03-27T06:54:21.517 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-03-27T06:54:21.518 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-03-27T06:54:21.518 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-03-27T06:54:21.518 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-03-27T06:54:21.518 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-03-27T06:54:21.518 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-03-27T06:54:21.518 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-03-27T06:54:21.518 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-03-27T06:54:21.518 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-03-27T06:54:21.518 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-03-27T06:54:21.518 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-03-27T06:54:21.519 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-03-27T06:54:21.519 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-03-27T06:54:21.519 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-03-27T06:54:21.519 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-03-27T06:54:21.519 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-03-27T06:54:21.519 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-03-27T06:54:21.519 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-03-27T06:54:21.519 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-03-27T06:54:21.519 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-03-27T06:54:21.519 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 100, 2024-03-27T06:54:21.519 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-03-27T06:54:21.520 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 14, 2024-03-27T06:54:21.520 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 9, 2024-03-27T06:54:21.520 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 9, 2024-03-27T06:54:21.520 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-03-27T06:54:21.520 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-03-27T06:54:21.520 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2024-03-27T06:54:21.520 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2024-03-27T06:54:21.520 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2024-03-27T06:54:21.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.520 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-03-27T06:54:21.520 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 0, 2024-03-27T06:54:21.521 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 2591, 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 36210378, 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 2591, 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:21.522 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2591, 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 36210378, 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:21.524 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:21.524 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 2308, 2024-03-27T06:54:21.524 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 34103407, 2024-03-27T06:54:21.524 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2308, 2024-03-27T06:54:21.525 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 34103407, 2024-03-27T06:54:21.525 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:21.525 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.525 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.525 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.525 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 2308, 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 2308, 2024-03-27T06:54:21.526 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2308, 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2308, 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:21.527 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x5629cc9e46e0": { 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-27T06:54:21.528 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-27T06:54:21.529 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:21.530 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-27T06:54:21.530 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-03-27T06:54:21.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:21 smithi170 ceph-mon[36438]: pgmap v336: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:54:21.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:21 smithi170 ceph-mon[36438]: osdmap e100: 12 total, 12 up, 12 in 2024-03-27T06:54:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:21 smithi016 ceph-mon[31908]: pgmap v336: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:54:21.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:21 smithi016 ceph-mon[31908]: osdmap e100: 12 total, 12 up, 12 in 2024-03-27T06:54:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:21 smithi156 ceph-mon[36261]: pgmap v336: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:54:21.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:21 smithi156 ceph-mon[36261]: osdmap e100: 12 total, 12 up, 12 in 2024-03-27T06:54:22.297 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-27T06:54:22.298 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-27T06:54:22.298 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-03-27T06:54:22.298 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-03-27T06:54:22.298 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-27T06:54:22.298 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-27T06:54:22.298 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-27T06:54:22.298 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-27T06:54:22.309 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-27T06:54:22.309 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:54:22.309 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-27T06:54:22.330 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:54:22.330 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-27T06:54:22.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:54:22.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.552+0000 7fd94793a640 1 -- 172.21.15.16:0/2228465606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd940107b80 msgr2=0x7fd940107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.552+0000 7fd94793a640 1 --2- 172.21.15.16:0/2228465606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd940107b80 0x7fd940107f60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd934009930 tx=0x7fd93402bc30 comp rx=0 tx=0).stop 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.552+0000 7fd94793a640 1 -- 172.21.15.16:0/2228465606 shutdown_connections 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.552+0000 7fd94793a640 1 --2- 172.21.15.16:0/2228465606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9401024a0 0x7fd94010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.552+0000 7fd94793a640 1 --2- 172.21.15.16:0/2228465606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd940101ae0 0x7fd940101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.552+0000 7fd94793a640 1 --2- 172.21.15.16:0/2228465606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd940107b80 0x7fd940107f60 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.552+0000 7fd94793a640 1 -- 172.21.15.16:0/2228465606 >> 172.21.15.16:0/2228465606 conn(0x7fd9400fd5f0 msgr2=0x7fd9400ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.552+0000 7fd94793a640 1 -- 172.21.15.16:0/2228465606 shutdown_connections 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.552+0000 7fd94793a640 1 -- 172.21.15.16:0/2228465606 wait complete. 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.553+0000 7fd94793a640 1 Processor -- start 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.553+0000 7fd94793a640 1 -- start start 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.553+0000 7fd94793a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101ae0 0x7fd940198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.553+0000 7fd94793a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9401024a0 0x7fd940198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:22.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.553+0000 7fd94793a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd940107b80 0x7fd94019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:22.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.553+0000 7fd94793a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd940111fd0 con 0x7fd9401024a0 2024-03-27T06:54:22.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.553+0000 7fd94793a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd940111e50 con 0x7fd940101ae0 2024-03-27T06:54:22.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.553+0000 7fd94793a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd940112150 con 0x7fd940107b80 2024-03-27T06:54:22.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd944ed5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9401024a0 0x7fd940198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:22.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9456d6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101ae0 0x7fd940198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:22.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9456d6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101ae0 0x7fd940198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.16:38306/0 (socket says 172.21.15.16:38306) 2024-03-27T06:54:22.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd944ed5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9401024a0 0x7fd940198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44186/0 (socket says 172.21.15.16:44186) 2024-03-27T06:54:22.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9456d6640 1 -- 172.21.15.16:0/2051628582 learned_addr learned my addr 172.21.15.16:0/2051628582 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:22.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd945ed7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd940107b80 0x7fd94019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:22.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9456d6640 1 -- 172.21.15.16:0/2051628582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd940107b80 msgr2=0x7fd94019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9456d6640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd940107b80 0x7fd94019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9456d6640 1 -- 172.21.15.16:0/2051628582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9401024a0 msgr2=0x7fd940198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9456d6640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9401024a0 0x7fd940198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9456d6640 1 -- 172.21.15.16:0/2051628582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94019d500 con 0x7fd940101ae0 2024-03-27T06:54:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd944ed5640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9401024a0 0x7fd940198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:54:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9456d6640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101ae0 0x7fd940198510 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd934009230 tx=0x7fd93402dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9327fc640 1 -- 172.21.15.16:0/2051628582 <== mon.2 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934043020 con 0x7fd940101ae0 2024-03-27T06:54:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9327fc640 1 -- 172.21.15.16:0/2051628582 <== mon.2 v2:172.21.15.156:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd934004250 con 0x7fd940101ae0 2024-03-27T06:54:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd94793a640 1 -- 172.21.15.16:0/2051628582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94019d790 con 0x7fd940101ae0 2024-03-27T06:54:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd9327fc640 1 -- 172.21.15.16:0/2051628582 <== mon.2 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934003970 con 0x7fd940101ae0 2024-03-27T06:54:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.554+0000 7fd94793a640 1 -- 172.21.15.16:0/2051628582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9401a5080 con 0x7fd940101ae0 2024-03-27T06:54:22.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.555+0000 7fd94793a640 1 -- 172.21.15.16:0/2051628582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd940068d90 con 0x7fd940101ae0 2024-03-27T06:54:22.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.556+0000 7fd9327fc640 1 -- 172.21.15.16:0/2051628582 <== mon.2 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fd934046020 con 0x7fd940101ae0 2024-03-27T06:54:22.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.557+0000 7fd9327fc640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd92c0a3800 0x7fd92c0a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:22.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.557+0000 7fd944ed5640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd92c0a3800 0x7fd92c0a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:22.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.557+0000 7fd9327fc640 1 -- 172.21.15.16:0/2051628582 <== mon.2 v2:172.21.15.156:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fd9340f2460 con 0x7fd940101ae0 2024-03-27T06:54:22.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.557+0000 7fd944ed5640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd92c0a3800 0x7fd92c0a5cc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd9400ffc70 tx=0x7fd93c005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:22.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.561+0000 7fd9327fc640 1 -- 172.21.15.16:0/2051628582 <== mon.2 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd93404d020 con 0x7fd940101ae0 2024-03-27T06:54:22.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.748+0000 7fd94793a640 1 -- 172.21.15.16:0/2051628582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd94010a050 con 0x7fd940101ae0 2024-03-27T06:54:22.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.750+0000 7fd9327fc640 1 -- 172.21.15.16:0/2051628582 <== mon.2 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23073 (secure 0 0 0) 0x7fd9340beda0 con 0x7fd940101ae0 2024-03-27T06:54:22.746 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:54:22.746 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:54:20.631213+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3655179049":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3519122217":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6800/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/212881673":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/361067788":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:54:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.752+0000 7fd94793a640 1 -- 172.21.15.16:0/2051628582 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd92c0a3800 msgr2=0x7fd92c0a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.752+0000 7fd94793a640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd92c0a3800 0x7fd92c0a5cc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd9400ffc70 tx=0x7fd93c005f90 comp rx=0 tx=0).stop 2024-03-27T06:54:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.752+0000 7fd94793a640 1 -- 172.21.15.16:0/2051628582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101ae0 msgr2=0x7fd940198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.752+0000 7fd94793a640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101ae0 0x7fd940198510 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd934009230 tx=0x7fd93402dc90 comp rx=0 tx=0).stop 2024-03-27T06:54:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.752+0000 7fd94793a640 1 -- 172.21.15.16:0/2051628582 shutdown_connections 2024-03-27T06:54:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.752+0000 7fd94793a640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd92c0a3800 0x7fd92c0a5cc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.752+0000 7fd94793a640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd940107b80 0x7fd94019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.752+0000 7fd94793a640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9401024a0 0x7fd940198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.752+0000 7fd94793a640 1 --2- 172.21.15.16:0/2051628582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd940101ae0 0x7fd940198510 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.752+0000 7fd94793a640 1 -- 172.21.15.16:0/2051628582 >> 172.21.15.16:0/2051628582 conn(0x7fd9400fd5f0 msgr2=0x7fd9400fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.753+0000 7fd94793a640 1 -- 172.21.15.16:0/2051628582 shutdown_connections 2024-03-27T06:54:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.753+0000 7fd94793a640 1 -- 172.21.15.16:0/2051628582 wait complete. 2024-03-27T06:54:22.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-27T06:36:47.581613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-27T06:41:26.393978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-27T06:41:27.707469+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-27T06:54:22.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-27T06:54:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.870+0000 7f9d011b6640 1 -- 172.21.15.16:0/1066698008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cfc100d30 msgr2=0x7f9cfc1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.870+0000 7f9d011b6640 1 --2- 172.21.15.16:0/1066698008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cfc100d30 0x7f9cfc1011b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009900 tx=0x7f9cf002bc30 comp rx=0 tx=0).stop 2024-03-27T06:54:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.870+0000 7f9d011b6640 1 -- 172.21.15.16:0/1066698008 shutdown_connections 2024-03-27T06:54:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.870+0000 7f9d011b6640 1 --2- 172.21.15.16:0/1066698008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cfc1016f0 0x7f9cfc10d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.870+0000 7f9d011b6640 1 --2- 172.21.15.16:0/1066698008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cfc100d30 0x7f9cfc1011b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.870+0000 7f9d011b6640 1 --2- 172.21.15.16:0/1066698008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cfc106a80 0x7f9cfc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.870+0000 7f9d011b6640 1 -- 172.21.15.16:0/1066698008 >> 172.21.15.16:0/1066698008 conn(0x7f9cfc077a30 msgr2=0x7f9cfc077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.870+0000 7f9d011b6640 1 -- 172.21.15.16:0/1066698008 shutdown_connections 2024-03-27T06:54:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.870+0000 7f9d011b6640 1 -- 172.21.15.16:0/1066698008 wait complete. 2024-03-27T06:54:22.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.871+0000 7f9d011b6640 1 Processor -- start 2024-03-27T06:54:22.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.871+0000 7f9d011b6640 1 -- start start 2024-03-27T06:54:22.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.871+0000 7f9d011b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cfc100d30 0x7f9cfc199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.871+0000 7f9d011b6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cfc1016f0 0x7f9cfc199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.871+0000 7f9d011b6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cfc106a80 0x7f9cfc1a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.871+0000 7f9d011b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc110ea0 con 0x7f9cfc100d30 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.871+0000 7f9d011b6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc110d20 con 0x7f9cfc1016f0 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.871+0000 7f9d011b6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc111020 con 0x7f9cfc106a80 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9cfad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cfc100d30 0x7f9cfc199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9cfad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cfc100d30 0x7f9cfc199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44196/0 (socket says 172.21.15.16:44196) 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9cfad9d640 1 -- 172.21.15.16:0/1043694663 learned_addr learned my addr 172.21.15.16:0/1043694663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9cfb59e640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cfc106a80 0x7f9cfc1a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9cfa59c640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cfc1016f0 0x7f9cfc199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9cfad9d640 1 -- 172.21.15.16:0/1043694663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cfc106a80 msgr2=0x7f9cfc1a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9cfad9d640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cfc106a80 0x7f9cfc1a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9cfad9d640 1 -- 172.21.15.16:0/1043694663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cfc1016f0 msgr2=0x7f9cfc199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:22.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9cfad9d640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cfc1016f0 0x7f9cfc199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:22.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9cfad9d640 1 -- 172.21.15.16:0/1043694663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc1a47c0 con 0x7f9cfc100d30 2024-03-27T06:54:22.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9cfad9d640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cfc100d30 0x7f9cfc199200 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e9f0 tx=0x7f9cf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:22.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9ce3fff640 1 -- 172.21.15.16:0/1043694663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400cd80 con 0x7f9cfc100d30 2024-03-27T06:54:22.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9d011b6640 1 -- 172.21.15.16:0/1043694663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cfc1a49f0 con 0x7f9cfc100d30 2024-03-27T06:54:22.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9ce3fff640 1 -- 172.21.15.16:0/1043694663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9cf4010930 con 0x7f9cfc100d30 2024-03-27T06:54:22.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9ce3fff640 1 -- 172.21.15.16:0/1043694663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4018b00 con 0x7f9cfc100d30 2024-03-27T06:54:22.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.872+0000 7f9d011b6640 1 -- 172.21.15.16:0/1043694663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cfc1a4f60 con 0x7f9cfc100d30 2024-03-27T06:54:22.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.873+0000 7f9d011b6640 1 -- 172.21.15.16:0/1043694663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cfc102170 con 0x7f9cfc100d30 2024-03-27T06:54:22.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.874+0000 7f9ce3fff640 1 -- 172.21.15.16:0/1043694663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f9cf4018ca0 con 0x7f9cfc100d30 2024-03-27T06:54:22.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.875+0000 7f9ce3fff640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f9ce40a3780 0x7f9ce40a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:22.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.875+0000 7f9cfa59c640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f9ce40a3780 0x7f9ce40a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:22.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.875+0000 7f9ce3fff640 1 -- 172.21.15.16:0/1043694663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f9cf40d0430 con 0x7f9cfc100d30 2024-03-27T06:54:22.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.876+0000 7f9cfa59c640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f9ce40a3780 0x7f9ce40a5c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0006010 tx=0x7f9cf0005e40 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:22.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:22.880+0000 7f9ce3fff640 1 -- 172.21.15.16:0/1043694663 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f9cf409c670 con 0x7f9cfc100d30 2024-03-27T06:54:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.063+0000 7f9d011b6640 1 -- 172.21.15.16:0/1043694663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9cfc108940 con 0x7f9cfc100d30 2024-03-27T06:54:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.064+0000 7f9ce3fff640 1 -- 172.21.15.16:0/1043694663 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f9cf40a0970 con 0x7f9cfc100d30 2024-03-27T06:54:23.059 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-03-27T06:54:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 -- 172.21.15.16:0/1043694663 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f9ce40a3780 msgr2=0x7f9ce40a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f9ce40a3780 0x7f9ce40a5c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0006010 tx=0x7f9cf0005e40 comp rx=0 tx=0).stop 2024-03-27T06:54:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 -- 172.21.15.16:0/1043694663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cfc100d30 msgr2=0x7f9cfc199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cfc100d30 0x7f9cfc199200 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e9f0 tx=0x7f9cf400eeb0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 -- 172.21.15.16:0/1043694663 shutdown_connections 2024-03-27T06:54:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f9ce40a3780 0x7f9ce40a5c40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cfc106a80 0x7f9cfc1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cfc1016f0 0x7f9cfc199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 --2- 172.21.15.16:0/1043694663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cfc100d30 0x7f9cfc199200 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 -- 172.21.15.16:0/1043694663 >> 172.21.15.16:0/1043694663 conn(0x7f9cfc077a30 msgr2=0x7f9cfc0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 -- 172.21.15.16:0/1043694663 shutdown_connections 2024-03-27T06:54:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.066+0000 7f9d011b6640 1 -- 172.21.15.16:0/1043694663 wait complete. 2024-03-27T06:54:23.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-27T06:54:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.182+0000 7f6781b5c640 1 -- 172.21.15.16:0/2641284206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c101430 msgr2=0x7f677c10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.182+0000 7f6781b5c640 1 --2- 172.21.15.16:0/2641284206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c101430 0x7f677c10d940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6774009ca0 tx=0x7f677402c0f0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.182+0000 7f6781b5c640 1 -- 172.21.15.16:0/2641284206 shutdown_connections 2024-03-27T06:54:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.182+0000 7f6781b5c640 1 --2- 172.21.15.16:0/2641284206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c101430 0x7f677c10d940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.182+0000 7f6781b5c640 1 --2- 172.21.15.16:0/2641284206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c100a70 0x7f677c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.182+0000 7f6781b5c640 1 --2- 172.21.15.16:0/2641284206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c106a80 0x7f677c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.182+0000 7f6781b5c640 1 -- 172.21.15.16:0/2641284206 >> 172.21.15.16:0/2641284206 conn(0x7f677c077bb0 msgr2=0x7f677c077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.182+0000 7f6781b5c640 1 -- 172.21.15.16:0/2641284206 shutdown_connections 2024-03-27T06:54:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.182+0000 7f6781b5c640 1 -- 172.21.15.16:0/2641284206 wait complete. 2024-03-27T06:54:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.182+0000 7f6781b5c640 1 Processor -- start 2024-03-27T06:54:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f6781b5c640 1 -- start start 2024-03-27T06:54:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f6781b5c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c100a70 0x7f677c10e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f6781b5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c101430 0x7f677c10e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f6781b5c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c106a80 0x7f677c10ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f6781b5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f677c112070 con 0x7f677c101430 2024-03-27T06:54:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f6781b5c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f677c111ef0 con 0x7f677c106a80 2024-03-27T06:54:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f6781b5c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1121f0 con 0x7f677c100a70 2024-03-27T06:54:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f677affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c101430 0x7f677c10e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f677affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c101430 0x7f677c10e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44208/0 (socket says 172.21.15.16:44208) 2024-03-27T06:54:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f677affd640 1 -- 172.21.15.16:0/4090544050 learned_addr learned my addr 172.21.15.16:0/4090544050 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f677b7fe640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c100a70 0x7f677c10e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f677bfff640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c106a80 0x7f677c10ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f677b7fe640 1 -- 172.21.15.16:0/4090544050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c106a80 msgr2=0x7f677c10ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f677b7fe640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c106a80 0x7f677c10ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f677b7fe640 1 -- 172.21.15.16:0/4090544050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c101430 msgr2=0x7f677c10e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f677b7fe640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c101430 0x7f677c10e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f677b7fe640 1 -- 172.21.15.16:0/4090544050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c1a6950 con 0x7f677c100a70 2024-03-27T06:54:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.183+0000 7f677affd640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c101430 0x7f677c10e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:54:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.184+0000 7f677b7fe640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c100a70 0x7f677c10e3d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f676c00e9f0 tx=0x7f676c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.184+0000 7f6778ff9640 1 -- 172.21.15.16:0/4090544050 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c00ccc0 con 0x7f677c100a70 2024-03-27T06:54:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.184+0000 7f6778ff9640 1 -- 172.21.15.16:0/4090544050 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f676c010930 con 0x7f677c100a70 2024-03-27T06:54:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.184+0000 7f6781b5c640 1 -- 172.21.15.16:0/4090544050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c1a6c40 con 0x7f677c100a70 2024-03-27T06:54:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.184+0000 7f6778ff9640 1 -- 172.21.15.16:0/4090544050 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0189f0 con 0x7f677c100a70 2024-03-27T06:54:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.184+0000 7f6781b5c640 1 -- 172.21.15.16:0/4090544050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c1a7190 con 0x7f677c100a70 2024-03-27T06:54:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.185+0000 7f6781b5c640 1 -- 172.21.15.16:0/4090544050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c068d90 con 0x7f677c100a70 2024-03-27T06:54:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.186+0000 7f6778ff9640 1 -- 172.21.15.16:0/4090544050 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f676c010ae0 con 0x7f677c100a70 2024-03-27T06:54:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.186+0000 7f6778ff9640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f67640a3780 0x7f67640a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.186+0000 7f677affd640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f67640a3780 0x7f67640a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.186+0000 7f6778ff9640 1 -- 172.21.15.16:0/4090544050 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f676c0d0430 con 0x7f677c100a70 2024-03-27T06:54:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.187+0000 7f677affd640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f67640a3780 0x7f67640a5c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f67700095f0 tx=0x7f6770005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.190+0000 7f6778ff9640 1 -- 172.21.15.16:0/4090544050 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f676c09c670 con 0x7f677c100a70 2024-03-27T06:54:23.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.375+0000 7f6781b5c640 1 -- 172.21.15.16:0/4090544050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f677c1a7390 con 0x7f677c100a70 2024-03-27T06:54:23.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.375+0000 7f6778ff9640 1 -- 172.21.15.16:0/4090544050 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f676c0a0970 con 0x7f677c100a70 2024-03-27T06:54:23.370 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 -- 172.21.15.16:0/4090544050 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f67640a3780 msgr2=0x7f67640a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f67640a3780 0x7f67640a5c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f67700095f0 tx=0x7f6770005f90 comp rx=0 tx=0).stop 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 -- 172.21.15.16:0/4090544050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c100a70 msgr2=0x7f677c10e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c100a70 0x7f677c10e3d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f676c00e9f0 tx=0x7f676c00eeb0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 -- 172.21.15.16:0/4090544050 shutdown_connections 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f677c106a80 0x7f677c10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677c101430 0x7f677c10e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f67640a3780 0x7f67640a5c40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 --2- 172.21.15.16:0/4090544050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c100a70 0x7f677c10e3d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 -- 172.21.15.16:0/4090544050 >> 172.21.15.16:0/4090544050 conn(0x7f677c077bb0 msgr2=0x7f677c1034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 -- 172.21.15.16:0/4090544050 shutdown_connections 2024-03-27T06:54:23.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.377+0000 7f6781b5c640 1 -- 172.21.15.16:0/4090544050 wait complete. 2024-03-27T06:54:23.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-27T06:54:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.494+0000 7fbeaff77640 1 -- 172.21.15.16:0/2103570493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea8107b80 msgr2=0x7fbea8107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.494+0000 7fbeaff77640 1 --2- 172.21.15.16:0/2103570493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea8107b80 0x7fbea8107f60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009930 tx=0x7fbea002bc30 comp rx=0 tx=0).stop 2024-03-27T06:54:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.494+0000 7fbeaff77640 1 -- 172.21.15.16:0/2103570493 shutdown_connections 2024-03-27T06:54:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.494+0000 7fbeaff77640 1 --2- 172.21.15.16:0/2103570493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbea81024a0 0x7fbea810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.494+0000 7fbeaff77640 1 --2- 172.21.15.16:0/2103570493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea8101ae0 0x7fbea8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.494+0000 7fbeaff77640 1 --2- 172.21.15.16:0/2103570493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea8107b80 0x7fbea8107f60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.494+0000 7fbeaff77640 1 -- 172.21.15.16:0/2103570493 >> 172.21.15.16:0/2103570493 conn(0x7fbea8078f50 msgr2=0x7fbea8079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeaff77640 1 -- 172.21.15.16:0/2103570493 shutdown_connections 2024-03-27T06:54:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeaff77640 1 -- 172.21.15.16:0/2103570493 wait complete. 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeaff77640 1 Processor -- start 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeaff77640 1 -- start start 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeaff77640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea8101ae0 0x7fbea8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeaff77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbea81024a0 0x7fbea8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeaff77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea8107b80 0x7fbea819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeaff77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8111fa0 con 0x7fbea8107b80 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeaff77640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8111e20 con 0x7fbea8101ae0 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeaff77640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8112120 con 0x7fbea81024a0 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeae514640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea8107b80 0x7fbea819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeae514640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea8107b80 0x7fbea819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44226/0 (socket says 172.21.15.16:44226) 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeae514640 1 -- 172.21.15.16:0/1790985554 learned_addr learned my addr 172.21.15.16:0/1790985554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbead512640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbea81024a0 0x7fbea8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.495+0000 7fbeadd13640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea8101ae0 0x7fbea8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.496+0000 7fbeae514640 1 -- 172.21.15.16:0/1790985554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbea81024a0 msgr2=0x7fbea8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.496+0000 7fbeae514640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbea81024a0 0x7fbea8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.496+0000 7fbeae514640 1 -- 172.21.15.16:0/1790985554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea8101ae0 msgr2=0x7fbea8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.496+0000 7fbeae514640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea8101ae0 0x7fbea8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.496+0000 7fbeae514640 1 -- 172.21.15.16:0/1790985554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea819d4f0 con 0x7fbea8107b80 2024-03-27T06:54:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.496+0000 7fbeae514640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea8107b80 0x7fbea819cdc0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800efd0 tx=0x7fbe9800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.496+0000 7fbea6ffd640 1 -- 172.21.15.16:0/1790985554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98003b70 con 0x7fbea8107b80 2024-03-27T06:54:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.496+0000 7fbea6ffd640 1 -- 172.21.15.16:0/1790985554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe98003d10 con 0x7fbea8107b80 2024-03-27T06:54:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.496+0000 7fbea6ffd640 1 -- 172.21.15.16:0/1790985554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98010a90 con 0x7fbea8107b80 2024-03-27T06:54:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.496+0000 7fbeaff77640 1 -- 172.21.15.16:0/1790985554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea819d7e0 con 0x7fbea8107b80 2024-03-27T06:54:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.496+0000 7fbeaff77640 1 -- 172.21.15.16:0/1790985554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea81a5070 con 0x7fbea8107b80 2024-03-27T06:54:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.497+0000 7fbeaff77640 1 -- 172.21.15.16:0/1790985554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea8068d90 con 0x7fbea8107b80 2024-03-27T06:54:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.498+0000 7fbea6ffd640 1 -- 172.21.15.16:0/1790985554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fbe98004000 con 0x7fbea8107b80 2024-03-27T06:54:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.498+0000 7fbea6ffd640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe900a3800 0x7fbe900a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.499+0000 7fbeadd13640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe900a3800 0x7fbe900a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.499+0000 7fbea6ffd640 1 -- 172.21.15.16:0/1790985554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fbe980d0490 con 0x7fbea8107b80 2024-03-27T06:54:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.499+0000 7fbeadd13640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe900a3800 0x7fbe900a5cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbea0005ab0 tx=0x7fbea0005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:23.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.503+0000 7fbea6ffd640 1 -- 172.21.15.16:0/1790985554 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fbe9809cd20 con 0x7fbea8107b80 2024-03-27T06:54:23.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.691+0000 7fbeaff77640 1 -- 172.21.15.16:0/1790985554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbea802d370 con 0x7fbea8107b80 2024-03-27T06:54:23.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.691+0000 7fbea6ffd640 1 -- 172.21.15.16:0/1790985554 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fbe9809c620 con 0x7fbea8107b80 2024-03-27T06:54:23.686 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-03-27T06:54:23.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 -- 172.21.15.16:0/1790985554 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe900a3800 msgr2=0x7fbe900a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe900a3800 0x7fbe900a5cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbea0005ab0 tx=0x7fbea0005fd0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 -- 172.21.15.16:0/1790985554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea8107b80 msgr2=0x7fbea819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea8107b80 0x7fbea819cdc0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800efd0 tx=0x7fbe9800c6a0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 -- 172.21.15.16:0/1790985554 shutdown_connections 2024-03-27T06:54:23.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbea8107b80 0x7fbea819cdc0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fbe900a3800 0x7fbe900a5cc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbea81024a0 0x7fbea8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 --2- 172.21.15.16:0/1790985554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea8101ae0 0x7fbea8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 -- 172.21.15.16:0/1790985554 >> 172.21.15.16:0/1790985554 conn(0x7fbea8078f50 msgr2=0x7fbea81045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:23.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 -- 172.21.15.16:0/1790985554 shutdown_connections 2024-03-27T06:54:23.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.693+0000 7fbeaff77640 1 -- 172.21.15.16:0/1790985554 wait complete. 2024-03-27T06:54:23.697 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:23 smithi016 ceph-mon[31908]: pgmap v338: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:54:23.697 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:23 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2051628582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:54:23.698 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:23 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1043694663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:54:23.698 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:23 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/4090544050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:54:23.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-27T06:54:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.808+0000 7f23f7476640 1 -- 172.21.15.16:0/3552590323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f01036e0 msgr2=0x7f23f010bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.808+0000 7f23f7476640 1 --2- 172.21.15.16:0/3552590323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f01036e0 0x7f23f010bbc0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009900 tx=0x7f23e002bcd0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.808+0000 7f23f7476640 1 -- 172.21.15.16:0/3552590323 shutdown_connections 2024-03-27T06:54:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.808+0000 7f23f7476640 1 --2- 172.21.15.16:0/3552590323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f010c190 0x7f23f010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.808+0000 7f23f7476640 1 --2- 172.21.15.16:0/3552590323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f01036e0 0x7f23f010bbc0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.808+0000 7f23f7476640 1 --2- 172.21.15.16:0/3552590323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f0102dc0 0x7f23f01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.808+0000 7f23f7476640 1 -- 172.21.15.16:0/3552590323 >> 172.21.15.16:0/3552590323 conn(0x7f23f0077bb0 msgr2=0x7f23f0077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.808+0000 7f23f7476640 1 -- 172.21.15.16:0/3552590323 shutdown_connections 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.808+0000 7f23f7476640 1 -- 172.21.15.16:0/3552590323 wait complete. 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f7476640 1 Processor -- start 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f7476640 1 -- start start 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f7476640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f0102dc0 0x7f23f0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f7476640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f01036e0 0x7f23f0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f7476640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f010c190 0x7f23f019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f7476640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23f01120d0 con 0x7f23f010c190 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f7476640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0111f50 con 0x7f23f0102dc0 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f7476640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0112250 con 0x7f23f01036e0 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f5a13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f010c190 0x7f23f019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f5a13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f010c190 0x7f23f019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44236/0 (socket says 172.21.15.16:44236) 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f5a13640 1 -- 172.21.15.16:0/2798798973 learned_addr learned my addr 172.21.15.16:0/2798798973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f4a11640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f01036e0 0x7f23f0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f5212640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f0102dc0 0x7f23f0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f5a13640 1 -- 172.21.15.16:0/2798798973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f01036e0 msgr2=0x7f23f0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f5a13640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f01036e0 0x7f23f0198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f5a13640 1 -- 172.21.15.16:0/2798798973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f0102dc0 msgr2=0x7f23f0198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f5a13640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f0102dc0 0x7f23f0198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f5a13640 1 -- 172.21.15.16:0/2798798973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f019d510 con 0x7f23f010c190 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.809+0000 7f23f5a13640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f010c190 0x7f23f019cde0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f23e800eb10 tx=0x7f23e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.810+0000 7f23e67fc640 1 -- 172.21.15.16:0/2798798973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8004b20 con 0x7f23f010c190 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.810+0000 7f23e67fc640 1 -- 172.21.15.16:0/2798798973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23e8004cc0 con 0x7f23f010c190 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.810+0000 7f23f7476640 1 -- 172.21.15.16:0/2798798973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f019d800 con 0x7f23f010c190 2024-03-27T06:54:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.810+0000 7f23e67fc640 1 -- 172.21.15.16:0/2798798973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e80107f0 con 0x7f23f010c190 2024-03-27T06:54:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.810+0000 7f23f7476640 1 -- 172.21.15.16:0/2798798973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f01a5090 con 0x7f23f010c190 2024-03-27T06:54:23.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.810+0000 7f23f7476640 1 -- 172.21.15.16:0/2798798973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f0068d90 con 0x7f23f010c190 2024-03-27T06:54:23.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.812+0000 7f23e67fc640 1 -- 172.21.15.16:0/2798798973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f23e8010990 con 0x7f23f010c190 2024-03-27T06:54:23.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.812+0000 7f23e67fc640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f23dc0a3780 0x7f23dc0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:23.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.812+0000 7f23f5212640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f23dc0a3780 0x7f23dc0a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:23.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.812+0000 7f23e67fc640 1 -- 172.21.15.16:0/2798798973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f23e80d07e0 con 0x7f23f010c190 2024-03-27T06:54:23.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.813+0000 7f23f5212640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f23dc0a3780 0x7f23dc0a5c40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009bd0 tx=0x7f23ec005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:23.816+0000 7f23e67fc640 1 -- 172.21.15.16:0/2798798973 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f23e8053cb0 con 0x7f23f010c190 2024-03-27T06:54:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:23 smithi170 ceph-mon[36438]: pgmap v338: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:54:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:23 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2051628582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:54:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:23 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1043694663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:54:23.844 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:23 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/4090544050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:54:23.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:23 smithi156 ceph-mon[36261]: pgmap v338: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:54:23.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:23 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2051628582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:54:23.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:23 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1043694663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-27T06:54:23.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:23 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/4090544050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-27T06:54:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.014+0000 7f23f7476640 1 -- 172.21.15.16:0/2798798973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f23f0199870 con 0x7f23f010c190 2024-03-27T06:54:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.014+0000 7f23e67fc640 1 -- 172.21.15.16:0/2798798973 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+944 (secure 0 0 0) 0x7f23f0199870 con 0x7f23f010c190 2024-03-27T06:54:24.009 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/4198806087 2024-03-28T06:43:33.780800+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3655179049 2024-03-28T06:43:33.780800+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3519122217 2024-03-28T06:43:33.780800+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/4198806087 2024-03-28T06:43:33.780800+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/212881673 2024-03-28T06:43:33.780800+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3456912888 2024-03-28T06:34:19.478814+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1685933023 2024-03-28T06:34:19.478814+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1685933023 2024-03-28T06:34:19.478814+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2765383798 2024-03-28T06:34:19.478814+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2967489845 2024-03-28T06:33:51.395962+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3026086781 2024-03-28T06:33:51.395962+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2945509975 2024-03-28T06:33:51.395962+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1221908125 2024-03-28T06:33:51.395962+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/361067788 2024-03-28T06:43:33.780800+0000 2024-03-27T06:54:24.010 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1221908125 2024-03-28T06:33:51.395962+0000 2024-03-27T06:54:24.011 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1636919118 2024-03-28T06:34:19.478814+0000 2024-03-27T06:54:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:listed 16 entries 2024-03-27T06:54:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 -- 172.21.15.16:0/2798798973 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f23dc0a3780 msgr2=0x7f23dc0a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f23dc0a3780 0x7f23dc0a5c40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009bd0 tx=0x7f23ec005f90 comp rx=0 tx=0).stop 2024-03-27T06:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 -- 172.21.15.16:0/2798798973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f010c190 msgr2=0x7f23f019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f010c190 0x7f23f019cde0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f23e800eb10 tx=0x7f23e800c2d0 comp rx=0 tx=0).stop 2024-03-27T06:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 -- 172.21.15.16:0/2798798973 shutdown_connections 2024-03-27T06:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f010c190 0x7f23f019cde0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f23dc0a3780 0x7f23dc0a5c40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f01036e0 0x7f23f0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 --2- 172.21.15.16:0/2798798973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23f0102dc0 0x7f23f0198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 -- 172.21.15.16:0/2798798973 >> 172.21.15.16:0/2798798973 conn(0x7f23f0077bb0 msgr2=0x7f23f00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 -- 172.21.15.16:0/2798798973 shutdown_connections 2024-03-27T06:54:24.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.016+0000 7f23f7476640 1 -- 172.21.15.16:0/2798798973 wait complete. 2024-03-27T06:54:24.022 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-27T06:54:24.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:54:24.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-27T06:54:24.336 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-03-27T06:54:24.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:54:24.337 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-27T06:54:24.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:54:24.375 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-03-27T06:54:24.431 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-27T06:54:24.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:54:24.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-27T06:54:24.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:54:24.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-03-27T06:54:24.603 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-27T06:54:24.603 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-27T06:54:24.603 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-27T06:54:24.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057145,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18624 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/a24dacfe97c5d27a5dd2c97361fcae094634cee2fdfbd50a3c66e7eb252f3e85/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MSBFMCNFUNB5CEIAAZ6G4JXYKR:/var/lib/containers/storage/overlay/l/BD6W6VU5Y66C4KT3MFVJQ5HFMB,upperdir=/var/lib/containers/storage/overlay/a24dacfe97c5d27a5dd2c97361fcae094634cee2fdfbd50a3c66e7eb252f3e85/diff,workdir=/var/lib/containers/storage/overlay/a24dacfe97c5d27a5dd2c97361fcae094634cee2fdfbd50a3c66e7eb252f3e85/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ac26f692833adce3dfcbae546a16f4a076711bd9b2f1434511c04d2d6a8eebe3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MSBFMCNFUNB5CEIAAZ6G4JXYKR:/var/lib/containers/storage/overlay/l/BD6W6VU5Y66C4KT3MFVJQ5HFMB,upperdir=/var/lib/containers/storage/overlay/ac26f692833adce3dfcbae546a16f4a076711bd9b2f1434511c04d2d6a8eebe3/diff,workdir=/var/lib/containers/storage/overlay/ac26f692833adce3dfcbae546a16f4a076711bd9b2f1434511c04d2d6a8eebe3/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/e5984692cf3933a29cba234107c08703d3e85f48ff61d69343e9d98f08c3d70c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MSBFMCNFUNB5CEIAAZ6G4JXYKR:/var/lib/containers/storage/overlay/l/BD6W6VU5Y66C4KT3MFVJQ5HFMB,upperdir=/var/lib/containers/storage/overlay/e5984692cf3933a29cba234107c08703d3e85f48ff61d69343e9d98f08c3d70c/diff,workdir=/var/lib/containers/storage/overlay/e5984692cf3933a29cba234107c08703d3e85f48ff61d69343e9d98f08c3d70c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/98d98ebedea1fb357ca00cd71bad705307805b81057ee23a660f5fbaed0cf2f9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MSBFMCNFUNB5CEIAAZ6G4JXYKR:/var/lib/containers/storage/overlay/l/BD6W6VU5Y66C4KT3MFVJQ5HFMB,upperdir=/var/lib/containers/storage/overlay/98d98ebedea1fb357ca00cd71bad705307805b81057ee23a660f5fbaed0cf2f9/diff,workdir=/var/lib/containers/storage/overlay/98d98ebedea1fb357ca00cd71bad705307805b81057ee23a660f5fbaed0cf2f9/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/588ea785c7f49564d492bdc3a271192b2095a87ff4ba9f9acd6605071927a7f7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MSBFMCNFUNB5CEIAAZ6G4JXYKR:/var/lib/containers/storage/overlay/l/BD6W6VU5Y66C4KT3MFVJQ5HFMB,upperdir=/var/lib/containers/storage/overlay/588ea785c7f49564d492bdc3a271192b2095a87ff4ba9f9acd6605071927a7f7/diff,workdir=/var/lib/containers/storage/overlay/588ea785c7f49564d492bdc3a271192b2095a87ff4ba9f9acd6605071927a7f7/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/be3b4c2063aa337ac09d2350f35f5ac9ff7e4b6694ee1489564cb2967b7bd5ab/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/MSBFMCNFUNB5CEIAAZ6G4JXYKR:/var/lib/containers/storage/overlay/l/BD6W6VU5Y66C4KT3MFVJQ5HFMB,upperdir=/var/lib/containers/storage/overlay/be3b4c2063aa337ac09d2350f35f5ac9ff7e4b6694ee1489564cb2967b7bd5ab/diff,workdir=/var/lib/containers/storage/overlay/be3b4c2063aa337ac09d2350f35f5ac9ff7e4b6694ee1489564cb2967b7bd5ab/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/479d62fd5d7ab9631fcf5b5191433533ed768359ed6b180f2635454022b672db/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MSBFMCNFUNB5CEIAAZ6G4JXYKR:/var/lib/containers/storage/overlay/l/BD6W6VU5Y66C4KT3MFVJQ5HFMB,upperdir=/var/lib/containers/storage/overlay/479d62fd5d7ab9631fcf5b5191433533ed768359ed6b180f2635454022b672db/diff,workdir=/var/lib/containers/storage/overlay/479d62fd5d7ab9631fcf5b5191433533ed768359ed6b180f2635454022b672db/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/aa2a9e86a23dddb2ce861668aced430c1fa3673215fcb171c73c09b0e15b7768/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MSBFMCNFUNB5CEIAAZ6G4JXYKR:/var/lib/containers/storage/overlay/l/BD6W6VU5Y66C4KT3MFVJQ5HFMB,upperdir=/var/lib/containers/storage/overlay/aa2a9e86a23dddb2ce861668aced430c1fa3673215fcb171c73c09b0e15b7768/diff,workdir=/var/lib/containers/storage/overlay/aa2a9e86a23dddb2ce861668aced430c1fa3673215fcb171c73c09b0e15b7768/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a9e51a9f486cd86e3775c15a1ae5c1e2ab9cabe543c7ed2c7533ef480f9e777b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MSBFMCNFUNB5CEIAAZ6G4JXYKR:/var/lib/containers/storage/overlay/l/BD6W6VU5Y66C4KT3MFVJQ5HFMB,upperdir=/var/lib/containers/storage/overlay/a9e51a9f486cd86e3775c15a1ae5c1e2ab9cabe543c7ed2c7533ef480f9e777b/diff,workdir=/var/lib/containers/storage/overlay/a9e51a9f486cd86e3775c15a1ae5c1e2ab9cabe543c7ed2c7533ef480f9e777b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e293705c24006b02ed623770af79178d102fa65d699778f79e99dff31b340796/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/MSBFMCNFUNB5CEIAAZ6G4JXYKR:/var/lib/containers/storage/overlay/l/BD6W6VU5Y66C4KT3MFVJQ5HFMB,upperdir=/var/lib/containers/storage/overlay/e293705c24006b02ed623770af79178d102fa65d699778f79e99dff31b340796/diff,workdir=/var/lib/containers/storage/overlay/e293705c24006b02ed623770af79178d102fa65d699778f79e99dff31b340796/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 2024-03-27T06:54:24.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:54:24.621 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-03-27T06:54:24.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-27T06:54:24.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-03-27T06:54:24.745 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-03-27T06:54:24.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:54:24.747 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-27T06:54:24.757 INFO:tasks.cephadm:Teardown begin 2024-03-27T06:54:24.758 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-27T06:54:24.758 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:54:24.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:24 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1790985554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:54:24.862 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:24 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2798798973' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-27T06:54:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.897+0000 7fa94d3f7640 1 -- 172.21.15.16:0/3919381122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948106a00 msgr2=0x7fa948106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.897+0000 7fa94d3f7640 1 --2- 172.21.15.16:0/3919381122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948106a00 0x7fa948106de0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa940009930 tx=0x7fa94002bc30 comp rx=0 tx=0).stop 2024-03-27T06:54:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.897+0000 7fa94d3f7640 1 -- 172.21.15.16:0/3919381122 shutdown_connections 2024-03-27T06:54:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.897+0000 7fa94d3f7640 1 --2- 172.21.15.16:0/3919381122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9481013b0 0x7fa94810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.897+0000 7fa94d3f7640 1 --2- 172.21.15.16:0/3919381122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9481009f0 0x7fa948100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.897+0000 7fa94d3f7640 1 --2- 172.21.15.16:0/3919381122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948106a00 0x7fa948106de0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.897+0000 7fa94d3f7640 1 -- 172.21.15.16:0/3919381122 >> 172.21.15.16:0/3919381122 conn(0x7fa948076dd0 msgr2=0x7fa9480771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.897+0000 7fa94d3f7640 1 -- 172.21.15.16:0/3919381122 shutdown_connections 2024-03-27T06:54:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.897+0000 7fa94d3f7640 1 -- 172.21.15.16:0/3919381122 wait complete. 2024-03-27T06:54:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa94d3f7640 1 Processor -- start 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa94d3f7640 1 -- start start 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa94d3f7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9481009f0 0x7fa948198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa94d3f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9481013b0 0x7fa9481989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa94d3f7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa948106a00 0x7fa94819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa94d3f7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa948110ee0 con 0x7fa9481013b0 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa94d3f7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fa948110d60 con 0x7fa9481009f0 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa94d3f7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa948111060 con 0x7fa948106a00 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa9467fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9481013b0 0x7fa9481989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa9467fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9481013b0 0x7fa9481989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44264/0 (socket says 172.21.15.16:44264) 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa9467fc640 1 -- 172.21.15.16:0/439597598 learned_addr learned my addr 172.21.15.16:0/439597598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa946ffd640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9481009f0 0x7fa948198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa9477fe640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa948106a00 0x7fa94819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa9467fc640 1 -- 172.21.15.16:0/439597598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa948106a00 msgr2=0x7fa94819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa9467fc640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa948106a00 0x7fa94819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa9467fc640 1 -- 172.21.15.16:0/439597598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9481009f0 msgr2=0x7fa948198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa9467fc640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9481009f0 0x7fa948198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.898+0000 7fa9467fc640 1 -- 172.21.15.16:0/439597598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94819d450 con 0x7fa9481013b0 2024-03-27T06:54:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.899+0000 7fa9467fc640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9481013b0 0x7fa9481989a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa93000e9f0 tx=0x7fa93000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.899+0000 7fa93ffff640 1 -- 172.21.15.16:0/439597598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93000cd80 con 0x7fa9481013b0 2024-03-27T06:54:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.899+0000 7fa93ffff640 1 -- 172.21.15.16:0/439597598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa930010930 con 0x7fa9481013b0 2024-03-27T06:54:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.899+0000 7fa93ffff640 1 -- 172.21.15.16:0/439597598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930018b00 con 0x7fa9481013b0 2024-03-27T06:54:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.899+0000 7fa94d3f7640 1 -- 172.21.15.16:0/439597598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94819d740 con 0x7fa9481013b0 2024-03-27T06:54:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.899+0000 7fa94d3f7640 1 -- 172.21.15.16:0/439597598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9481a4fd0 con 0x7fa9481013b0 2024-03-27T06:54:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.900+0000 7fa94d3f7640 1 -- 172.21.15.16:0/439597598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa948102130 con 0x7fa9481013b0 2024-03-27T06:54:24.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.901+0000 7fa93ffff640 1 -- 172.21.15.16:0/439597598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fa930018ca0 con 0x7fa9481013b0 2024-03-27T06:54:24.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.901+0000 7fa93ffff640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9340a3780 0x7fa9340a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:24.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.902+0000 7fa946ffd640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9340a3780 0x7fa9340a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:24.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.902+0000 7fa93ffff640 1 -- 172.21.15.16:0/439597598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fa9300d0780 con 0x7fa9481013b0 2024-03-27T06:54:24.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.902+0000 7fa946ffd640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9340a3780 0x7fa9340a5c40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa940005ab0 tx=0x7fa940005fd0 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:24.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:24.905+0000 7fa93ffff640 1 -- 172.21.15.16:0/439597598 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fa93009c9c0 con 0x7fa9481013b0 2024-03-27T06:54:24.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:24 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1790985554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:54:24.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:24 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2798798973' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-27T06:54:25.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:24 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1790985554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-27T06:54:25.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:24 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2798798973' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-27T06:54:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.129+0000 7fa94d3f7640 1 -- 172.21.15.16:0/439597598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa948068d90 con 0x7fa9481013b0 2024-03-27T06:54:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.129+0000 7fa93ffff640 1 -- 172.21.15.16:0/439597598 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11933 (secure 0 0 0) 0x7fa948068d90 con 0x7fa9481013b0 2024-03-27T06:54:25.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:54:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:49:29.272810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":67,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":65,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":65,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":64,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25852":{"gid":25852,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/681514902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":681514902},{"type":"v1","addr":"172.21.15.170:6835","nonce":681514902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24493,24472,24500,24467,24527]},"id":1}]} 2024-03-27T06:54:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 58 2024-03-27T06:54:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.131+0000 7fa94d3f7640 1 -- 172.21.15.16:0/439597598 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9340a3780 msgr2=0x7fa9340a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.131+0000 7fa94d3f7640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9340a3780 0x7fa9340a5c40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa940005ab0 tx=0x7fa940005fd0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.131+0000 7fa94d3f7640 1 -- 172.21.15.16:0/439597598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9481013b0 msgr2=0x7fa9481989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.131+0000 7fa94d3f7640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9481013b0 0x7fa9481989a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa93000e9f0 tx=0x7fa93000eeb0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.131+0000 7fa94d3f7640 1 -- 172.21.15.16:0/439597598 shutdown_connections 2024-03-27T06:54:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.131+0000 7fa94d3f7640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fa9340a3780 0x7fa9340a5c40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.131+0000 7fa94d3f7640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa948106a00 0x7fa94819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.131+0000 7fa94d3f7640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9481013b0 0x7fa9481989a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.131+0000 7fa94d3f7640 1 --2- 172.21.15.16:0/439597598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9481009f0 0x7fa948198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.131+0000 7fa94d3f7640 1 -- 172.21.15.16:0/439597598 >> 172.21.15.16:0/439597598 conn(0x7fa948076dd0 msgr2=0x7fa94806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.132+0000 7fa94d3f7640 1 -- 172.21.15.16:0/439597598 shutdown_connections 2024-03-27T06:54:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.132+0000 7fa94d3f7640 1 -- 172.21.15.16:0/439597598 wait complete. 2024-03-27T06:54:25.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-27T06:54:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.245+0000 7fb674c0e640 1 -- 172.21.15.16:0/1803193153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67006a450 msgr2=0x7fb67006a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.245+0000 7fb674c0e640 1 --2- 172.21.15.16:0/1803193153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67006a450 0x7fb67006a830 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb668009900 tx=0x7fb66802bc80 comp rx=0 tx=0).stop 2024-03-27T06:54:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.245+0000 7fb674c0e640 1 -- 172.21.15.16:0/1803193153 shutdown_connections 2024-03-27T06:54:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.245+0000 7fb674c0e640 1 --2- 172.21.15.16:0/1803193153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb670106ee0 0x7fb67010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.245+0000 7fb674c0e640 1 --2- 172.21.15.16:0/1803193153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb67006ae00 0x7fb6701067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.245+0000 7fb674c0e640 1 --2- 172.21.15.16:0/1803193153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67006a450 0x7fb67006a830 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.245+0000 7fb674c0e640 1 -- 172.21.15.16:0/1803193153 >> 172.21.15.16:0/1803193153 conn(0x7fb670077af0 msgr2=0x7fb670077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.245+0000 7fb674c0e640 1 -- 172.21.15.16:0/1803193153 shutdown_connections 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.245+0000 7fb674c0e640 1 -- 172.21.15.16:0/1803193153 wait complete. 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb674c0e640 1 Processor -- start 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb674c0e640 1 -- start start 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb674c0e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb67006a450 0x7fb6701984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb674c0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb670106ee0 0x7fb6701989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb674c0e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67019cd70 0x7fb67019d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb674c0e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb670112050 con 0x7fb670106ee0 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb674c0e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fb670111ed0 con 0x7fb67006a450 2024-03-27T06:54:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb674c0e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701121d0 con 0x7fb67019cd70 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb66dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb670106ee0 0x7fb6701989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb66dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb670106ee0 0x7fb6701989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44282/0 (socket says 172.21.15.16:44282) 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb66dd9b640 1 -- 172.21.15.16:0/1710233709 learned_addr learned my addr 172.21.15.16:0/1710233709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb66ed9d640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67019cd70 0x7fb67019d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb66e59c640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb67006a450 0x7fb6701984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb66ed9d640 1 -- 172.21.15.16:0/1710233709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb67006a450 msgr2=0x7fb6701984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb66ed9d640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb67006a450 0x7fb6701984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb66ed9d640 1 -- 172.21.15.16:0/1710233709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb670106ee0 msgr2=0x7fb6701989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb66ed9d640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb670106ee0 0x7fb6701989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb66ed9d640 1 -- 172.21.15.16:0/1710233709 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67019d860 con 0x7fb67019cd70 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.246+0000 7fb66dd9b640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb670106ee0 0x7fb6701989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.247+0000 7fb66ed9d640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67019cd70 0x7fb67019d220 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb660009fd0 tx=0x7fb66000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.247+0000 7fb6677fe640 1 -- 172.21.15.16:0/1710233709 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66000ce20 con 0x7fb67019cd70 2024-03-27T06:54:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.247+0000 7fb6677fe640 1 -- 172.21.15.16:0/1710233709 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb660004b20 con 0x7fb67019cd70 2024-03-27T06:54:25.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.247+0000 7fb674c0e640 1 -- 172.21.15.16:0/1710233709 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67019db50 con 0x7fb67019cd70 2024-03-27T06:54:25.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.247+0000 7fb6677fe640 1 -- 172.21.15.16:0/1710233709 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660010790 con 0x7fb67019cd70 2024-03-27T06:54:25.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.247+0000 7fb674c0e640 1 -- 172.21.15.16:0/1710233709 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6701a5390 con 0x7fb67019cd70 2024-03-27T06:54:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.248+0000 7fb674c0e640 1 -- 172.21.15.16:0/1710233709 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb670068d90 con 0x7fb67019cd70 2024-03-27T06:54:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.249+0000 7fb6677fe640 1 -- 172.21.15.16:0/1710233709 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fb660005000 con 0x7fb67019cd70 2024-03-27T06:54:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.249+0000 7fb6677fe640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb65c0a3800 0x7fb65c0a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.249+0000 7fb66e59c640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb65c0a3800 0x7fb65c0a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.249+0000 7fb6677fe640 1 -- 172.21.15.16:0/1710233709 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fb6600d0730 con 0x7fb67019cd70 2024-03-27T06:54:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.250+0000 7fb66e59c640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb65c0a3800 0x7fb65c0a5cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb668005ab0 tx=0x7fb668005a20 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.253+0000 7fb6677fe640 1 -- 172.21.15.16:0/1710233709 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fb66009cf60 con 0x7fb67019cd70 2024-03-27T06:54:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.480+0000 7fb674c0e640 1 -- 172.21.15.16:0/1710233709 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb670199800 con 0x7fb67019cd70 2024-03-27T06:54:25.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.481+0000 7fb6677fe640 1 -- 172.21.15.16:0/1710233709 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11933 (secure 0 0 0) 0x7fb670199800 con 0x7fb67019cd70 2024-03-27T06:54:25.476 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:54:25.476 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1554082728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1554082728},{"type":"v1","addr":"172.21.15.16:6835","nonce":1554082728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/1241440055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":1241440055},{"type":"v1","addr":"172.21.15.16:6837","nonce":1241440055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-27T06:41:28.746131+0000","modified":"2024-03-27T06:49:29.272810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24500,"mds_2":24493,"mds_3":24472,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/420981926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":420981926},{"type":"v1","addr":"172.21.15.16:6839","nonce":420981926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2365316099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2365316099},{"type":"v1","addr":"172.21.15.16:6841","nonce":2365316099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/1368007304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1368007304},{"type":"v1","addr":"172.21.15.170:6833","nonce":1368007304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6833/521338694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":521338694},{"type":"v1","addr":"172.21.15.156:6833","nonce":521338694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":4,"incarnation":23,"state":"up:active","state_seq":67,"addr":"172.21.15.156:6835/4256913032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4256913032},{"type":"v1","addr":"172.21.15.156:6835","nonce":4256913032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":3,"incarnation":21,"state":"up:active","state_seq":65,"addr":"172.21.15.170:6837/2408489138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2408489138},{"type":"v1","addr":"172.21.15.170:6837","nonce":2408489138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":65,"addr":"172.21.15.170:6839/246078270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":246078270},{"type":"v1","addr":"172.21.15.170:6839","nonce":246078270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":64,"addr":"172.21.15.156:6837/3016592835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":3016592835},{"type":"v1","addr":"172.21.15.156:6837","nonce":3016592835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6839/479168397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":479168397},{"type":"v1","addr":"172.21.15.156:6839","nonce":479168397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25852":{"gid":25852,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/681514902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":681514902},{"type":"v1","addr":"172.21.15.170:6835","nonce":681514902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24493,24472,24500,24467,24527]},"id":1}]} 2024-03-27T06:54:25.477 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 58 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.483+0000 7fb674c0e640 1 -- 172.21.15.16:0/1710233709 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb65c0a3800 msgr2=0x7fb65c0a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.483+0000 7fb674c0e640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb65c0a3800 0x7fb65c0a5cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb668005ab0 tx=0x7fb668005a20 comp rx=0 tx=0).stop 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.483+0000 7fb674c0e640 1 -- 172.21.15.16:0/1710233709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67019cd70 msgr2=0x7fb67019d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.483+0000 7fb674c0e640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67019cd70 0x7fb67019d220 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb660009fd0 tx=0x7fb66000ef10 comp rx=0 tx=0).stop 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.483+0000 7fb674c0e640 1 -- 172.21.15.16:0/1710233709 shutdown_connections 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.483+0000 7fb674c0e640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fb65c0a3800 0x7fb65c0a5cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.483+0000 7fb674c0e640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67019cd70 0x7fb67019d220 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.483+0000 7fb674c0e640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb670106ee0 0x7fb6701989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.483+0000 7fb674c0e640 1 --2- 172.21.15.16:0/1710233709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb67006a450 0x7fb6701984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.483+0000 7fb674c0e640 1 -- 172.21.15.16:0/1710233709 >> 172.21.15.16:0/1710233709 conn(0x7fb670077af0 msgr2=0x7fb6700fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.484+0000 7fb674c0e640 1 -- 172.21.15.16:0/1710233709 shutdown_connections 2024-03-27T06:54:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.484+0000 7fb674c0e640 1 -- 172.21.15.16:0/1710233709 wait complete. 2024-03-27T06:54:25.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-27T06:54:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.598+0000 7f6a259e2640 1 -- 172.21.15.16:0/1817168721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a20102dc0 msgr2=0x7f6a201031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.598+0000 7f6a259e2640 1 --2- 172.21.15.16:0/1817168721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a20102dc0 0x7f6a201031a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009900 tx=0x7f6a1002bc30 comp rx=0 tx=0).stop 2024-03-27T06:54:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.598+0000 7f6a259e2640 1 -- 172.21.15.16:0/1817168721 shutdown_connections 2024-03-27T06:54:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.598+0000 7f6a259e2640 1 --2- 172.21.15.16:0/1817168721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a2010c190 0x7f6a2010e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.598+0000 7f6a259e2640 1 --2- 172.21.15.16:0/1817168721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a201036e0 0x7f6a2010bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.598+0000 7f6a259e2640 1 --2- 172.21.15.16:0/1817168721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a20102dc0 0x7f6a201031a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.598+0000 7f6a259e2640 1 -- 172.21.15.16:0/1817168721 >> 172.21.15.16:0/1817168721 conn(0x7f6a20077b50 msgr2=0x7f6a20077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.598+0000 7f6a259e2640 1 -- 172.21.15.16:0/1817168721 shutdown_connections 2024-03-27T06:54:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.598+0000 7f6a259e2640 1 -- 172.21.15.16:0/1817168721 wait complete. 2024-03-27T06:54:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.598+0000 7f6a259e2640 1 Processor -- start 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.598+0000 7f6a259e2640 1 -- start start 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a259e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20102dc0 0x7f6a20198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a259e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a201036e0 0x7f6a20198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a259e2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a2010c190 0x7f6a2019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a259e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20112060 con 0x7f6a20102dc0 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a259e2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20111ee0 con 0x7f6a2010c190 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a259e2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6a201121e0 con 0x7f6a201036e0 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20102dc0 0x7f6a20198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20102dc0 0x7f6a20198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44300/0 (socket says 172.21.15.16:44300) 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a1effd640 1 -- 172.21.15.16:0/2867893399 learned_addr learned my addr 172.21.15.16:0/2867893399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a1e7fc640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a201036e0 0x7f6a20198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a1f7fe640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a2010c190 0x7f6a2019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a1effd640 1 -- 172.21.15.16:0/2867893399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a201036e0 msgr2=0x7f6a20198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a1effd640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a201036e0 0x7f6a20198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a1effd640 1 -- 172.21.15.16:0/2867893399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a2010c190 msgr2=0x7f6a2019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a1effd640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a2010c190 0x7f6a2019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a1effd640 1 -- 172.21.15.16:0/2867893399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2019d500 con 0x7f6a20102dc0 2024-03-27T06:54:25.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a1effd640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20102dc0 0x7f6a20198560 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6a10005940 tx=0x7f6a1002dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:25.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a249e0640 1 -- 172.21.15.16:0/2867893399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10043020 con 0x7f6a20102dc0 2024-03-27T06:54:25.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a259e2640 1 -- 172.21.15.16:0/2867893399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2019d790 con 0x7f6a20102dc0 2024-03-27T06:54:25.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a249e0640 1 -- 172.21.15.16:0/2867893399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6a10004250 con 0x7f6a20102dc0 2024-03-27T06:54:25.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a249e0640 1 -- 172.21.15.16:0/2867893399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a100038e0 con 0x7f6a20102dc0 2024-03-27T06:54:25.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.599+0000 7f6a259e2640 1 -- 172.21.15.16:0/2867893399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a201a5080 con 0x7f6a20102dc0 2024-03-27T06:54:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.600+0000 7f6a259e2640 1 -- 172.21.15.16:0/2867893399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a20068940 con 0x7f6a20102dc0 2024-03-27T06:54:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.601+0000 7f6a249e0640 1 -- 172.21.15.16:0/2867893399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f6a10046020 con 0x7f6a20102dc0 2024-03-27T06:54:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.602+0000 7f6a249e0640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f6a080a3780 0x7f6a080a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.602+0000 7f6a1e7fc640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f6a080a3780 0x7f6a080a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.602+0000 7f6a249e0640 1 -- 172.21.15.16:0/2867893399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f6a100f2660 con 0x7f6a20102dc0 2024-03-27T06:54:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.603+0000 7f6a1e7fc640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f6a080a3780 0x7f6a080a5c40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a2010df10 tx=0x7f6a14005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.606+0000 7f6a249e0640 1 -- 172.21.15.16:0/2867893399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f6a1004d020 con 0x7f6a20102dc0 2024-03-27T06:54:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.792+0000 7f6a259e2640 1 -- 172.21.15.16:0/2867893399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a20107280 con 0x7f6a20102dc0 2024-03-27T06:54:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.793+0000 7f6a249e0640 1 -- 172.21.15.16:0/2867893399 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23073 (secure 0 0 0) 0x7f6a100bf000 con 0x7f6a20102dc0 2024-03-27T06:54:25.789 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-27T06:54:25.789 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"fsid":"c9095bc4-ec03-11ee-95d0-87774f69a715","created":"2024-03-27T06:33:25.534035+0000","modified":"2024-03-27T06:54:20.631213+0000","last_up_change":"2024-03-27T06:40:40.266959+0000","last_in_change":"2024-03-27T06:40:19.698590+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-27T06:36:47.581613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-27T06:41:26.393978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-27T06:41:27.707469+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e99199df-d2f6-4677-8f3c-b96df2232304","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6803","nonce":1317357078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6805","nonce":1317357078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6809","nonce":1317357078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1317357078},{"type":"v1","addr":"172.21.15.16:6807","nonce":1317357078}]},"public_addr":"172.21.15.16:6803/1317357078","cluster_addr":"172.21.15.16:6805/1317357078","heartbeat_back_addr":"172.21.15.16:6809/1317357078","heartbeat_front_addr":"172.21.15.16:6807/1317357078","state":["exists","up"]},{"osd":1,"uuid":"948ae15e-e954-4653-9d1d-da3bac45bd28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6801","nonce":1327802995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6803","nonce":1327802995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6807","nonce":1327802995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1327802995},{"type":"v1","addr":"172.21.15.156:6805","nonce":1327802995}]},"public_addr":"172.21.15.156:6801/1327802995","cluster_addr":"172.21.15.156:6803/1327802995","heartbeat_back_addr":"172.21.15.156:6807/1327802995","heartbeat_front_addr":"172.21.15.156:6805/1327802995","state":["exists","up"]},{"osd":2,"uuid":"a7ca64c3-13dc-4b3d-9f8e-76bd783ac23c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6801","nonce":3044432379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6803","nonce":3044432379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6807","nonce":3044432379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3044432379},{"type":"v1","addr":"172.21.15.170:6805","nonce":3044432379}]},"public_addr":"172.21.15.170:6801/3044432379","cluster_addr":"172.21.15.170:6803/3044432379","heartbeat_back_addr":"172.21.15.170:6807/3044432379","heartbeat_front_addr":"172.21.15.170:6805/3044432379","state":["exists","up"]},{"osd":3,"uuid":"f77f4328-207f-48e0-bd4e-8c2cb212ae4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6811","nonce":875136606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6813","nonce":875136606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6817","nonce":875136606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":875136606},{"type":"v1","addr":"172.21.15.16:6815","nonce":875136606}]},"public_addr":"172.21.15.16:6811/875136606","cluster_addr":"172.21.15.16:6813/875136606","heartbeat_back_addr":"172.21.15.16:6817/875136606","heartbeat_front_addr":"172.21.15.16:6815/875136606","state":["exists","up"]},{"osd":4,"uuid":"78647188-4493-4037-97d1-92e1f6f82fc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6809","nonce":3654211974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6811","nonce":3654211974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6815","nonce":3654211974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":3654211974},{"type":"v1","addr":"172.21.15.156:6813","nonce":3654211974}]},"public_addr":"172.21.15.156:6809/3654211974","cluster_addr":"172.21.15.156:6811/3654211974","heartbeat_back_addr":"172.21.15.156:6815/3654211974","heartbeat_front_addr":"172.21.15.156:6813/3654211974","state":["exists","up"]},{"osd":5,"uuid":"9f4efed7-91d9-4adc-881a-ce3f44f80813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6809","nonce":493340882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6811","nonce":493340882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6815","nonce":493340882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":493340882},{"type":"v1","addr":"172.21.15.170:6813","nonce":493340882}]},"public_addr":"172.21.15.170:6809/493340882","cluster_addr":"172.21.15.170:6811/493340882","heartbeat_back_addr":"172.21.15.170:6815/493340882","heartbeat_front_addr":"172.21.15.170:6813/493340882","state":["exists","up"]},{"osd":6,"uuid":"ffbf31b8-a44e-4f81-819b-48213bf4cc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6819","nonce":237875547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6821","nonce":237875547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6825","nonce":237875547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":237875547},{"type":"v1","addr":"172.21.15.16:6823","nonce":237875547}]},"public_addr":"172.21.15.16:6819/237875547","cluster_addr":"172.21.15.16:6821/237875547","heartbeat_back_addr":"172.21.15.16:6825/237875547","heartbeat_front_addr":"172.21.15.16:6823/237875547","state":["exists","up"]},{"osd":7,"uuid":"fb83f843-12b4-4b70-9a85-dd2ca56ed9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6817","nonce":229134476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6819","nonce":229134476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6823","nonce":229134476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":229134476},{"type":"v1","addr":"172.21.15.156:6821","nonce":229134476}]},"public_addr":"172.21.15.156:6817/229134476","cluster_addr":"172.21.15.156:6819/229134476","heartbeat_back_addr":"172.21.15.156:6823/229134476","heartbeat_front_addr":"172.21.15.156:6821/229134476","state":["exists","up"]},{"osd":8,"uuid":"4849c1e0-60fa-4333-b423-20b759447717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6817","nonce":504489551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6819","nonce":504489551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6823","nonce":504489551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":504489551},{"type":"v1","addr":"172.21.15.170:6821","nonce":504489551}]},"public_addr":"172.21.15.170:6817/504489551","cluster_addr":"172.21.15.170:6819/504489551","heartbeat_back_addr":"172.21.15.170:6823/504489551","heartbeat_front_addr":"172.21.15.170:6821/504489551","state":["exists","up"]},{"osd":9,"uuid":"d6bb6987-e0df-4d98-9044-63946f8567cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6827","nonce":2308288184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6829","nonce":2308288184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6833","nonce":2308288184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2308288184},{"type":"v1","addr":"172.21.15.16:6831","nonce":2308288184}]},"public_addr":"172.21.15.16:6827/2308288184","cluster_addr":"172.21.15.16:6829/2308288184","heartbeat_back_addr":"172.21.15.16:6833/2308288184","heartbeat_front_addr":"172.21.15.16:6831/2308288184","state":["exists","up"]},{"osd":10,"uuid":"eb7e2309-4c4e-496e-9043-ff264a805f66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6825","nonce":399510561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6827","nonce":399510561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6831","nonce":399510561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":399510561},{"type":"v1","addr":"172.21.15.156:6829","nonce":399510561}]},"public_addr":"172.21.15.156:6825/399510561","cluster_addr":"172.21.15.156:6827/399510561","heartbeat_back_addr":"172.21.15.156:6831/399510561","heartbeat_front_addr":"172.21.15.156:6829/399510561","state":["exists","up"]},{"osd":11,"uuid":"a73e6132-619a-4b5c-85a7-e5c3ee40188e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6825","nonce":703899164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6827","nonce":703899164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6831","nonce":703899164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":703899164},{"type":"v1","addr":"172.21.15.170:6829","nonce":703899164}]},"public_addr":"172.21.15.170:6825/703899164","cluster_addr":"172.21.15.170:6827/703899164","heartbeat_back_addr":"172.21.15.170:6831/703899164","heartbeat_front_addr":"172.21.15.170:6829/703899164","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:35:56.506212+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:20.609030+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:36:44.343711+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:10.134133+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:37:34.991328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:00.422205+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:26.824989+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:38:52.654079+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:18.647791+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:39:45.503414+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:11.613195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-27T06:40:38.588192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3655179049":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3519122217":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6800/4198806087":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/212881673":"2024-03-28T06:43:33.780800+0000","172.21.15.16:0/3456912888":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6801/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:6800/1685933023":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2765383798":"2024-03-28T06:34:19.478814+0000","172.21.15.16:0/2967489845":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/3026086781":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/2945509975":"2024-03-28T06:33:51.395962+0000","172.21.15.16:6800/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/361067788":"2024-03-28T06:43:33.780800+0000","172.21.15.16:6801/1221908125":"2024-03-28T06:33:51.395962+0000","172.21.15.16:0/1636919118":"2024-03-28T06:34:19.478814+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-27T06:54:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 -- 172.21.15.16:0/2867893399 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f6a080a3780 msgr2=0x7f6a080a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f6a080a3780 0x7f6a080a5c40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a2010df10 tx=0x7f6a14005f90 comp rx=0 tx=0).stop 2024-03-27T06:54:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 -- 172.21.15.16:0/2867893399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20102dc0 msgr2=0x7f6a20198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20102dc0 0x7f6a20198560 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6a10005940 tx=0x7f6a1002dc90 comp rx=0 tx=0).stop 2024-03-27T06:54:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 -- 172.21.15.16:0/2867893399 shutdown_connections 2024-03-27T06:54:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a2010c190 0x7f6a2019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f6a080a3780 0x7f6a080a5c40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a201036e0 0x7f6a20198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 --2- 172.21.15.16:0/2867893399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20102dc0 0x7f6a20198560 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 -- 172.21.15.16:0/2867893399 >> 172.21.15.16:0/2867893399 conn(0x7f6a20077b50 msgr2=0x7f6a200fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 -- 172.21.15.16:0/2867893399 shutdown_connections 2024-03-27T06:54:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.796+0000 7f6a259e2640 1 -- 172.21.15.16:0/2867893399 wait complete. 2024-03-27T06:54:25.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:25 smithi016 ceph-mon[31908]: pgmap v339: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:54:25.801 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:25 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/439597598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:54:25.801 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:25 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/1710233709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:54:25.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-27T06:54:25.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.910+0000 7fedf52d4640 1 -- 172.21.15.16:0/2132797199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf01036b0 msgr2=0x7fedf010bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.910+0000 7fedf52d4640 1 --2- 172.21.15.16:0/2132797199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf01036b0 0x7fedf010bb90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009900 tx=0x7fedd802bcd0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.911+0000 7fedf52d4640 1 -- 172.21.15.16:0/2132797199 shutdown_connections 2024-03-27T06:54:25.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.911+0000 7fedf52d4640 1 --2- 172.21.15.16:0/2132797199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf010c160 0x7fedf010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.911+0000 7fedf52d4640 1 --2- 172.21.15.16:0/2132797199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf01036b0 0x7fedf010bb90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.911+0000 7fedf52d4640 1 --2- 172.21.15.16:0/2132797199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedf0102d90 0x7fedf0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.911+0000 7fedf52d4640 1 -- 172.21.15.16:0/2132797199 >> 172.21.15.16:0/2132797199 conn(0x7fedf0076dd0 msgr2=0x7fedf00771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.911+0000 7fedf52d4640 1 -- 172.21.15.16:0/2132797199 shutdown_connections 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.911+0000 7fedf52d4640 1 -- 172.21.15.16:0/2132797199 wait complete. 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.911+0000 7fedf52d4640 1 Processor -- start 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.911+0000 7fedf52d4640 1 -- start start 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.911+0000 7fedf52d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf0102d90 0x7fedf0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.911+0000 7fedf52d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedf01036b0 0x7fedf0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedf52d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf010c160 0x7fedf019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedf52d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0112030 con 0x7fedf010c160 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedf52d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0111eb0 con 0x7fedf01036b0 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedf52d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fedf01121b0 con 0x7fedf0102d90 2024-03-27T06:54:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf010c160 0x7fedf019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf010c160 0x7fedf019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50844/0 (socket says 172.21.15.16:50844) 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedef7fe640 1 -- 172.21.15.16:0/157498200 learned_addr learned my addr 172.21.15.16:0/157498200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedeeffd640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf0102d90 0x7fedf0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedee7fc640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedf01036b0 0x7fedf0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedef7fe640 1 -- 172.21.15.16:0/157498200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf0102d90 msgr2=0x7fedf0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedef7fe640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf0102d90 0x7fedf0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedef7fe640 1 -- 172.21.15.16:0/157498200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedf01036b0 msgr2=0x7fedf0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedef7fe640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedf01036b0 0x7fedf0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedef7fe640 1 -- 172.21.15.16:0/157498200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf019d4b0 con 0x7fedf010c160 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedef7fe640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf010c160 0x7fedf019cd80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fede000eb40 tx=0x7fede000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fede7fff640 1 -- 172.21.15.16:0/157498200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0004b20 con 0x7fedf010c160 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedf52d4640 1 -- 172.21.15.16:0/157498200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf019d7a0 con 0x7fedf010c160 2024-03-27T06:54:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fede7fff640 1 -- 172.21.15.16:0/157498200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fede0004cc0 con 0x7fedf010c160 2024-03-27T06:54:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fede7fff640 1 -- 172.21.15.16:0/157498200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede00108f0 con 0x7fedf010c160 2024-03-27T06:54:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.912+0000 7fedf52d4640 1 -- 172.21.15.16:0/157498200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf01a5030 con 0x7fedf010c160 2024-03-27T06:54:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.913+0000 7fedf52d4640 1 -- 172.21.15.16:0/157498200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf006a940 con 0x7fedf010c160 2024-03-27T06:54:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.914+0000 7fede7fff640 1 -- 172.21.15.16:0/157498200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fede0005020 con 0x7fedf010c160 2024-03-27T06:54:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.915+0000 7fede7fff640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feddc0a3780 0x7feddc0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.915+0000 7fede7fff640 1 -- 172.21.15.16:0/157498200 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fede00d07f0 con 0x7fedf010c160 2024-03-27T06:54:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.915+0000 7fedeeffd640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feddc0a3780 0x7feddc0a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.915+0000 7fedeeffd640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feddc0a3780 0x7feddc0a5c40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fede8009bd0 tx=0x7fede8005f90 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:25.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:25.920+0000 7fede7fff640 1 -- 172.21.15.16:0/157498200 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fede009d080 con 0x7fedf010c160 2024-03-27T06:54:25.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:25 smithi156 ceph-mon[36261]: pgmap v339: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:54:25.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:25 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/439597598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:54:25.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:25 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/1710233709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:54:26.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:25 smithi170 ceph-mon[36438]: pgmap v339: 97 pgs: 97 active+clean; 42 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-27T06:54:26.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:25 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/439597598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:54:26.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:25 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/1710233709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-27T06:54:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.143+0000 7fedf52d4640 1 -- 172.21.15.16:0/157498200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fedf0199810 con 0x7fedf010c160 2024-03-27T06:54:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.145+0000 7fede7fff640 1 -- 172.21.15.16:0/157498200 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) ==== 161+0+0 (secure 0 0 0) 0x7fede009c980 con 0x7fedf010c160 2024-03-27T06:54:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.148+0000 7fedf52d4640 1 -- 172.21.15.16:0/157498200 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feddc0a3780 msgr2=0x7feddc0a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.148+0000 7fedf52d4640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feddc0a3780 0x7feddc0a5c40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fede8009bd0 tx=0x7fede8005f90 comp rx=0 tx=0).stop 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.148+0000 7fedf52d4640 1 -- 172.21.15.16:0/157498200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf010c160 msgr2=0x7fedf019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.148+0000 7fedf52d4640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf010c160 0x7fedf019cd80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fede000eb40 tx=0x7fede000c2d0 comp rx=0 tx=0).stop 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.148+0000 7fedf52d4640 1 -- 172.21.15.16:0/157498200 shutdown_connections 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.148+0000 7fedf52d4640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf010c160 0x7fedf019cd80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.148+0000 7fedf52d4640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fedf01036b0 0x7fedf0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.148+0000 7fedf52d4640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7feddc0a3780 0x7feddc0a5c40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.148+0000 7fedf52d4640 1 --2- 172.21.15.16:0/157498200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf0102d90 0x7fedf0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.149+0000 7fedf52d4640 1 -- 172.21.15.16:0/157498200 >> 172.21.15.16:0/157498200 conn(0x7fedf0076dd0 msgr2=0x7fedf00fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.149+0000 7fedf52d4640 1 -- 172.21.15.16:0/157498200 shutdown_connections 2024-03-27T06:54:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.149+0000 7fedf52d4640 1 -- 172.21.15.16:0/157498200 wait complete. 2024-03-27T06:54:26.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-27T06:54:26.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.276+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/713417144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d84101380 msgr2=0x7f7d8410d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.276+0000 7f7d8c6c9640 1 --2- 172.21.15.16:0/713417144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d84101380 0x7f7d8410d890 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009930 tx=0x7f7d7c02bc30 comp rx=0 tx=0).stop 2024-03-27T06:54:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.276+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/713417144 shutdown_connections 2024-03-27T06:54:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.276+0000 7f7d8c6c9640 1 --2- 172.21.15.16:0/713417144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d84101380 0x7f7d8410d890 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.276+0000 7f7d8c6c9640 1 --2- 172.21.15.16:0/713417144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d841009c0 0x7f7d84100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.276+0000 7f7d8c6c9640 1 --2- 172.21.15.16:0/713417144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d841069d0 0x7f7d84106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.276+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/713417144 >> 172.21.15.16:0/713417144 conn(0x7f7d84077b50 msgr2=0x7f7d84077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/713417144 shutdown_connections 2024-03-27T06:54:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/713417144 wait complete. 2024-03-27T06:54:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8c6c9640 1 Processor -- start 2024-03-27T06:54:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8c6c9640 1 -- start start 2024-03-27T06:54:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8c6c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d841009c0 0x7f7d84102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8c6c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d84101380 0x7f7d84103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8c6c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d841069d0 0x7f7d84103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8c6c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84110eb0 con 0x7f7d841009c0 2024-03-27T06:54:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8c6c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84110d30 con 0x7f7d841069d0 2024-03-27T06:54:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8c6c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84111030 con 0x7f7d84101380 2024-03-27T06:54:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8a465640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d841009c0 0x7f7d84102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8a465640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d841009c0 0x7f7d84102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50910/0 (socket says 172.21.15.16:50910) 2024-03-27T06:54:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.277+0000 7f7d8a465640 1 -- 172.21.15.16:0/3093170537 learned_addr learned my addr 172.21.15.16:0/3093170537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d89c64640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d84101380 0x7f7d84103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d8ac66640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d841069d0 0x7f7d84103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d8a465640 1 -- 172.21.15.16:0/3093170537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d84101380 msgr2=0x7f7d84103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d8a465640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d84101380 0x7f7d84103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d8a465640 1 -- 172.21.15.16:0/3093170537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d841069d0 msgr2=0x7f7d84103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d8a465640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d841069d0 0x7f7d84103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d8a465640 1 -- 172.21.15.16:0/3093170537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d841a6980 con 0x7f7d841009c0 2024-03-27T06:54:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d89c64640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d84101380 0x7f7d84103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-27T06:54:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d8a465640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d841009c0 0x7f7d84102ef0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800db80 tx=0x7f7d7800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d837fe640 1 -- 172.21.15.16:0/3093170537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78017070 con 0x7f7d841009c0 2024-03-27T06:54:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d837fe640 1 -- 172.21.15.16:0/3093170537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7d78003b70 con 0x7f7d841009c0 2024-03-27T06:54:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/3093170537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d841a6c70 con 0x7f7d841009c0 2024-03-27T06:54:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d837fe640 1 -- 172.21.15.16:0/3093170537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780106c0 con 0x7f7d841009c0 2024-03-27T06:54:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.278+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/3093170537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d841a71c0 con 0x7f7d841009c0 2024-03-27T06:54:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.279+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/3093170537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d84068940 con 0x7f7d841009c0 2024-03-27T06:54:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.281+0000 7f7d837fe640 1 -- 172.21.15.16:0/3093170537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f7d78004a40 con 0x7f7d841009c0 2024-03-27T06:54:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.281+0000 7f7d837fe640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d6c0a3780 0x7f7d6c0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.282+0000 7f7d89c64640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d6c0a3780 0x7f7d6c0a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.282+0000 7f7d837fe640 1 -- 172.21.15.16:0/3093170537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8899+0+0 (secure 0 0 0) 0x7f7d780d0440 con 0x7f7d841009c0 2024-03-27T06:54:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.282+0000 7f7d89c64640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d6c0a3780 0x7f7d6c0a5c40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7d740058c0 tx=0x7f7d74005850 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.285+0000 7f7d837fe640 1 -- 172.21.15.16:0/3093170537 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f7d78020070 con 0x7f7d841009c0 2024-03-27T06:54:26.473 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: ignoring --setuser ceph since I am not root 2024-03-27T06:54:26.473 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: ignoring --setgroup ceph since I am not root 2024-03-27T06:54:26.473 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mds[66469]: -- 172.21.15.156:0/530468120 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597bfa79a40 con 0x5597bfa4c800 2024-03-27T06:54:26.473 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: starting mds.h at 2024-03-27T06:54:26.473 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k[65834]: ignoring --setuser ceph since I am not root 2024-03-27T06:54:26.473 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k[65834]: ignoring --setgroup ceph since I am not root 2024-03-27T06:54:26.474 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k[65834]: starting mds.k at 2024-03-27T06:54:26.474 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e[66157]: ignoring --setuser ceph since I am not root 2024-03-27T06:54:26.474 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e[66157]: ignoring --setgroup ceph since I am not root 2024-03-27T06:54:26.474 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mds[66161]: -- 172.21.15.156:0/4067703986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d534513a40 con 0x55d5344e7000 2024-03-27T06:54:26.474 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e[66157]: starting mds.e at 2024-03-27T06:54:26.474 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b[66776]: ignoring --setuser ceph since I am not root 2024-03-27T06:54:26.474 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b[66776]: ignoring --setgroup ceph since I am not root 2024-03-27T06:54:26.475 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mds[66780]: -- 172.21.15.156:0/1591066506 <== mon.2 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d95f6bfa40 con 0x55d95f692c00 2024-03-27T06:54:26.475 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b[66776]: starting mds.b at 2024-03-27T06:54:26.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:26.514+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/3093170537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7d841099b0 con 0x7f7d841009c0 2024-03-27T06:54:26.595 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: ignoring --setuser ceph since I am not root 2024-03-27T06:54:26.595 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: ignoring --setgroup ceph since I am not root 2024-03-27T06:54:26.595 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mds[67882]: -- 172.21.15.170:0/3471972754 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2cab7fa40 con 0x55b2cab52800 2024-03-27T06:54:26.595 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: starting mds.i at 2024-03-27T06:54:26.595 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l[66934]: ignoring --setuser ceph since I am not root 2024-03-27T06:54:26.595 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l[66934]: ignoring --setgroup ceph since I am not root 2024-03-27T06:54:26.595 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mds[66938]: -- 172.21.15.170:0/3313061197 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ff27f5a40 con 0x557ff27c9000 2024-03-27T06:54:26.595 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l[66934]: starting mds.l at 2024-03-27T06:54:26.596 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c[67568]: ignoring --setuser ceph since I am not root 2024-03-27T06:54:26.596 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c[67568]: ignoring --setgroup ceph since I am not root 2024-03-27T06:54:26.596 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mds[67572]: -- 172.21.15.170:0/3664807588 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ebc2a7a40 con 0x558ebc27b000 2024-03-27T06:54:26.596 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c[67568]: starting mds.c at 2024-03-27T06:54:26.596 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[71016]: ignoring --setuser ceph since I am not root 2024-03-27T06:54:26.597 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[71016]: ignoring --setgroup ceph since I am not root 2024-03-27T06:54:26.597 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mds[71020]: -- 172.21.15.170:0/2994050879 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562bc0289a40 con 0x562bc025cc00 2024-03-27T06:54:26.597 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[71016]: starting mds.f at 2024-03-27T06:54:26.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31904]: 2024-03-27T06:54:26.144+0000 7f6c3ea48700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-27T06:54:26.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a[74037]: ignoring --setuser ceph since I am not root 2024-03-27T06:54:26.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a[74037]: ignoring --setgroup ceph since I am not root 2024-03-27T06:54:26.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a[74037]: starting mds.a at 2024-03-27T06:54:26.612 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g[73676]: ignoring --setuser ceph since I am not root 2024-03-27T06:54:26.613 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g[73676]: ignoring --setgroup ceph since I am not root 2024-03-27T06:54:26.613 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-mds[73680]: -- 172.21.15.16:0/2349074847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df909e1a40 con 0x55df909b4c00 2024-03-27T06:54:26.613 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g[73676]: starting mds.g at 2024-03-27T06:54:26.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2867893399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:54:26.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/157498200' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-27T06:54:26.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mon[36261]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-27T06:54:26.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mon[36261]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-27T06:54:26.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mon[36261]: Health check cleared: MDS_DAMAGE (was: 4 MDSs report damaged metadata) 2024-03-27T06:54:26.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/157498200' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-27T06:54:26.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mon[36261]: osdmap e101: 12 total, 12 up, 12 in 2024-03-27T06:54:26.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mon[36261]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-27T06:54:26.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:26 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3093170537' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-27T06:54:27.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2867893399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:54:27.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/157498200' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-27T06:54:27.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mon[36438]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-27T06:54:27.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mon[36438]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-27T06:54:27.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mon[36438]: Health check cleared: MDS_DAMAGE (was: 4 MDSs report damaged metadata) 2024-03-27T06:54:27.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/157498200' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-27T06:54:27.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mon[36438]: osdmap e101: 12 total, 12 up, 12 in 2024-03-27T06:54:27.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mon[36438]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-27T06:54:27.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:26 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3093170537' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-27T06:54:27.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2867893399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-27T06:54:27.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/157498200' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-27T06:54:27.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-mon[31908]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-27T06:54:27.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-mon[31908]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-27T06:54:27.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-mon[31908]: Health check cleared: MDS_DAMAGE (was: 4 MDSs report damaged metadata) 2024-03-27T06:54:27.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/157498200' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-27T06:54:27.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-mon[31908]: osdmap e101: 12 total, 12 up, 12 in 2024-03-27T06:54:27.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-mon[31908]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-27T06:54:27.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:26 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3093170537' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-27T06:54:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.147+0000 7f7d837fe640 1 -- 172.21.15.16:0/3093170537 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7f7d7809caf0 con 0x7f7d841009c0 2024-03-27T06:54:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/3093170537 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d6c0a3780 msgr2=0x7f7d6c0a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d6c0a3780 0x7f7d6c0a5c40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7d740058c0 tx=0x7f7d74005850 comp rx=0 tx=0).stop 2024-03-27T06:54:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/3093170537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d841009c0 msgr2=0x7f7d84102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d841009c0 0x7f7d84102ef0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800db80 tx=0x7f7d7800b630 comp rx=0 tx=0).stop 2024-03-27T06:54:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/3093170537 shutdown_connections 2024-03-27T06:54:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d841069d0 0x7f7d84103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f7d6c0a3780 0x7f7d6c0a5c40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d84101380 0x7f7d84103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 --2- 172.21.15.16:0/3093170537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d841009c0 0x7f7d84102ef0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/3093170537 >> 172.21.15.16:0/3093170537 conn(0x7f7d84077b50 msgr2=0x7f7d84068ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/3093170537 shutdown_connections 2024-03-27T06:54:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.151+0000 7f7d8c6c9640 1 -- 172.21.15.16:0/3093170537 wait complete. 2024-03-27T06:54:27.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-27T06:54:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.281+0000 7f56a6da1640 1 -- 172.21.15.16:0/309554030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a01024a0 msgr2=0x7f56a010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.281+0000 7f56a6da1640 1 --2- 172.21.15.16:0/309554030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a01024a0 0x7f56a010e9b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5690009930 tx=0x7f569002bc30 comp rx=0 tx=0).stop 2024-03-27T06:54:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 -- 172.21.15.16:0/309554030 shutdown_connections 2024-03-27T06:54:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 --2- 172.21.15.16:0/309554030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a01024a0 0x7f56a010e9b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 --2- 172.21.15.16:0/309554030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56a0101ae0 0x7f56a0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 --2- 172.21.15.16:0/309554030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56a0107b80 0x7f56a0107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 -- 172.21.15.16:0/309554030 >> 172.21.15.16:0/309554030 conn(0x7f56a0078f50 msgr2=0x7f56a0079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 -- 172.21.15.16:0/309554030 shutdown_connections 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 -- 172.21.15.16:0/309554030 wait complete. 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 Processor -- start 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 -- start start 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a0101ae0 0x7f56a019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56a01024a0 0x7f56a019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56a0107b80 0x7f56a019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0111fa0 con 0x7f56a0101ae0 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0111e20 con 0x7f56a01024a0 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.282+0000 7f56a6da1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0112120 con 0x7f56a0107b80 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f56a4b3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a0101ae0 0x7f56a019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f56a4b3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a0101ae0 0x7f56a019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50936/0 (socket says 172.21.15.16:50936) 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f56a4b3d640 1 -- 172.21.15.16:0/2188001862 learned_addr learned my addr 172.21.15.16:0/2188001862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f569ffff640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56a01024a0 0x7f56a019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f56a4b3d640 1 -- 172.21.15.16:0/2188001862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56a0107b80 msgr2=0x7f56a019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f56a4b3d640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56a0107b80 0x7f56a019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f56a4b3d640 1 -- 172.21.15.16:0/2188001862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56a01024a0 msgr2=0x7f56a019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f56a4b3d640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56a01024a0 0x7f56a019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f56a4b3d640 1 -- 172.21.15.16:0/2188001862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a019f670 con 0x7f56a0101ae0 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f56a4b3d640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a0101ae0 0x7f56a019a680 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f569800eab0 tx=0x7f569800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f569dffb640 1 -- 172.21.15.16:0/2188001862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569800ce40 con 0x7f56a0101ae0 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f569dffb640 1 -- 172.21.15.16:0/2188001862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5698011970 con 0x7f56a0101ae0 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f56a6da1640 1 -- 172.21.15.16:0/2188001862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a019f960 con 0x7f56a0101ae0 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f569dffb640 1 -- 172.21.15.16:0/2188001862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698010bb0 con 0x7f56a0101ae0 2024-03-27T06:54:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.283+0000 7f56a6da1640 1 -- 172.21.15.16:0/2188001862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a01a71f0 con 0x7f56a0101ae0 2024-03-27T06:54:27.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.284+0000 7f56a6da1640 1 -- 172.21.15.16:0/2188001862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a0068d90 con 0x7f56a0101ae0 2024-03-27T06:54:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.285+0000 7f569dffb640 1 -- 172.21.15.16:0/2188001862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7f5698010d50 con 0x7f56a0101ae0 2024-03-27T06:54:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.286+0000 7f569dffb640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f56880a3780 0x7f56880a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.286+0000 7f569ffff640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f56880a3780 0x7f56880a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.286+0000 7f569dffb640 1 -- 172.21.15.16:0/2188001862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8899+0+0 (secure 0 0 0) 0x7f56980d09a0 con 0x7f56a0101ae0 2024-03-27T06:54:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.286+0000 7f569ffff640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f56880a3780 0x7f56880a5c40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56a0103f00 tx=0x7f569400b590 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.291+0000 7f569dffb640 1 -- 172.21.15.16:0/2188001862 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7f5698016030 con 0x7f56a0101ae0 2024-03-27T06:54:27.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:27.475+0000 7f56a6da1640 1 -- 172.21.15.16:0/2188001862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f56a01a7580 con 0x7f56a0101ae0 2024-03-27T06:54:27.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: pgmap v340: 97 pgs: 97 active+clean; 42 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-27T06:54:27.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-27T06:54:27.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-27T06:54:27.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: Cluster is now healthy 2024-03-27T06:54:27.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/3093170537' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-27T06:54:27.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6833/71393234,v1:172.21.15.156:6836/71393234] up:boot 2024-03-27T06:54:27.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6834/2235544741,v1:172.21.15.170:6835/2235544741] up:boot 2024-03-27T06:54:27.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6834/414811414,v1:172.21.15.156:6837/414811414] up:boot 2024-03-27T06:54:27.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6832/52195311,v1:172.21.15.170:6833/52195311] up:boot 2024-03-27T06:54:27.973 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6838/875927070,v1:172.21.15.156:6839/875927070] up:boot 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.16:6838/3973197963,v1:172.21.15.16:6839/3973197963] up:boot 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6836/3305115262,v1:172.21.15.170:6838/3305115262] up:boot 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.170:6837/28804647,v1:172.21.15.170:6839/28804647] up:boot 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.16:6840/3137422714,v1:172.21.15.16:6841/3137422714] up:boot 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: mds.? [v2:172.21.15.156:6832/3963966564,v1:172.21.15.156:6835/3963966564] up:boot 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: fsmap 12 up:standby 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T06:54:27.974 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T06:54:27.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T06:54:27.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:54:27.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T06:54:27.975 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:27 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2188001862' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-27T06:54:28.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: pgmap v340: 97 pgs: 97 active+clean; 42 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-27T06:54:28.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-27T06:54:28.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-27T06:54:28.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: Cluster is now healthy 2024-03-27T06:54:28.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/3093170537' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-27T06:54:28.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6833/71393234,v1:172.21.15.156:6836/71393234] up:boot 2024-03-27T06:54:28.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6834/2235544741,v1:172.21.15.170:6835/2235544741] up:boot 2024-03-27T06:54:28.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6834/414811414,v1:172.21.15.156:6837/414811414] up:boot 2024-03-27T06:54:28.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6832/52195311,v1:172.21.15.170:6833/52195311] up:boot 2024-03-27T06:54:28.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6838/875927070,v1:172.21.15.156:6839/875927070] up:boot 2024-03-27T06:54:28.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.16:6838/3973197963,v1:172.21.15.16:6839/3973197963] up:boot 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6836/3305115262,v1:172.21.15.170:6838/3305115262] up:boot 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.170:6837/28804647,v1:172.21.15.170:6839/28804647] up:boot 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.16:6840/3137422714,v1:172.21.15.16:6841/3137422714] up:boot 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: mds.? [v2:172.21.15.156:6832/3963966564,v1:172.21.15.156:6835/3963966564] up:boot 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: fsmap 12 up:standby 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:54:28.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T06:54:28.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:27 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2188001862' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-27T06:54:28.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: pgmap v340: 97 pgs: 97 active+clean; 42 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-27T06:54:28.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-27T06:54:28.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-27T06:54:28.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: Cluster is now healthy 2024-03-27T06:54:28.112 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/3093170537' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6833/71393234,v1:172.21.15.156:6836/71393234] up:boot 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6834/2235544741,v1:172.21.15.170:6835/2235544741] up:boot 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6834/414811414,v1:172.21.15.156:6837/414811414] up:boot 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6832/52195311,v1:172.21.15.170:6833/52195311] up:boot 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6838/875927070,v1:172.21.15.156:6839/875927070] up:boot 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.16:6838/3973197963,v1:172.21.15.16:6839/3973197963] up:boot 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6836/3305115262,v1:172.21.15.170:6838/3305115262] up:boot 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.170:6837/28804647,v1:172.21.15.170:6839/28804647] up:boot 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.16:6840/3137422714,v1:172.21.15.16:6841/3137422714] up:boot 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: mds.? [v2:172.21.15.156:6832/3963966564,v1:172.21.15.156:6835/3963966564] up:boot 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: fsmap 12 up:standby 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-27T06:54:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-27T06:54:28.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-27T06:54:28.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-27T06:54:28.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-27T06:54:28.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-27T06:54:28.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-27T06:54:28.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-27T06:54:28.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='mgr.15354 172.21.15.16:0/2722011257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-27T06:54:28.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:27 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2188001862' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-27T06:54:28.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.149+0000 7f569dffb640 1 -- 172.21.15.16:0/2188001862 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7f569809d050 con 0x7f56a0101ae0 2024-03-27T06:54:28.144 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-03-27T06:54:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.152+0000 7f56a6da1640 1 -- 172.21.15.16:0/2188001862 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f56880a3780 msgr2=0x7f56880a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.152+0000 7f56a6da1640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f56880a3780 0x7f56880a5c40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56a0103f00 tx=0x7f569400b590 comp rx=0 tx=0).stop 2024-03-27T06:54:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.152+0000 7f56a6da1640 1 -- 172.21.15.16:0/2188001862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a0101ae0 msgr2=0x7f56a019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.152+0000 7f56a6da1640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a0101ae0 0x7f56a019a680 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f569800eab0 tx=0x7f569800ef70 comp rx=0 tx=0).stop 2024-03-27T06:54:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.152+0000 7f56a6da1640 1 -- 172.21.15.16:0/2188001862 shutdown_connections 2024-03-27T06:54:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.152+0000 7f56a6da1640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7f56880a3780 0x7f56880a5c40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.152+0000 7f56a6da1640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56a0107b80 0x7f56a019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.152+0000 7f56a6da1640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56a01024a0 0x7f56a019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.152+0000 7f56a6da1640 1 --2- 172.21.15.16:0/2188001862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56a0101ae0 0x7f56a019a680 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.152+0000 7f56a6da1640 1 -- 172.21.15.16:0/2188001862 >> 172.21.15.16:0/2188001862 conn(0x7f56a0078f50 msgr2=0x7f56a010d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.152+0000 7f56a6da1640 1 -- 172.21.15.16:0/2188001862 shutdown_connections 2024-03-27T06:54:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.153+0000 7f56a6da1640 1 -- 172.21.15.16:0/2188001862 wait complete. 2024-03-27T06:54:28.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-27T06:54:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.282+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/3289252429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e8100910 msgr2=0x7fd3e8100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.282+0000 7fd3ee4c9640 1 --2- 172.21.15.16:0/3289252429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e8100910 0x7fd3e8100cf0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009900 tx=0x7fd3dc02bc30 comp rx=0 tx=0).stop 2024-03-27T06:54:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.282+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/3289252429 shutdown_connections 2024-03-27T06:54:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.282+0000 7fd3ee4c9640 1 --2- 172.21.15.16:0/3289252429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3e810e4c0 0x7fd3e81108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.282+0000 7fd3ee4c9640 1 --2- 172.21.15.16:0/3289252429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3e8101230 0x7fd3e810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.282+0000 7fd3ee4c9640 1 --2- 172.21.15.16:0/3289252429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e8100910 0x7fd3e8100cf0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.282+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/3289252429 >> 172.21.15.16:0/3289252429 conn(0x7fd3e8076dd0 msgr2=0x7fd3e80771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.282+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/3289252429 shutdown_connections 2024-03-27T06:54:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.282+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/3289252429 wait complete. 2024-03-27T06:54:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.283+0000 7fd3ee4c9640 1 Processor -- start 2024-03-27T06:54:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.283+0000 7fd3ee4c9640 1 -- start start 2024-03-27T06:54:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.283+0000 7fd3ee4c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3e8100910 0x7fd3e8102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.283+0000 7fd3ee4c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3e8101230 0x7fd3e8103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.283+0000 7fd3ee4c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e810e4c0 0x7fd3e8103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.283+0000 7fd3ee4c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8114210 con 0x7fd3e810e4c0 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.283+0000 7fd3ee4c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8114090 con 0x7fd3e8100910 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.283+0000 7fd3ee4c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8114390 con 0x7fd3e8101230 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.283+0000 7fd3eca66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e810e4c0 0x7fd3e8103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.283+0000 7fd3eca66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e810e4c0 0x7fd3e8103670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50948/0 (socket says 172.21.15.16:50948) 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.283+0000 7fd3eca66640 1 -- 172.21.15.16:0/704539552 learned_addr learned my addr 172.21.15.16:0/704539552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3e7fff640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3e8100910 0x7fd3e8102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3e77fe640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3e8101230 0x7fd3e8103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3eca66640 1 -- 172.21.15.16:0/704539552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3e8101230 msgr2=0x7fd3e8103130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3eca66640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3e8101230 0x7fd3e8103130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3eca66640 1 -- 172.21.15.16:0/704539552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3e8100910 msgr2=0x7fd3e8102bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3eca66640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3e8100910 0x7fd3e8102bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3eca66640 1 -- 172.21.15.16:0/704539552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e81aad80 con 0x7fd3e810e4c0 2024-03-27T06:54:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3eca66640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e810e4c0 0x7fd3e8103670 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000cae0 tx=0x7fd3e0007240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3e57fa640 1 -- 172.21.15.16:0/704539552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0007e20 con 0x7fd3e810e4c0 2024-03-27T06:54:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3e57fa640 1 -- 172.21.15.16:0/704539552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd3e0004b20 con 0x7fd3e810e4c0 2024-03-27T06:54:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/704539552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e81ab070 con 0x7fd3e810e4c0 2024-03-27T06:54:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3e57fa640 1 -- 172.21.15.16:0/704539552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000f740 con 0x7fd3e810e4c0 2024-03-27T06:54:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.284+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/704539552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e81ab5c0 con 0x7fd3e810e4c0 2024-03-27T06:54:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.285+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/704539552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e806a940 con 0x7fd3e810e4c0 2024-03-27T06:54:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.286+0000 7fd3e57fa640 1 -- 172.21.15.16:0/704539552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138501+0+0 (secure 0 0 0) 0x7fd3e0004cd0 con 0x7fd3e810e4c0 2024-03-27T06:54:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.287+0000 7fd3e57fa640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd3d00a3780 0x7fd3d00a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-27T06:54:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.287+0000 7fd3e7fff640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd3d00a3780 0x7fd3d00a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-27T06:54:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.287+0000 7fd3e57fa640 1 -- 172.21.15.16:0/704539552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8345+0+0 (secure 0 0 0) 0x7fd3e00cffd0 con 0x7fd3e810e4c0 2024-03-27T06:54:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.287+0000 7fd3e7fff640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd3d00a3780 0x7fd3d00a5c40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc000c00 tx=0x7fd3dc02c890 comp rx=0 tx=0).ready entity=mgr.15354 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-27T06:54:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.291+0000 7fd3e57fa640 1 -- 172.21.15.16:0/704539552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187541 (secure 0 0 0) 0x7fd3e0014030 con 0x7fd3e810e4c0 2024-03-27T06:54:28.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:28.490+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/704539552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd3e802d370 con 0x7fd3e810e4c0 2024-03-27T06:54:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.155+0000 7fd3e57fa640 1 -- 172.21.15.16:0/704539552 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7fd3e009c910 con 0x7fd3e810e4c0 2024-03-27T06:54:29.150 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-03-27T06:54:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.159+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/704539552 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd3d00a3780 msgr2=0x7fd3d00a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.159+0000 7fd3ee4c9640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd3d00a3780 0x7fd3d00a5c40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc000c00 tx=0x7fd3dc02c890 comp rx=0 tx=0).stop 2024-03-27T06:54:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.159+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/704539552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e810e4c0 msgr2=0x7fd3e8103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-27T06:54:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.159+0000 7fd3ee4c9640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e810e4c0 0x7fd3e8103670 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000cae0 tx=0x7fd3e0007240 comp rx=0 tx=0).stop 2024-03-27T06:54:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.159+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/704539552 shutdown_connections 2024-03-27T06:54:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.159+0000 7fd3ee4c9640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e810e4c0 0x7fd3e8103670 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.159+0000 7fd3ee4c9640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.16:6800/3715976676,v1:172.21.15.16:6801/3715976676] conn(0x7fd3d00a3780 0x7fd3d00a5c40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.159+0000 7fd3ee4c9640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3e8101230 0x7fd3e8103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.159+0000 7fd3ee4c9640 1 --2- 172.21.15.16:0/704539552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3e8100910 0x7fd3e8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-27T06:54:29.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.159+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/704539552 >> 172.21.15.16:0/704539552 conn(0x7fd3e8076dd0 msgr2=0x7fd3e810fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-27T06:54:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.160+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/704539552 shutdown_connections 2024-03-27T06:54:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-27T06:54:29.160+0000 7fd3ee4c9640 1 -- 172.21.15.16:0/704539552 wait complete. 2024-03-27T06:54:29.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T06:54:29.207 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T06:54:29.241 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T06:54:29.275 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-27T06:54:29.276 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-27T06:54:29.291 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-27T06:54:29.310 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-27T06:54:29.332 INFO:tasks.cephadm:Stopping all daemons... 2024-03-27T06:54:29.332 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-27T06:54:29.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.a 2024-03-27T06:54:29.472 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:29 smithi156 ceph-mon[36261]: pgmap v342: 97 pgs: 97 active+clean; 42 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-27T06:54:29.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:29 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/2188001862' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-27T06:54:29.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:29 smithi156 ceph-mon[36261]: osdmap e102: 12 total, 12 up, 12 in 2024-03-27T06:54:29.473 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:29 smithi156 ceph-mon[36261]: from='client.? 172.21.15.16:0/704539552' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-27T06:54:29.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:29 smithi170 ceph-mon[36438]: pgmap v342: 97 pgs: 97 active+clean; 42 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-27T06:54:29.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:29 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/2188001862' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-27T06:54:29.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:29 smithi170 ceph-mon[36438]: osdmap e102: 12 total, 12 up, 12 in 2024-03-27T06:54:29.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:29 smithi170 ceph-mon[36438]: from='client.? 172.21.15.16:0/704539552' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-27T06:54:29.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:29 smithi016 ceph-mon[31908]: pgmap v342: 97 pgs: 97 active+clean; 42 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-27T06:54:29.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:29 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/2188001862' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-27T06:54:29.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:29 smithi016 ceph-mon[31908]: osdmap e102: 12 total, 12 up, 12 in 2024-03-27T06:54:29.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:29 smithi016 ceph-mon[31908]: from='client.? 172.21.15.16:0/704539552' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-27T06:54:29.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:29 smithi016 systemd[1]: Stopping Ceph mon.a for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:54:30.055 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:29 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31904]: 2024-03-27T06:54:29.675+0000 7f6c44253700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:54:30.055 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:29 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a[31904]: 2024-03-27T06:54:29.675+0000 7f6c44253700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-27T06:54:30.055 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:29 smithi016 podman[93990]: 2024-03-27 06:54:29.758033601 +0000 UTC m=+0.208818523 container died 06aedba2800884d1ea2407fd87a86eadf06a7409b67b48ac2dffa27527891498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-27T06:54:30.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:30 smithi016 podman[93990]: 2024-03-27 06:54:30.160042281 +0000 UTC m=+0.610827200 container cleanup 06aedba2800884d1ea2407fd87a86eadf06a7409b67b48ac2dffa27527891498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux ) 2024-03-27T06:54:30.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Mar 27 06:54:30 smithi016 bash[93990]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-a 2024-03-27T06:54:30.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.a.service' 2024-03-27T06:54:30.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:54:30.624 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-27T06:54:30.624 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-27T06:54:30.624 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.b 2024-03-27T06:54:30.972 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:30 smithi156 systemd[1]: Stopping Ceph mon.b for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:54:31.436 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-b[36256]: 2024-03-27T06:54:31.005+0000 7fa9aaf9b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:54:31.436 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-b[36256]: 2024-03-27T06:54:31.005+0000 7fa9aaf9b700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-27T06:54:31.436 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:31 smithi156 podman[72146]: 2024-03-27 06:54:31.128310376 +0000 UTC m=+0.306213654 container died 1e165e1b9c54f18f360afe3b46bd4398d8741e1c703ae9a7a3979a25f7a74959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:54:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:31 smithi156 podman[72146]: 2024-03-27 06:54:31.574229157 +0000 UTC m=+0.752132430 container cleanup 1e165e1b9c54f18f360afe3b46bd4398d8741e1c703ae9a7a3979a25f7a74959 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-b, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:54:31.722 INFO:journalctl@ceph.mon.b.smithi156.stdout:Mar 27 06:54:31 smithi156 bash[72146]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-b 2024-03-27T06:54:32.037 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.b.service' 2024-03-27T06:54:32.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:54:32.088 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-27T06:54:32.088 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-27T06:54:32.089 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.c 2024-03-27T06:54:32.402 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:32 smithi170 systemd[1]: Stopping Ceph mon.c for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:54:32.829 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-c[36421]: 2024-03-27T06:54:32.407+0000 7f7b02010700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:54:32.829 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-c[36421]: 2024-03-27T06:54:32.407+0000 7f7b02010700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-27T06:54:32.829 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:32 smithi170 podman[73541]: 2024-03-27 06:54:32.550750158 +0000 UTC m=+0.256518861 container died 610ce06acfe8a5f4e7a8a91f8e1c2571e0c120b4c3052b78ac6c73611c1af404 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True) 2024-03-27T06:54:33.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:32 smithi170 podman[73541]: 2024-03-27 06:54:32.933298496 +0000 UTC m=+0.639067205 container cleanup 610ce06acfe8a5f4e7a8a91f8e1c2571e0c120b4c3052b78ac6c73611c1af404 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-c, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-27T06:54:33.095 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 27 06:54:32 smithi170 bash[73541]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mon-c 2024-03-27T06:54:33.348 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mon.c.service' 2024-03-27T06:54:33.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:54:33.399 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-27T06:54:33.399 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-27T06:54:33.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.x 2024-03-27T06:54:33.742 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:54:33 smithi016 systemd[1]: Stopping Ceph mgr.x for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:54:34.068 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:54:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:54:33.746+0000 7f70fa83b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:54:34.068 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:54:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x[32142]: 2024-03-27T06:54:33.746+0000 7f70fa83b700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-27T06:54:34.069 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:54:33 smithi016 podman[94113]: 2024-03-27 06:54:33.778573577 +0000 UTC m=+0.174478748 container died 9699eb46f80f04de2b3a020d648c9e9520ead1845aee4b07bdab4b953ae58a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=) 2024-03-27T06:54:34.362 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:54:34 smithi016 podman[94113]: 2024-03-27 06:54:34.172691972 +0000 UTC m=+0.568597158 container cleanup 9699eb46f80f04de2b3a020d648c9e9520ead1845aee4b07bdab4b953ae58a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:155268c4e432a12433aa833f174f9fe3b1016ae0, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True) 2024-03-27T06:54:34.362 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Mar 27 06:54:34 smithi016 bash[94113]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-x 2024-03-27T06:54:34.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.x.service' 2024-03-27T06:54:34.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:54:34.618 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-27T06:54:34.618 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-27T06:54:34.618 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.y 2024-03-27T06:54:34.944 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:54:34 smithi156 systemd[1]: Stopping Ceph mgr.y for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:54:35.222 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:54:34 smithi156 podman[72268]: 2024-03-27 06:54:34.949377897 +0000 UTC m=+0.130820419 container died 1df63da56546fd9e9c9602442eb08cee2d8a55004ee851151ce498a56bf93236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-27T06:54:35.630 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:54:35 smithi156 podman[72268]: 2024-03-27 06:54:35.304260261 +0000 UTC m=+0.485702775 container cleanup 1df63da56546fd9e9c9602442eb08cee2d8a55004ee851151ce498a56bf93236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-27T06:54:35.631 INFO:journalctl@ceph.mgr.y.smithi156.stdout:Mar 27 06:54:35 smithi156 bash[72268]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-y 2024-03-27T06:54:35.771 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.y.service' 2024-03-27T06:54:35.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:54:35.808 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-27T06:54:35.808 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-27T06:54:35.808 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.z 2024-03-27T06:54:36.095 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:54:35 smithi170 systemd[1]: Stopping Ceph mgr.z for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:54:36.502 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:54:36 smithi170 podman[73664]: 2024-03-27 06:54:36.176976474 +0000 UTC m=+0.168925330 container died 712531754fddc89c071e10a2d79afc25a9ab8e4c466446190da2bbeec26c7621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:54:36.837 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:54:36 smithi170 podman[73664]: 2024-03-27 06:54:36.60736798 +0000 UTC m=+0.599316835 container cleanup 712531754fddc89c071e10a2d79afc25a9ab8e4c466446190da2bbeec26c7621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-27T06:54:36.837 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:54:36 smithi170 bash[73664]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z 2024-03-27T06:54:37.083 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.z.service' 2024-03-27T06:54:37.094 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:54:36 smithi170 podman[73679]: 2024-03-27 06:54:36.842421622 +0000 UTC m=+0.658158729 container remove 712531754fddc89c071e10a2d79afc25a9ab8e4c466446190da2bbeec26c7621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mgr-z, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5) 2024-03-27T06:54:37.094 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:54:36 smithi170 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-27T06:54:37.094 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:54:37 smithi170 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.z.service: Failed with result 'exit-code'. 2024-03-27T06:54:37.094 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:54:37 smithi170 systemd[1]: Stopped Ceph mgr.z for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:54:37.094 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 27 06:54:37 smithi170 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mgr.z.service: Consumed 19.731s CPU time. 2024-03-27T06:54:37.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:54:37.399 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-27T06:54:37.399 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-27T06:54:37.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.0 2024-03-27T06:54:37.697 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:37 smithi016 systemd[1]: Stopping Ceph osd.0 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:54:38.112 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:37 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0[47018]: 2024-03-27T06:54:37.700+0000 7f05da6ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:54:38.112 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:37 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0[47018]: 2024-03-27T06:54:37.700+0000 7f05da6ef700 -1 osd.0 103 *** Got signal Terminated *** 2024-03-27T06:54:38.112 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:37 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0[47018]: 2024-03-27T06:54:37.700+0000 7f05da6ef700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:54:43.362 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:43 smithi016 podman[94237]: 2024-03-27 06:54:43.109530935 +0000 UTC m=+5.510679137 container died 3b50ffebef817db992b9227a2427b0e343fef2ef5c62651b5ce15ffd9773b82c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-27T06:54:43.781 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:43 smithi016 podman[94237]: 2024-03-27 06:54:43.485660445 +0000 UTC m=+5.886808648 container cleanup 3b50ffebef817db992b9227a2427b0e343fef2ef5c62651b5ce15ffd9773b82c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-27T06:54:43.781 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:43 smithi016 bash[94237]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0 2024-03-27T06:54:44.107 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:43 smithi016 podman[94253]: 2024-03-27 06:54:43.787059635 +0000 UTC m=+0.674901062 container remove 3b50ffebef817db992b9227a2427b0e343fef2ef5c62651b5ce15ffd9773b82c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=) 2024-03-27T06:54:44.362 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:44 smithi016 podman[94346]: 2024-03-27 06:54:44.015737206 +0000 UTC m=+0.018125646 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:54:44.362 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:44 smithi016 podman[94346]: 2024-03-27 06:54:44.171545344 +0000 UTC m=+0.173933826 container create 1340ca10f7030615e55ce6395c4028379d9f5077d752e2db34beb3e44f12826f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T06:54:44.723 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:44 smithi016 podman[94346]: 2024-03-27 06:54:44.432586762 +0000 UTC m=+0.434975202 container init 1340ca10f7030615e55ce6395c4028379d9f5077d752e2db34beb3e44f12826f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T06:54:44.724 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:44 smithi016 podman[94346]: 2024-03-27 06:54:44.435954724 +0000 UTC m=+0.438343173 container start 1340ca10f7030615e55ce6395c4028379d9f5077d752e2db34beb3e44f12826f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-27T06:54:44.724 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:44 smithi016 podman[94346]: 2024-03-27 06:54:44.524367914 +0000 UTC m=+0.526756374 container attach 1340ca10f7030615e55ce6395c4028379d9f5077d752e2db34beb3e44f12826f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:54:45.021 INFO:journalctl@ceph.osd.0.smithi016.stdout:Mar 27 06:54:44 smithi016 podman[94346]: 2024-03-27 06:54:44.728913211 +0000 UTC m=+0.731301653 container died 1340ca10f7030615e55ce6395c4028379d9f5077d752e2db34beb3e44f12826f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux ) 2024-03-27T06:54:45.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.0.service' 2024-03-27T06:54:45.382 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:54:45.382 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-27T06:54:45.382 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-27T06:54:45.382 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.1 2024-03-27T06:54:45.695 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:45 smithi156 systemd[1]: Stopping Ceph osd.1 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:54:45.972 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:45 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1[40822]: 2024-03-27T06:54:45.698+0000 7f7498d7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:54:45.972 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:45 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1[40822]: 2024-03-27T06:54:45.698+0000 7f7498d7f700 -1 osd.1 103 *** Got signal Terminated *** 2024-03-27T06:54:45.973 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:45 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1[40822]: 2024-03-27T06:54:45.698+0000 7f7498d7f700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:54:51.562 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:51 smithi156 podman[72392]: 2024-03-27 06:54:51.229583382 +0000 UTC m=+5.647198856 container died 5c7ca1add42ed477044682e31ce796065bda07447db61875d235e8f8cff84174 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True) 2024-03-27T06:54:51.562 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:51 smithi156 podman[72392]: 2024-03-27 06:54:51.567050288 +0000 UTC m=+5.984665754 container cleanup 5c7ca1add42ed477044682e31ce796065bda07447db61875d235e8f8cff84174 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-27T06:54:51.972 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:51 smithi156 bash[72392]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1 2024-03-27T06:54:51.973 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:51 smithi156 podman[72409]: 2024-03-27 06:54:51.77689891 +0000 UTC m=+0.547604225 container remove 5c7ca1add42ed477044682e31ce796065bda07447db61875d235e8f8cff84174 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:54:52.473 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:52 smithi156 podman[72502]: 2024-03-27 06:54:52.078240594 +0000 UTC m=+0.017714910 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:54:52.473 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:52 smithi156 podman[72502]: 2024-03-27 06:54:52.18607142 +0000 UTC m=+0.125545715 container create 431266e18457534c32cc724469f6fc37433cc78a3a6f23db534aecbb769f1d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-27T06:54:52.807 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:52 smithi156 podman[72502]: 2024-03-27 06:54:52.52208246 +0000 UTC m=+0.461556771 container init 431266e18457534c32cc724469f6fc37433cc78a3a6f23db534aecbb769f1d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:54:52.807 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:52 smithi156 podman[72502]: 2024-03-27 06:54:52.525411973 +0000 UTC m=+0.464886276 container start 431266e18457534c32cc724469f6fc37433cc78a3a6f23db534aecbb769f1d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-27T06:54:52.808 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:52 smithi156 podman[72502]: 2024-03-27 06:54:52.663929684 +0000 UTC m=+0.603403994 container attach 431266e18457534c32cc724469f6fc37433cc78a3a6f23db534aecbb769f1d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1-deactivate, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-27T06:54:53.170 INFO:journalctl@ceph.osd.1.smithi156.stdout:Mar 27 06:54:52 smithi156 podman[72502]: 2024-03-27 06:54:52.813214815 +0000 UTC m=+0.752689117 container died 431266e18457534c32cc724469f6fc37433cc78a3a6f23db534aecbb769f1d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, ceph=True) 2024-03-27T06:54:53.503 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.1.service' 2024-03-27T06:54:53.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:54:53.546 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-27T06:54:53.546 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-27T06:54:53.546 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.2 2024-03-27T06:54:53.844 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:54:53 smithi170 systemd[1]: Stopping Ceph osd.2 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:54:54.344 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:54:53 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2[41564]: 2024-03-27T06:54:53.869+0000 7fbe1db9a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:54:54.344 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:54:53 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2[41564]: 2024-03-27T06:54:53.869+0000 7fbe1db9a700 -1 osd.2 103 *** Got signal Terminated *** 2024-03-27T06:54:54.344 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:54:53 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2[41564]: 2024-03-27T06:54:53.869+0000 7fbe1db9a700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:54:59.500 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:54:59 smithi170 podman[73797]: 2024-03-27 06:54:59.187116374 +0000 UTC m=+5.441550421 container died 9cb38c41d9662bd17f8a50d9bd897ca501630aa512e64f0ac5cb5a16b19d55da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS) 2024-03-27T06:54:59.842 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:54:59 smithi170 podman[73797]: 2024-03-27 06:54:59.595198168 +0000 UTC m=+5.849632211 container cleanup 9cb38c41d9662bd17f8a50d9bd897ca501630aa512e64f0ac5cb5a16b19d55da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:54:59.842 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:54:59 smithi170 bash[73797]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2 2024-03-27T06:55:00.094 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:54:59 smithi170 podman[73814]: 2024-03-27 06:54:59.846632993 +0000 UTC m=+0.654592998 container remove 9cb38c41d9662bd17f8a50d9bd897ca501630aa512e64f0ac5cb5a16b19d55da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-27T06:55:00.412 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:55:00 smithi170 podman[73906]: 2024-03-27 06:55:00.058325994 +0000 UTC m=+0.017558741 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:55:00.412 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:55:00 smithi170 podman[73906]: 2024-03-27 06:55:00.172480042 +0000 UTC m=+0.131712781 container create 6d82132b1a94d7da8d98059a58b9d519fd911bc58d6ede6cb37f197889ea57cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-03-27T06:55:00.706 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:55:00 smithi170 podman[73906]: 2024-03-27 06:55:00.417210127 +0000 UTC m=+0.376442880 container init 6d82132b1a94d7da8d98059a58b9d519fd911bc58d6ede6cb37f197889ea57cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-03-27T06:55:00.706 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:55:00 smithi170 podman[73906]: 2024-03-27 06:55:00.420583034 +0000 UTC m=+0.379815776 container start 6d82132b1a94d7da8d98059a58b9d519fd911bc58d6ede6cb37f197889ea57cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-27T06:55:00.706 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:55:00 smithi170 podman[73906]: 2024-03-27 06:55:00.508764018 +0000 UTC m=+0.467996767 container attach 6d82132b1a94d7da8d98059a58b9d519fd911bc58d6ede6cb37f197889ea57cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, io.buildah.version=1.33.5) 2024-03-27T06:55:01.094 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 27 06:55:00 smithi170 podman[73906]: 2024-03-27 06:55:00.711903463 +0000 UTC m=+0.671136200 container died 6d82132b1a94d7da8d98059a58b9d519fd911bc58d6ede6cb37f197889ea57cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:55:01.445 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.2.service' 2024-03-27T06:55:01.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:55:01.490 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-27T06:55:01.490 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-27T06:55:01.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.3 2024-03-27T06:55:01.817 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:01 smithi016 systemd[1]: Stopping Ceph osd.3 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:55:02.095 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:01 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3[53978]: 2024-03-27T06:55:01.821+0000 7f9c3dc07700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:55:02.096 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:01 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3[53978]: 2024-03-27T06:55:01.821+0000 7f9c3dc07700 -1 osd.3 103 *** Got signal Terminated *** 2024-03-27T06:55:02.096 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:01 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3[53978]: 2024-03-27T06:55:01.821+0000 7f9c3dc07700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:55:03.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:03 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:03.650+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:04.112 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:03 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3[53978]: 2024-03-27T06:55:03.797+0000 7f9c361f3700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:37.950107+0000 front 2024-03-27T06:54:37.950142+0000 (oldest deadline 2024-03-27T06:55:03.250476+0000) 2024-03-27T06:55:04.612 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:04 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:04.324+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:04.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:04 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:04.182+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:04.972 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:04 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:04.603+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:05.112 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:04 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3[53978]: 2024-03-27T06:55:04.795+0000 7f9c361f3700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:37.950107+0000 front 2024-03-27T06:54:37.950142+0000 (oldest deadline 2024-03-27T06:55:03.250476+0000) 2024-03-27T06:55:05.612 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:05 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:05.307+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:05.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:05 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:05.178+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:05.798 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:05 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:05.389+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:05.874 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:05 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:05.589+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:06.112 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:05 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3[53978]: 2024-03-27T06:55:05.843+0000 7f9c361f3700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:37.950107+0000 front 2024-03-27T06:54:37.950142+0000 (oldest deadline 2024-03-27T06:55:03.250476+0000) 2024-03-27T06:55:06.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:06 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:06.096+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:06.612 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:06 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:06.346+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:06.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:06 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:06.209+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:06.808 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:06 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:06.382+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:06.972 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:06 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:06.601+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:07.094 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:07 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:07.051+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:07.609 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:07 smithi016 podman[94451]: 2024-03-27 06:55:07.193177525 +0000 UTC m=+5.494070224 container died 908f355be05b0a798f9e913a1c9cc411431816a0359e4bdfe41f29966bd326f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:55:07.609 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:07 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:07.387+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:07.610 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:07 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:07.230+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:07.844 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:07 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:07.377+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:07.862 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:07 smithi016 podman[94451]: 2024-03-27 06:55:07.705512237 +0000 UTC m=+6.006404925 container cleanup 908f355be05b0a798f9e913a1c9cc411431816a0359e4bdfe41f29966bd326f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T06:55:07.862 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:07 smithi016 bash[94451]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3 2024-03-27T06:55:07.972 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:07 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:07.592+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:08.257 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:07 smithi016 podman[94480]: 2024-03-27 06:55:07.948741245 +0000 UTC m=+0.755932032 container remove 908f355be05b0a798f9e913a1c9cc411431816a0359e4bdfe41f29966bd326f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True) 2024-03-27T06:55:08.335 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:08 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:08.064+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:08.508 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:08 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:08.262+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:08.508 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:08 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:08.364+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:08.508 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:08 smithi016 podman[94572]: 2024-03-27 06:55:08.262315569 +0000 UTC m=+0.017764549 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:55:08.509 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:08 smithi016 podman[94572]: 2024-03-27 06:55:08.383102865 +0000 UTC m=+0.138551861 container create c74f52cd70ad5dad3697856d16ab609b7642dfee65c1618fcdd73d36d5e6f4e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:55:08.594 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:08 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:08.340+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:08.862 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:08 smithi016 podman[94572]: 2024-03-27 06:55:08.644232265 +0000 UTC m=+0.399681220 container init c74f52cd70ad5dad3697856d16ab609b7642dfee65c1618fcdd73d36d5e6f4e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:55:08.862 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:08 smithi016 podman[94572]: 2024-03-27 06:55:08.6473086 +0000 UTC m=+0.402757555 container start c74f52cd70ad5dad3697856d16ab609b7642dfee65c1618fcdd73d36d5e6f4e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:55:08.862 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:08 smithi016 podman[94572]: 2024-03-27 06:55:08.744297366 +0000 UTC m=+0.499746331 container attach c74f52cd70ad5dad3697856d16ab609b7642dfee65c1618fcdd73d36d5e6f4e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-27T06:55:08.972 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:08 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:08.592+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:09.274 INFO:journalctl@ceph.osd.3.smithi016.stdout:Mar 27 06:55:08 smithi016 podman[94572]: 2024-03-27 06:55:08.938469663 +0000 UTC m=+0.693918617 container died c74f52cd70ad5dad3697856d16ab609b7642dfee65c1618fcdd73d36d5e6f4e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-27T06:55:09.291 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:09 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:09.038+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:09.595 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:09 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:09.296+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:09.601 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:09 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:09.371+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:09.601 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:09 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:09.279+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:09.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.3.service' 2024-03-27T06:55:09.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:55:09.675 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-27T06:55:09.675 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-27T06:55:09.675 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.4 2024-03-27T06:55:09.973 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:09 smithi156 systemd[1]: Stopping Ceph osd.4 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:55:09.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:09 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:09.576+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:10.344 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:10 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:10.316+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:10.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:10 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:10.070+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:10.472 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:10 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4[47353]: 2024-03-27T06:55:10.003+0000 7efd6ef27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:55:10.473 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:10 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4[47353]: 2024-03-27T06:55:10.003+0000 7efd6ef27700 -1 osd.4 103 *** Got signal Terminated *** 2024-03-27T06:55:10.473 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:10 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4[47353]: 2024-03-27T06:55:10.003+0000 7efd6ef27700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:55:10.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:10 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:10.322+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:10.612 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:10 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:10.407+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:10.972 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:10 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:10.623+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:11.594 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:11 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:11.278+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:11.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:11 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:11.112+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:11.612 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:11 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:11.378+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:11.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:11 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:11.366+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:11.972 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:11 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:11.661+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:12.289 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:12 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:12.137+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:12.290 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:12 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:12.137+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:12.290 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:11 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:11.954+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:12.594 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:12 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:12.294+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:12.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:12 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:12.393+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:12.612 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:12 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:12.356+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:12.973 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:12 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:12.561+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:12.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:12 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:12.674+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:12.974 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:12 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:12.674+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:13.294 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:12 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:12.964+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:13.295 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:13 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:13.155+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:13.295 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:13 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:13.155+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:13.383 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:13 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:13.373+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:13.383 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:13 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:13.373+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:13.594 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:13 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:13.298+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:13.595 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:13 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:13.298+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.429610+0000 front 2024-03-27T06:54:48.429582+0000 (oldest deadline 2024-03-27T06:55:12.529624+0000) 2024-03-27T06:55:13.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:13 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:13.387+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:13.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:13 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:13.387+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:13.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:13 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:13.583+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:13.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:13 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:13.653+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:13.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:13 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:13.653+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:14.248 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:14 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:14.002+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:14.248 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:14 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:14.141+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:14.248 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:14 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:14.141+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:14.594 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:14 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:14.253+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:14.594 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:14 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:14.253+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.429610+0000 front 2024-03-27T06:54:48.429582+0000 (oldest deadline 2024-03-27T06:55:12.529624+0000) 2024-03-27T06:55:14.627 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:14 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4[47353]: 2024-03-27T06:55:14.311+0000 7efd67513700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:49.458604+0000 front 2024-03-27T06:54:49.458632+0000 (oldest deadline 2024-03-27T06:55:14.158869+0000) 2024-03-27T06:55:14.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:14.419+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:14.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:14.419+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:14.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:14.394+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:14.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:14 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:14.394+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:14.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:14 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:14.631+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:14.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:14 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:14.671+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:14.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:14 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:14.671+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:15.344 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:15 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:15.018+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:15.344 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:15 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:15.232+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:15.345 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:15 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:15.232+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.429610+0000 front 2024-03-27T06:54:48.429582+0000 (oldest deadline 2024-03-27T06:55:12.529624+0000) 2024-03-27T06:55:15.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:15 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:15.160+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:15.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:15 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:15.160+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:15.657 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:15 smithi156 podman[72632]: 2024-03-27 06:55:15.355135478 +0000 UTC m=+5.479971536 container died 70d98398ff6f6bbb7e9f66a8e3fbeef4be39908ef98db6cfa6e57a809b2da257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-27T06:55:15.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:15 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:15.438+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:15.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:15 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:15.438+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:15.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:15 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:15.419+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:15.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:15 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:15.419+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:15.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:15 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:15.674+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:15.973 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:15 smithi156 podman[72632]: 2024-03-27 06:55:15.770627231 +0000 UTC m=+5.895463277 container cleanup 70d98398ff6f6bbb7e9f66a8e3fbeef4be39908ef98db6cfa6e57a809b2da257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=) 2024-03-27T06:55:15.973 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:15 smithi156 bash[72632]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4 2024-03-27T06:55:15.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:15 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:15.699+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:15.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:15 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:15.699+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:16.344 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:16 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:16.040+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:16.345 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:16 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:16.229+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:16.345 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:16 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:16.229+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.429610+0000 front 2024-03-27T06:54:48.429582+0000 (oldest deadline 2024-03-27T06:55:12.529624+0000) 2024-03-27T06:55:16.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:16 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:16.207+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:16.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:16 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:16.207+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:16.408 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:16 smithi156 podman[72649]: 2024-03-27 06:55:16.00527565 +0000 UTC m=+0.649240232 container remove 70d98398ff6f6bbb7e9f66a8e3fbeef4be39908ef98db6cfa6e57a809b2da257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-03-27T06:55:16.678 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:16 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:16.647+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:16.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:16 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:16.650+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:16.678 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:16 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:16.650+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:16.678 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:16 smithi156 podman[72745]: 2024-03-27 06:55:16.317409499 +0000 UTC m=+0.017388900 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:55:16.679 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:16 smithi156 podman[72745]: 2024-03-27 06:55:16.422635468 +0000 UTC m=+0.122614844 container create d3ac68a0ef35be46574725a14ad7b26f747f41acdbb695bb649b45eadfabec94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:55:16.679 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:16 smithi156 podman[72745]: 2024-03-27 06:55:16.683510528 +0000 UTC m=+0.383489944 container init d3ac68a0ef35be46574725a14ad7b26f747f41acdbb695bb649b45eadfabec94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, ceph=True) 2024-03-27T06:55:16.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:16 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:16.455+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:16.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:16 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:16.455+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:16.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:16 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:16.410+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:16.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:16 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:16.410+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:16.970 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:16 smithi156 podman[72745]: 2024-03-27 06:55:16.686830879 +0000 UTC m=+0.386810266 container start d3ac68a0ef35be46574725a14ad7b26f747f41acdbb695bb649b45eadfabec94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-27T06:55:16.970 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:16 smithi156 podman[72745]: 2024-03-27 06:55:16.758430486 +0000 UTC m=+0.458409902 container attach d3ac68a0ef35be46574725a14ad7b26f747f41acdbb695bb649b45eadfabec94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-03-27T06:55:17.188 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:17 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:17.063+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:17.223 INFO:journalctl@ceph.osd.4.smithi156.stdout:Mar 27 06:55:16 smithi156 podman[72745]: 2024-03-27 06:55:16.975811097 +0000 UTC m=+0.675790490 container died d3ac68a0ef35be46574725a14ad7b26f747f41acdbb695bb649b45eadfabec94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-03-27T06:55:17.594 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:17 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:17.193+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:17.595 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:17 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:17.193+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.429610+0000 front 2024-03-27T06:54:48.429582+0000 (oldest deadline 2024-03-27T06:55:12.529624+0000) 2024-03-27T06:55:17.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:17 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:17.234+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:17.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:17 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:17.234+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:17.619 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.4.service' 2024-03-27T06:55:17.660 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:55:17.660 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-27T06:55:17.661 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-27T06:55:17.661 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.5 2024-03-27T06:55:17.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:17 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:17.487+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:17.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:17 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:17.487+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:17.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:17 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:17.453+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:17.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:17 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:17.453+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:17.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:17 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:17.669+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:17.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:17 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:17.664+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:17.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:17 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:17.664+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:17.994 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:17 smithi170 systemd[1]: Stopping Ceph osd.5 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:55:18.254 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:18 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:18.108+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:18.254 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:17 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:17.998+0000 7f7eab639700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:55:18.254 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:17 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:17.998+0000 7f7eab639700 -1 osd.5 103 *** Got signal Terminated *** 2024-03-27T06:55:18.254 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:17 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:17.998+0000 7f7eab639700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:55:18.254 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:18 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:18.201+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:18.254 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:18 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:18.201+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.429610+0000 front 2024-03-27T06:54:48.429582+0000 (oldest deadline 2024-03-27T06:55:12.529624+0000) 2024-03-27T06:55:18.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:18 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:18.258+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:18.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:18 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:18.258+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:18.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:18 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:18.479+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:18.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:18 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:18.479+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:18.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:18 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:18.487+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:18.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:18 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:18.487+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:18.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:18 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:18.648+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:18.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:18 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:18.710+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:18.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:18 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:18.710+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:19.344 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:19 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:19.074+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:19.345 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:19 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:19.188+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:19.345 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:19 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:19.188+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.429610+0000 front 2024-03-27T06:54:48.429582+0000 (oldest deadline 2024-03-27T06:55:12.529624+0000) 2024-03-27T06:55:19.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:19 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:19.273+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:19.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:19 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:19.273+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:19.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:19 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:19.441+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:19.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:19 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:19.441+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:19.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:19 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:19.451+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:19.863 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:19 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:19.451+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:19.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:19 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:19.611+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:19.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:19 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:19.747+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:19.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:19 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:19.747+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:20.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:20 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:20.111+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:20.595 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:20 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:20.178+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:20.595 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:20 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:20.178+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.429610+0000 front 2024-03-27T06:54:48.429582+0000 (oldest deadline 2024-03-27T06:55:12.529624+0000) 2024-03-27T06:55:20.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:20 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:20.276+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:20.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:20 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:20.276+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:20.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:20 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:20.465+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:20.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:20 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:20.465+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:20.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:20 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:20.451+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:20.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:20 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:20.451+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:20.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:20 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:20.451+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.833911+0000 front 2024-03-27T06:54:54.833945+0000 (oldest deadline 2024-03-27T06:55:20.134097+0000) 2024-03-27T06:55:20.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:20.610+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:20.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:20.771+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:20.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:20 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:20.771+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:21.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:21.103+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:21.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:21.103+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:21.595 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:21.213+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:21.595 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:21.213+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.429610+0000 front 2024-03-27T06:54:48.429582+0000 (oldest deadline 2024-03-27T06:55:12.529624+0000) 2024-03-27T06:55:21.595 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:21.213+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.830496+0000 front 2024-03-27T06:54:54.830572+0000 (oldest deadline 2024-03-27T06:55:20.730585+0000) 2024-03-27T06:55:21.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:21.250+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:21.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:21 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:21.250+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:21.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:21 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:21.488+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:21.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:21 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:21.488+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:21.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:21 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:21.488+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:21.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:21 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:21.458+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:21.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:21 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:21.458+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:21.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:21 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:21.458+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.833911+0000 front 2024-03-27T06:54:54.833945+0000 (oldest deadline 2024-03-27T06:55:20.134097+0000) 2024-03-27T06:55:21.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:21.582+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:21.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:21.722+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:21.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:21.722+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:21.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:21 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:21.722+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:22.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:22 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:22.134+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:22.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:22 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:22.134+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:22.595 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:22 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:22.236+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:39.028449+0000 front 2024-03-27T06:54:39.028528+0000 (oldest deadline 2024-03-27T06:55:04.928465+0000) 2024-03-27T06:55:22.595 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:22 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:22.236+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.429610+0000 front 2024-03-27T06:54:48.429582+0000 (oldest deadline 2024-03-27T06:55:12.529624+0000) 2024-03-27T06:55:22.595 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:22 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5[48492]: 2024-03-27T06:55:22.236+0000 7f7ea7c2d700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.830496+0000 front 2024-03-27T06:54:54.830572+0000 (oldest deadline 2024-03-27T06:55:20.730585+0000) 2024-03-27T06:55:22.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:22 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:22.223+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:22.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:22 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:22.223+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:22.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:22 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:22.223+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:22.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:22 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:22.519+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:22.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:22 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:22.519+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:22.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:22 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:22.519+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:22.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:22 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:22.499+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:22.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:22 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:22.499+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:22.863 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:22 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:22.499+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.833911+0000 front 2024-03-27T06:54:54.833945+0000 (oldest deadline 2024-03-27T06:55:20.134097+0000) 2024-03-27T06:55:22.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:22 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:22.562+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:22.973 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:22 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:22.562+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:22.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:22 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:22.725+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:22.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:22 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:22.725+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:22.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:22 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:22.725+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:23.424 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:23 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:23.156+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:23.425 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:23 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:23.156+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:23.425 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:23 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:23.230+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:23.425 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:23 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:23.230+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:23.425 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:23 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:23.230+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:23.694 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:23 smithi170 podman[74023]: 2024-03-27 06:55:23.429396013 +0000 UTC m=+5.565192549 container died 66769cd36ca02680f64267aee3c1ee60bb950f4faa6f206a2e220b37a6cb0a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD) 2024-03-27T06:55:23.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:23 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:23.467+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:23.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:23 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:23.467+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:23.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:23 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:23.467+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.833911+0000 front 2024-03-27T06:54:54.833945+0000 (oldest deadline 2024-03-27T06:55:20.134097+0000) 2024-03-27T06:55:23.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:23 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:23.556+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:23.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:23 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:23.556+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:23.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:23 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:23.556+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:23.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:23 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:23.549+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:23.973 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:23 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:23.549+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:23.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:23 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:23.746+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:23.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:23 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:23.746+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:23.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:23 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:23.746+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:24.053 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:23 smithi170 podman[74023]: 2024-03-27 06:55:23.790909368 +0000 UTC m=+5.926705884 container cleanup 66769cd36ca02680f64267aee3c1ee60bb950f4faa6f206a2e220b37a6cb0a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:55:24.053 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:23 smithi170 bash[74023]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5 2024-03-27T06:55:24.344 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:24 smithi170 podman[74042]: 2024-03-27 06:55:24.058881319 +0000 UTC m=+0.625038566 container remove 66769cd36ca02680f64267aee3c1ee60bb950f4faa6f206a2e220b37a6cb0a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-27T06:55:24.345 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:24 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:24.183+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:24.345 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:24 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:24.183+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:24.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:24 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:24.253+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:24.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:24 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:24.253+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:24.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:24 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:24.253+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:24.646 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:24 smithi170 podman[74134]: 2024-03-27 06:55:24.401475547 +0000 UTC m=+0.103164536 container create 5e89121caa012e53f28428504ea76d4a0996c05428b3664aebf3c678f7719fd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux ) 2024-03-27T06:55:24.646 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:24 smithi170 podman[74134]: 2024-03-27 06:55:24.31605307 +0000 UTC m=+0.017742073 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:55:24.647 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:24 smithi170 podman[74134]: 2024-03-27 06:55:24.652019755 +0000 UTC m=+0.353708757 container init 5e89121caa012e53f28428504ea76d4a0996c05428b3664aebf3c678f7719fd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-27T06:55:24.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:24 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:24.570+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:24.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:24 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:24.570+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:24.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:24 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:24.570+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:24.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:24 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:24.457+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:24.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:24 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:24.457+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:24.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:24 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:24.457+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.833911+0000 front 2024-03-27T06:54:54.833945+0000 (oldest deadline 2024-03-27T06:55:20.134097+0000) 2024-03-27T06:55:24.942 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:24 smithi170 podman[74134]: 2024-03-27 06:55:24.655442884 +0000 UTC m=+0.357131874 container start 5e89121caa012e53f28428504ea76d4a0996c05428b3664aebf3c678f7719fd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-27T06:55:24.942 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:24 smithi170 podman[74134]: 2024-03-27 06:55:24.754551789 +0000 UTC m=+0.456240780 container attach 5e89121caa012e53f28428504ea76d4a0996c05428b3664aebf3c678f7719fd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-27T06:55:24.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:24 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:24.533+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:24.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:24 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:24.533+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:24.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:24 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:24.712+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:24.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:24 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:24.712+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:24.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:24 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:24.712+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:25.261 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:25 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:25.148+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:25.261 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:25 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:25.148+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:25.261 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 27 06:55:24 smithi170 podman[74134]: 2024-03-27 06:55:24.947862156 +0000 UTC m=+0.649551152 container died 5e89121caa012e53f28428504ea76d4a0996c05428b3664aebf3c678f7719fd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-27T06:55:25.536 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:25 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:25.293+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:25.536 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:25 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:25.293+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:25.536 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:25 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:25.293+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:25.564 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.5.service' 2024-03-27T06:55:25.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:55:25.606 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-27T06:55:25.606 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-27T06:55:25.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.6 2024-03-27T06:55:25.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:25 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:25.582+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:25.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:25 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:25.582+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:25.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:25 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:25.582+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:25.863 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:25 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:25.490+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:25.863 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:25 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:25.490+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:25.863 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:25 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:25.490+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.833911+0000 front 2024-03-27T06:54:54.833945+0000 (oldest deadline 2024-03-27T06:55:20.134097+0000) 2024-03-27T06:55:25.863 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:25 smithi016 systemd[1]: Stopping Ceph osd.6 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:55:25.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:25 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:25.541+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:25.973 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:25 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:25.541+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:25.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:25 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:25.697+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:25.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:25 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:25.697+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:25.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:25 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:25.697+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:26.362 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:25 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:25.925+0000 7f6160d55700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:55:26.362 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:25 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:25.925+0000 7f6160d55700 -1 osd.6 103 *** Got signal Terminated *** 2024-03-27T06:55:26.362 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:25 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:25.925+0000 7f6160d55700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:55:26.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:26.343+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:26.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:26.343+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:26.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:26.343+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:26.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:26.127+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:26.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:26 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:26.127+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:26.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:26.580+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:26.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:26.580+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:26.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:26.580+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:26.863 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:26.497+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:26.863 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:26.497+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:26.863 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:26 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:26.497+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.833911+0000 front 2024-03-27T06:54:54.833945+0000 (oldest deadline 2024-03-27T06:55:20.134097+0000) 2024-03-27T06:55:26.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:26.536+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:26.973 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:26.536+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:26.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:26.735+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:26.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:26.735+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:26.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:26 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:26.735+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:27.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:27 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:27.129+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:27.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:27 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:27.129+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:27.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:27 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:27.312+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:27.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:27 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:27.312+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:27.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:27 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:27.312+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:27.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:27.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:27.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:27.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:27.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:27.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:27.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:27.528+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:27.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:27.528+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:27.863 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:27.528+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.833911+0000 front 2024-03-27T06:54:54.833945+0000 (oldest deadline 2024-03-27T06:55:20.134097+0000) 2024-03-27T06:55:27.974 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:27 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:27.562+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:27.974 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:27 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:27.562+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:27.974 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:27 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:27.766+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:27.975 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:27 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:27.766+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:27.975 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:27 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:27.766+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:28.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:28.171+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:28.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:28.171+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:28.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:28.333+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:28.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:28.333+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:28.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:28 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:28.333+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:28.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:28.565+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:28.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:28.565+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:28.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:28.565+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.833911+0000 front 2024-03-27T06:54:54.833945+0000 (oldest deadline 2024-03-27T06:55:20.134097+0000) 2024-03-27T06:55:28.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:28.568+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:28.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:28.568+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:28.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:28.568+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:28.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:28 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:28.568+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:28.973 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:28 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:28.553+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:28.973 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:28 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:28.553+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:28.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:28 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:28.774+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:28.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:28 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:28.774+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:28.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:28 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:28.774+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:29.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:29 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:29.185+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:29.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:29 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:29.185+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:29.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:29 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:29.185+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:29.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:29 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:29.321+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:29.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:29 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:29.321+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:29.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:29 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:29.321+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:29.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:29 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:29.519+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:29.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:29 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:29.519+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:29.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:29 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:29.519+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.833911+0000 front 2024-03-27T06:54:54.833945+0000 (oldest deadline 2024-03-27T06:55:20.134097+0000) 2024-03-27T06:55:29.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:29 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:29.519+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:29.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:29 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:29.519+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:29.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:29 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:29.519+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:29.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:29 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:29.520+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:29.972 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:29 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:29.553+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:29.973 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:29 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:29.553+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:29.973 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:29 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:29.553+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:05.650582+0000 front 2024-03-27T06:55:05.650658+0000 (oldest deadline 2024-03-27T06:55:29.150520+0000) 2024-03-27T06:55:29.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:29 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:29.784+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:29.974 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:29 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:29.784+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:29.974 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:29 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:29.784+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:30.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:30 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:30.221+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:30.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:30 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:30.221+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:30.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:30 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:30.221+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:30.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:30 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:30.306+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:30.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:30 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:30.306+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:30.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:30 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:30.306+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:30.799 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:30 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:30.539+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:30.799 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:30 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:30.539+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:30.799 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:30 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:30.539+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:05.650582+0000 front 2024-03-27T06:55:05.650658+0000 (oldest deadline 2024-03-27T06:55:29.150520+0000) 2024-03-27T06:55:30.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:30.567+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.630494+0000 front 2024-03-27T06:54:41.630574+0000 (oldest deadline 2024-03-27T06:55:03.330647+0000) 2024-03-27T06:55:30.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:30.567+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:47.832705+0000 front 2024-03-27T06:54:47.832742+0000 (oldest deadline 2024-03-27T06:55:12.532676+0000) 2024-03-27T06:55:30.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6[60649]: 2024-03-27T06:55:30.567+0000 7f615d349700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.833911+0000 front 2024-03-27T06:54:54.833945+0000 (oldest deadline 2024-03-27T06:55:20.134097+0000) 2024-03-27T06:55:30.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:30.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:30.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:30.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:30.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:30.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:30.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:30 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:30.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:31.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:30 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:30.804+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:31.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:30 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:30.804+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:31.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:30 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:30.804+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:31.579 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:31 smithi016 podman[94677]: 2024-03-27 06:55:31.300468769 +0000 UTC m=+5.494695961 container died ca65ed3daf3f7949889110ad5e72e81372940806d1db3c312df5401a59884d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2) 2024-03-27T06:55:31.580 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:31 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:31.543+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:31.580 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:31 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:31.543+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:31.580 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:31 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:31.543+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:31.580 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:31 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:31.543+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:31.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:31.246+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:31.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:31.246+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:31.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:31.246+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:31.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:31.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:31.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:31.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:31.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:31 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:31.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:31.844 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:31.588+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:31.844 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:31.588+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:31.844 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:31.588+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:05.650582+0000 front 2024-03-27T06:55:05.650658+0000 (oldest deadline 2024-03-27T06:55:29.150520+0000) 2024-03-27T06:55:31.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:31 smithi016 podman[94677]: 2024-03-27 06:55:31.809084131 +0000 UTC m=+6.003311325 container cleanup ca65ed3daf3f7949889110ad5e72e81372940806d1db3c312df5401a59884d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-27T06:55:31.862 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:31 smithi016 bash[94677]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6 2024-03-27T06:55:32.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:31.849+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:32.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:31.849+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:32.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:31.849+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:32.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:31.849+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:32.362 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:32 smithi016 podman[94695]: 2024-03-27 06:55:32.035548892 +0000 UTC m=+0.738729304 container remove ca65ed3daf3f7949889110ad5e72e81372940806d1db3c312df5401a59884d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:55:32.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:32.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:32.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:32.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:32.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:32.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:32.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:32.274+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:32.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:32.274+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:32.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:32 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:32.274+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:32.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:32 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:32.530+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:32.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:32 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:32.530+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:32.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:32 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:32.530+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:32.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:32 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:32.530+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:32.859 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:32 smithi016 podman[94786]: 2024-03-27 06:55:32.381833899 +0000 UTC m=+0.018184154 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:55:32.859 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:32 smithi016 podman[94786]: 2024-03-27 06:55:32.494723585 +0000 UTC m=+0.131073840 container create e03a7f58d17cda2f0d8f4135cb31b8efbbb6bb5e8493be2c33ff350bd32c5549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-27T06:55:32.859 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:32 smithi016 podman[94786]: 2024-03-27 06:55:32.863885007 +0000 UTC m=+0.500235264 container init e03a7f58d17cda2f0d8f4135cb31b8efbbb6bb5e8493be2c33ff350bd32c5549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6-deactivate, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-27T06:55:32.886 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:32 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:32.560+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:32.886 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:32 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:32.560+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:32.886 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:32 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:32.560+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:05.650582+0000 front 2024-03-27T06:55:05.650658+0000 (oldest deadline 2024-03-27T06:55:29.150520+0000) 2024-03-27T06:55:33.112 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:32 smithi016 podman[94786]: 2024-03-27 06:55:32.867099254 +0000 UTC m=+0.503449509 container start e03a7f58d17cda2f0d8f4135cb31b8efbbb6bb5e8493be2c33ff350bd32c5549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-27T06:55:33.112 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:32 smithi016 podman[94786]: 2024-03-27 06:55:32.980575732 +0000 UTC m=+0.616926004 container attach e03a7f58d17cda2f0d8f4135cb31b8efbbb6bb5e8493be2c33ff350bd32c5549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, org.label-schema.build-date=20240326, GIT_CLEAN=True) 2024-03-27T06:55:33.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:32 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:32.891+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:33.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:32 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:32.891+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:33.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:32 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:32.891+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:33.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:32 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:32.891+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:33.441 INFO:journalctl@ceph.osd.6.smithi016.stdout:Mar 27 06:55:33 smithi016 podman[94786]: 2024-03-27 06:55:33.163001571 +0000 UTC m=+0.799351829 container died e03a7f58d17cda2f0d8f4135cb31b8efbbb6bb5e8493be2c33ff350bd32c5549 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-6-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-27T06:55:33.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:33 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:33.281+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:33.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:33 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:33.281+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:33.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:33 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:33.281+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:33.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:33 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:33.318+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:33.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:33 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:33.318+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:33.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:33 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:33.318+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:33.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:33.520+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:33.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:33.520+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:33.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:33.520+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:33.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:33 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:33.520+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:33.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.6.service' 2024-03-27T06:55:33.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:55:33.812 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-27T06:55:33.813 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-27T06:55:33.813 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.7 2024-03-27T06:55:33.841 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:33 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:33.589+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:33.841 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:33 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:33.589+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:33.841 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:33 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:33.589+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:05.650582+0000 front 2024-03-27T06:55:05.650658+0000 (oldest deadline 2024-03-27T06:55:29.150520+0000) 2024-03-27T06:55:34.109 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:33 smithi156 systemd[1]: Stopping Ceph osd.7 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:55:34.109 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:33 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:33.902+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:34.109 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:33 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:33.902+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:34.110 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:33 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:33.902+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:34.110 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:33 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:33.902+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:34.472 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:34.114+0000 7f3a97845700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:55:34.472 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:34.114+0000 7f3a97845700 -1 osd.7 103 *** Got signal Terminated *** 2024-03-27T06:55:34.473 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:34.114+0000 7f3a97845700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:55:34.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:34.298+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:34.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:34.298+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:34.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:34.298+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:34.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:34.355+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:34.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:34.355+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:34.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:34 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:34.355+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:34.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:34.484+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:34.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:34.484+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:34.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:34.484+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:34.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:34 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:34.484+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:34.935 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:34.563+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:34.936 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:34.563+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:34.936 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:34.563+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:05.650582+0000 front 2024-03-27T06:55:05.650658+0000 (oldest deadline 2024-03-27T06:55:29.150520+0000) 2024-03-27T06:55:35.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:34.939+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:35.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:34.939+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:35.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:34.939+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:35.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:34 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:34.939+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:35.508 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:35 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:35.259+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:35.508 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:35 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:35.259+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:35.508 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:35 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:35.259+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:35.508 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:35 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:35.387+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:35.508 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:35 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:35.387+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:35.508 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:35 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:35.387+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:35.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:35 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:35.513+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:35.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:35 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:35.513+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:35.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:35 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:35.513+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:35.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:35 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:35.513+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:35.916 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:35.595+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:35.916 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:35.595+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:35.916 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:35.595+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:05.650582+0000 front 2024-03-27T06:55:05.650658+0000 (oldest deadline 2024-03-27T06:55:29.150520+0000) 2024-03-27T06:55:36.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:35.921+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:36.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:35.921+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:36.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:35.921+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:36.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:35.921+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:36.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:36 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:36.218+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:36.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:36 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:36.218+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:36.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:36 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:36.218+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:36.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:36 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:36.361+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:36.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:36 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:36.361+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:36.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:36 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:36.361+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:36.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:36 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:36.361+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:36.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:36 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:36.555+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:36.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:36 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:36.555+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:36.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:36 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:36.555+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:36.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:36 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:36.555+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:36.868 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:36 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:36.644+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:36.868 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:36 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:36.644+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:36.868 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:36 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:36.644+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:05.650582+0000 front 2024-03-27T06:55:05.650658+0000 (oldest deadline 2024-03-27T06:55:29.150520+0000) 2024-03-27T06:55:37.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:36 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:36.872+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:37.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:36 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:36.872+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:37.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:36 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:36.872+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:37.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:36 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:36.872+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:37.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:36 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:36.872+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:37.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:37 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:37.217+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:37.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:37 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:37.217+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:37.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:37 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:37.217+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:37.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:37 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:37.313+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:37.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:37 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:37.313+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:37.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:37 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:37.313+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:37.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:37 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:37.313+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:37.722 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:37.671+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:37.723 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:37.671+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:37.723 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:37.671+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:05.650582+0000 front 2024-03-27T06:55:05.650658+0000 (oldest deadline 2024-03-27T06:55:29.150520+0000) 2024-03-27T06:55:37.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:37 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:37.583+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:37.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:37 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:37.583+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:37.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:37 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:37.583+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:37.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:37 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:37.583+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:37.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:37 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:37.583+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:38.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:37.893+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:38.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:37.893+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:38.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:37.893+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:38.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:37.893+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:38.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:37 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:37.893+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:38.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:38 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:38.212+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:38.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:38 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:38.212+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:38.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:38 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:38.212+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:38.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:38 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:38.358+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:38.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:38 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:38.358+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:38.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:38 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:38.358+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:38.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:38 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:38.358+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:38.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:38 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:38.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:38.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:38 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:38.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:38.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:38 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:38.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:38.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:38 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:38.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:38.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:38 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:38.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:38.933 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:38 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:38.648+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.647841+0000 front 2024-03-27T06:54:50.647786+0000 (oldest deadline 2024-03-27T06:55:12.347861+0000) 2024-03-27T06:55:38.933 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:38 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:38.648+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:58.049436+0000 front 2024-03-27T06:54:58.049504+0000 (oldest deadline 2024-03-27T06:55:22.149439+0000) 2024-03-27T06:55:38.933 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:38 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:38.648+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:05.650582+0000 front 2024-03-27T06:55:05.650658+0000 (oldest deadline 2024-03-27T06:55:29.150520+0000) 2024-03-27T06:55:38.933 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:38 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7[54346]: 2024-03-27T06:55:38.648+0000 7f3a93e39700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.151961+0000 front 2024-03-27T06:55:13.151997+0000 (oldest deadline 2024-03-27T06:55:37.852022+0000) 2024-03-27T06:55:39.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:38 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:38.937+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:39.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:38 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:38.938+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:39.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:38 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:38.938+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:39.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:38 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:38.938+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:39.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:38 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:38.938+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:39.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:39 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:39.208+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:39.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:39 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:39.208+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:39.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:39 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:39.208+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:39.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:39 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:39.208+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:14.206095+0000 front 2024-03-27T06:55:14.206162+0000 (oldest deadline 2024-03-27T06:55:38.306242+0000) 2024-03-27T06:55:39.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:39 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:39.333+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:39.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:39 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:39.333+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:39.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:39 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:39.333+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:39.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:39 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:39.333+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:39.809 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:39 smithi156 podman[72853]: 2024-03-27 06:55:39.54238924 +0000 UTC m=+5.535821005 container died fc7658ca7ad6fbebf26ca60ae9d7b73a27dd65064b98aa098f9789ec36974448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-03-27T06:55:39.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:39 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:39.585+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:39.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:39 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:39.585+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:39.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:39 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:39.585+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:39.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:39 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:39.585+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:39.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:39 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:39.585+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:40.193 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:39 smithi156 podman[72853]: 2024-03-27 06:55:39.913696306 +0000 UTC m=+5.907128068 container cleanup fc7658ca7ad6fbebf26ca60ae9d7b73a27dd65064b98aa098f9789ec36974448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:55:40.193 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:39 smithi156 bash[72853]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7 2024-03-27T06:55:40.193 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:39 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:39.917+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:40.193 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:39 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:39.917+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:40.193 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:39 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:39.917+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:40.194 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:39 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:39.917+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:40.194 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:39 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:39.917+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:40.472 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:40 smithi156 podman[72882]: 2024-03-27 06:55:40.198390539 +0000 UTC m=+0.653337465 container remove fc7658ca7ad6fbebf26ca60ae9d7b73a27dd65064b98aa098f9789ec36974448 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-27T06:55:40.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:40 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:40.181+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:40.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:40 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:40.181+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:40.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:40 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:40.181+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:40.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:40 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:40.181+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:14.206095+0000 front 2024-03-27T06:55:14.206162+0000 (oldest deadline 2024-03-27T06:55:38.306242+0000) 2024-03-27T06:55:40.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:40 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:40.284+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:40.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:40 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:40.284+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:40.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:40 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:40.284+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:40.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:40 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:40.284+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:40.855 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:40 smithi156 podman[72972]: 2024-03-27 06:55:40.53287335 +0000 UTC m=+0.113465612 container create d934fad36a32f31fea236716ed8782f76c2d1ccfc9e63fbe35a097c8accfd4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-27T06:55:40.855 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:40 smithi156 podman[72972]: 2024-03-27 06:55:40.436964533 +0000 UTC m=+0.017556801 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:55:40.855 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:40 smithi156 podman[72972]: 2024-03-27 06:55:40.777320958 +0000 UTC m=+0.357913220 container init d934fad36a32f31fea236716ed8782f76c2d1ccfc9e63fbe35a097c8accfd4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-27T06:55:40.855 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:40 smithi156 podman[72972]: 2024-03-27 06:55:40.78076937 +0000 UTC m=+0.361361622 container start d934fad36a32f31fea236716ed8782f76c2d1ccfc9e63fbe35a097c8accfd4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:55:40.855 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:40 smithi156 podman[72972]: 2024-03-27 06:55:40.860092331 +0000 UTC m=+0.440684576 container attach d934fad36a32f31fea236716ed8782f76c2d1ccfc9e63fbe35a097c8accfd4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:55:41.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:40 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:40.628+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:41.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:40 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:40.628+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:41.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:40 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:40.628+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:41.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:40 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:40.628+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:41.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:40 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:40.628+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:41.223 INFO:journalctl@ceph.osd.7.smithi156.stdout:Mar 27 06:55:41 smithi156 podman[72972]: 2024-03-27 06:55:41.068994488 +0000 UTC m=+0.649586749 container died d934fad36a32f31fea236716ed8782f76c2d1ccfc9e63fbe35a097c8accfd4ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:55:41.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:40 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:40.910+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:41.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:40 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:40.910+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:41.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:40 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:40.910+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:41.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:40 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:40.910+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:41.224 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:40 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:40.910+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:41.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:41 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:41.194+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:41.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:41 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:41.194+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:41.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:41 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:41.194+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:41.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:41 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:41.194+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:14.206095+0000 front 2024-03-27T06:55:14.206162+0000 (oldest deadline 2024-03-27T06:55:38.306242+0000) 2024-03-27T06:55:41.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:41 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:41.267+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:41.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:41 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:41.267+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:41.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:41 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:41.267+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:41.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:41 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:41.267+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:41.687 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.7.service' 2024-03-27T06:55:41.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:55:41.768 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-27T06:55:41.769 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-27T06:55:41.769 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.8 2024-03-27T06:55:41.972 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:41 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:41.905+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:41.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:41 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:41.905+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:41.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:41 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:41.905+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:41.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:41 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:41.905+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:41.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:41 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:41.905+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:42.095 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:41 smithi170 systemd[1]: Stopping Ceph osd.8 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:55:42.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:41 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:41.641+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:42.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:41 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:41.641+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:42.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:41 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:41.641+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:42.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:41 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:41.641+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:42.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:41 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:41.641+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:42.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:42 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:42.243+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:42.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:42 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:42.243+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:42.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:42 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:42.243+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:42.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:42 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:42.243+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:42.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:42 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:42.102+0000 7f92e17d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:55:42.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:42 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:42.102+0000 7f92e17d0700 -1 osd.8 103 *** Got signal Terminated *** 2024-03-27T06:55:42.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:42 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:42.102+0000 7f92e17d0700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:55:42.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:42 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:42.193+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:42.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:42 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:42.193+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:42.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:42 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:42.193+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:42.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:42 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:42.193+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:14.206095+0000 front 2024-03-27T06:55:14.206162+0000 (oldest deadline 2024-03-27T06:55:38.306242+0000) 2024-03-27T06:55:43.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:42 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:42.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:43.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:42 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:42.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:43.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:42 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:42.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:43.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:42 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:42.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:43.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:42 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:42.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:43.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:42 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:42.889+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:43.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:42 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:42.889+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:43.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:42 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:42.889+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:43.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:42 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:42.889+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:43.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:42 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:42.889+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:43.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:43 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:43.201+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:43.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:43 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:43.201+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:43.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:43 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:43.201+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:43.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:43 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:43.201+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:14.206095+0000 front 2024-03-27T06:55:14.206162+0000 (oldest deadline 2024-03-27T06:55:38.306242+0000) 2024-03-27T06:55:43.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:43 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:43.265+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:43.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:43 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:43.265+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:43.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:43 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:43.265+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:43.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:43 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:43.265+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:43.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:43 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:43.597+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:43.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:43 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:43.597+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:43.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:43 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:43.597+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:43.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:43 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:43.597+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:43.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:43 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:43.597+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:44.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:43 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:43.865+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:44.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:43 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:43.865+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:44.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:43 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:43.865+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:44.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:43 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:43.865+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:44.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:43 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:43.865+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:44.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:43 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:43.865+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:44.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:44 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:44.232+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:44.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:44 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:44.232+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:44.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:44 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:44.232+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:44.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:44 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:44.232+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:14.206095+0000 front 2024-03-27T06:55:14.206162+0000 (oldest deadline 2024-03-27T06:55:38.306242+0000) 2024-03-27T06:55:44.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:44 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:44.267+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:44.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:44 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:44.267+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:44.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:44 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:44.267+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:44.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:44 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:44.267+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:44.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:44 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:44.608+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:44.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:44 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:44.608+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:44.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:44 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:44.608+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:44.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:44 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:44.608+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:44.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:44 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:44.608+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:44.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:44 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:44.608+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:19.859698+0000 front 2024-03-27T06:55:19.859654+0000 (oldest deadline 2024-03-27T06:55:43.959388+0000) 2024-03-27T06:55:45.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:44 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:44.855+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:45.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:44 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:44.855+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:45.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:44 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:44.855+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:45.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:44 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:44.855+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:45.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:44 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:44.855+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:45.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:44 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:44.855+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:45.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:45 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:45.208+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:45.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:45 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:45.208+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:45.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:45 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:45.208+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:45.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:45 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:45.208+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:14.206095+0000 front 2024-03-27T06:55:14.206162+0000 (oldest deadline 2024-03-27T06:55:38.306242+0000) 2024-03-27T06:55:45.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:45 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:45.313+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:45.596 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:45 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:45.313+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:45.596 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:45 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:45.313+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:45.596 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:45 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:45.313+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:45.596 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:45 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:45.313+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:45.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:45 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:45.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:45.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:45 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:45.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:45.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:45 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:45.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:45.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:45 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:45.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:45.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:45 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:45.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:45.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:45 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:45.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:19.859698+0000 front 2024-03-27T06:55:19.859654+0000 (oldest deadline 2024-03-27T06:55:43.959388+0000) 2024-03-27T06:55:46.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:45 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:45.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:46.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:45 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:45.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:46.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:45 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:45.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:46.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:45 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:45.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:46.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:45 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:45.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:46.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:45 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:45.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:46.594 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:46 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:46.234+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:45.502382+0000 front 2024-03-27T06:54:45.502220+0000 (oldest deadline 2024-03-27T06:55:11.402241+0000) 2024-03-27T06:55:46.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:46 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:46.234+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.903318+0000 front 2024-03-27T06:54:54.903343+0000 (oldest deadline 2024-03-27T06:55:20.203510+0000) 2024-03-27T06:55:46.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:46 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:46.234+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.005164+0000 front 2024-03-27T06:55:06.005079+0000 (oldest deadline 2024-03-27T06:55:28.905102+0000) 2024-03-27T06:55:46.595 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:46 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8[55334]: 2024-03-27T06:55:46.234+0000 7f92dddc4700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:14.206095+0000 front 2024-03-27T06:55:14.206162+0000 (oldest deadline 2024-03-27T06:55:38.306242+0000) 2024-03-27T06:55:46.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:46 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:46.347+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:46.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:46 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:46.347+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:46.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:46 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:46.347+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:46.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:46 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:46.347+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:46.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:46 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:46.347+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:46.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:46.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:46.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:46.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:46.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:46.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:46.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:46.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:46.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:46.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:46.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:46 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:46.545+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:19.859698+0000 front 2024-03-27T06:55:19.859654+0000 (oldest deadline 2024-03-27T06:55:43.959388+0000) 2024-03-27T06:55:47.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:46 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:46.844+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:47.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:46 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:46.844+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:47.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:46 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:46.844+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:47.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:46 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:46.844+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:47.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:46 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:46.844+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:47.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:46 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:46.844+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:47.831 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:47 smithi170 podman[74252]: 2024-03-27 06:55:47.569650686 +0000 UTC m=+5.597189037 container died 61c64560a94a0c56c8251dd83ff17695b46cad4eeeab5f21af9a85eeb5d0b9e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-27T06:55:47.832 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:47 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:47.364+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:47.832 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:47 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:47.364+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:47.832 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:47 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:47.364+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:47.832 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:47 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:47.364+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:47.832 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:47 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:47.364+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:47.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:47 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:47.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:47.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:47 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:47.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:47.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:47 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:47.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:47.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:47 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:47.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:47.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:47 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:47.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:47.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:47 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:47.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:19.859698+0000 front 2024-03-27T06:55:19.859654+0000 (oldest deadline 2024-03-27T06:55:43.959388+0000) 2024-03-27T06:55:48.094 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:47 smithi170 podman[74252]: 2024-03-27 06:55:47.927724889 +0000 UTC m=+5.955263239 container cleanup 61c64560a94a0c56c8251dd83ff17695b46cad4eeeab5f21af9a85eeb5d0b9e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-03-27T06:55:48.094 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:47 smithi170 bash[74252]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8 2024-03-27T06:55:48.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:47 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:47.859+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:48.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:47 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:47.859+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:48.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:47 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:47.859+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:48.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:47 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:47.859+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:48.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:47 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:47.859+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:48.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:47 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:47.859+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:48.538 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:48 smithi170 podman[74269]: 2024-03-27 06:55:48.237380978 +0000 UTC m=+0.659822149 container remove 61c64560a94a0c56c8251dd83ff17695b46cad4eeeab5f21af9a85eeb5d0b9e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD) 2024-03-27T06:55:48.538 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:48 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:48.398+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:48.539 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:48 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:48.398+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:48.539 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:48 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:48.398+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:48.539 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:48 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:48.398+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:48.539 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:48 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:48.398+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:48.789 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:48 smithi170 podman[74355]: 2024-03-27 06:55:48.446565634 +0000 UTC m=+0.017654216 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:55:48.789 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:48 smithi170 podman[74355]: 2024-03-27 06:55:48.554992998 +0000 UTC m=+0.126081558 container create 74d4d606d5f57d4006f3dd0b919b8a6eab52b090ea11a2381f85409f6676a069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:55:48.789 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:48 smithi170 podman[74355]: 2024-03-27 06:55:48.790989039 +0000 UTC m=+0.362077611 container init 74d4d606d5f57d4006f3dd0b919b8a6eab52b090ea11a2381f85409f6676a069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326) 2024-03-27T06:55:48.789 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:48 smithi170 podman[74355]: 2024-03-27 06:55:48.794743496 +0000 UTC m=+0.365832054 container start 74d4d606d5f57d4006f3dd0b919b8a6eab52b090ea11a2381f85409f6676a069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-27T06:55:49.080 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:48 smithi170 podman[74355]: 2024-03-27 06:55:48.891061024 +0000 UTC m=+0.462149598 container attach 74d4d606d5f57d4006f3dd0b919b8a6eab52b090ea11a2381f85409f6676a069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-27T06:55:49.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:48 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:48.638+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:49.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:48 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:48.638+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:49.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:48 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:48.638+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:49.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:48 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:48.638+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:49.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:48 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:48.638+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:49.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:48 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:48.638+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:19.859698+0000 front 2024-03-27T06:55:19.859654+0000 (oldest deadline 2024-03-27T06:55:43.959388+0000) 2024-03-27T06:55:49.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:48 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:48.818+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:49.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:48 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:48.818+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:49.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:48 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:48.818+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:49.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:48 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:48.818+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:49.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:48 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:48.818+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:49.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:48 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:48.818+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:49.344 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 27 06:55:49 smithi170 podman[74355]: 2024-03-27 06:55:49.08540589 +0000 UTC m=+0.656494471 container died 74d4d606d5f57d4006f3dd0b919b8a6eab52b090ea11a2381f85409f6676a069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:55:49.698 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:49 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:49.366+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:49.698 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:49 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:49.366+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:49.698 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:49 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:49.366+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:49.699 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:49 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:49.366+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:49.699 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:49 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:49.366+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:49.720 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.8.service' 2024-03-27T06:55:49.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:55:49.761 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-27T06:55:49.761 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-27T06:55:49.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.9 2024-03-27T06:55:50.082 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:49 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:49.630+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:50.082 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:49 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:49.630+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:50.082 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:49 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:49.630+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:50.082 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:49 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:49.630+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:50.083 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:49 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:49.630+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:50.083 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:49 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:49.630+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:19.859698+0000 front 2024-03-27T06:55:19.859654+0000 (oldest deadline 2024-03-27T06:55:43.959388+0000) 2024-03-27T06:55:50.083 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:49 smithi016 systemd[1]: Stopping Ceph osd.9 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:55:50.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:49 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:49.815+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:50.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:49 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:49.815+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:50.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:49 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:49.815+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:50.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:49 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:49.815+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:50.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:49 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:49.815+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:50.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:49 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:49.815+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:50.362 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:50 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:50.085+0000 7f4b6532d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:55:50.362 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:50 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:50.085+0000 7f4b6532d700 -1 osd.9 103 *** Got signal Terminated *** 2024-03-27T06:55:50.362 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:50 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:50.085+0000 7f4b6532d700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:55:50.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:50 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:50.348+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:50.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:50 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:50.348+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:50.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:50 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:50.348+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:50.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:50 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:50.348+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:50.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:50 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:50.348+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:50.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:50 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:50.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:50.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:50 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:50.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:50.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:50 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:50.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:50.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:50 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:50.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:50.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:50 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:50.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:50.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:50 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:50.590+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:19.859698+0000 front 2024-03-27T06:55:19.859654+0000 (oldest deadline 2024-03-27T06:55:43.959388+0000) 2024-03-27T06:55:51.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:50 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:50.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:51.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:50 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:50.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:51.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:50 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:50.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:51.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:50 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:50.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:51.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:50 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:50.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:51.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:50 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:50.812+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:51.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:51 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:51.382+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:51.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:51 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:51.382+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:51.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:51 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:51.382+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:51.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:51 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:51.382+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:51.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:51 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:51.382+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:51.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:51 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:51.586+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:51.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:51 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:51.586+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:51.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:51 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:51.586+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:51.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:51 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:51.586+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:51.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:51 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:51.586+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:51.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:51 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:51.586+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:19.859698+0000 front 2024-03-27T06:55:19.859654+0000 (oldest deadline 2024-03-27T06:55:43.959388+0000) 2024-03-27T06:55:52.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:51 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:51.847+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:52.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:51 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:51.847+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:52.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:51 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:51.847+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:52.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:51 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:51.847+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:52.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:51 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:51.847+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:52.224 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:51 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:51.847+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:52.224 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:51 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:51.847+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:55:52.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:52 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:52.361+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:52.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:52 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:52.361+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:52.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:52 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:52.361+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:52.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:52 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:52.361+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:52.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:52 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:52.361+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:52.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:52 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:52.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:52.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:52 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:52.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:52.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:52 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:52.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:52.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:52 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:52.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:52.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:52 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:52.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:52.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:52 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:52.578+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:19.859698+0000 front 2024-03-27T06:55:19.859654+0000 (oldest deadline 2024-03-27T06:55:43.959388+0000) 2024-03-27T06:55:53.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:52 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:52.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:53.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:52 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:52.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:53.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:52 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:52.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:53.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:52 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:52.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:53.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:52 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:52.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:53.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:52 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:52.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:53.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:52 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:52.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:55:53.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:53 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:53.371+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:53.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:53 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:53.371+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:53.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:53 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:53.371+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:53.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:53 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:53.371+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:53.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:53 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:53.371+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:53.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:53 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:53.582+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:53.862 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:53 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:53.582+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:53.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:53 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:53.583+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:53.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:53 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:53.583+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:53.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:53 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:53.583+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:53.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:53 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:53.583+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:19.859698+0000 front 2024-03-27T06:55:19.859654+0000 (oldest deadline 2024-03-27T06:55:43.959388+0000) 2024-03-27T06:55:53.863 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:53 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:53.583+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:30.360518+0000 front 2024-03-27T06:55:30.360539+0000 (oldest deadline 2024-03-27T06:55:53.260476+0000) 2024-03-27T06:55:54.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:53 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:53.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:54.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:53 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:53.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:54.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:53 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:53.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:54.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:53 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:53.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:54.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:53 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:53.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:54.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:53 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:53.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:54.224 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:53 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:53.852+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:55:54.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:54 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:54.357+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:54.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:54 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:54.357+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:54.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:54 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:54.357+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:54.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:54 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:54.357+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:54.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:54 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:54.357+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:55.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:54 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:54.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:42.351337+0000 front 2024-03-27T06:54:42.351131+0000 (oldest deadline 2024-03-27T06:55:03.451291+0000) 2024-03-27T06:55:55.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:54 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:54.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.353784+0000 front 2024-03-27T06:54:50.354006+0000 (oldest deadline 2024-03-27T06:55:12.653818+0000) 2024-03-27T06:55:55.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:54 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:54.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:55.554883+0000 front 2024-03-27T06:54:55.554754+0000 (oldest deadline 2024-03-27T06:55:20.854878+0000) 2024-03-27T06:55:55.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:54 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:54.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.155942+0000 front 2024-03-27T06:55:06.155892+0000 (oldest deadline 2024-03-27T06:55:28.456013+0000) 2024-03-27T06:55:55.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:54 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:54.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:12.858588+0000 front 2024-03-27T06:55:12.858814+0000 (oldest deadline 2024-03-27T06:55:37.558494+0000) 2024-03-27T06:55:55.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:54 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:54.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:19.859698+0000 front 2024-03-27T06:55:19.859654+0000 (oldest deadline 2024-03-27T06:55:43.959388+0000) 2024-03-27T06:55:55.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:54 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9[67643]: 2024-03-27T06:55:54.625+0000 7f4b61921700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:30.360518+0000 front 2024-03-27T06:55:30.360539+0000 (oldest deadline 2024-03-27T06:55:53.260476+0000) 2024-03-27T06:55:55.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:54 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:54.803+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:55.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:54 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:54.803+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:55.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:54 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:54.803+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:55.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:54 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:54.803+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:55.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:54 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:54.803+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:55.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:54 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:54.803+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:55.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:54 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:54.803+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:55:55.757 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:55 smithi016 podman[94906]: 2024-03-27 06:55:55.479611796 +0000 UTC m=+5.516553549 container stop 0084ba05291199fa36b5fa71041fc0c50b7e195d2a893dd7fd4db44b0e548d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:55:55.757 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:55 smithi016 podman[94906]: 2024-03-27 06:55:55.50525919 +0000 UTC m=+5.542200957 container died 0084ba05291199fa36b5fa71041fc0c50b7e195d2a893dd7fd4db44b0e548d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS) 2024-03-27T06:55:55.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:55 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:55.363+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:55.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:55 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:55.363+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:55.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:55 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:55.363+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:55.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:55 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:55.363+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:55.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:55 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:55.363+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:55.845 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:55 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:55.363+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:55:56.040 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:55 smithi016 podman[94906]: 2024-03-27 06:55:55.861743716 +0000 UTC m=+5.898685468 container cleanup 0084ba05291199fa36b5fa71041fc0c50b7e195d2a893dd7fd4db44b0e548d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-27T06:55:56.040 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:55 smithi016 bash[94906]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9 2024-03-27T06:55:56.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:55 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:55.770+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:56.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:55 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:55.770+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:56.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:55 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:55.770+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:56.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:55 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:55.771+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:56.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:55 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:55.771+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:56.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:55 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:55.771+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:56.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:55 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:55.771+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:55:56.361 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:56 smithi016 podman[94923]: 2024-03-27 06:55:56.045508499 +0000 UTC m=+0.559152337 container remove 0084ba05291199fa36b5fa71041fc0c50b7e195d2a893dd7fd4db44b0e548d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-03-27T06:55:56.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:56 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:56.321+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:56.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:56 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:56.321+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:56.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:56 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:56.321+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:56.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:56 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:56.321+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:56.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:56 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:56.321+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:56.596 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:56 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:56.321+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:55:56.759 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:56 smithi016 podman[95015]: 2024-03-27 06:55:56.390390181 +0000 UTC m=+0.017494018 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:55:56.760 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:56 smithi016 podman[95015]: 2024-03-27 06:55:56.496458409 +0000 UTC m=+0.123562255 container create 356968fe1fdbb1a9dd4883a214fbe45d400c9a7bd9aa665147bc4d83f23294b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-27T06:55:56.760 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:56 smithi016 podman[95015]: 2024-03-27 06:55:56.765266219 +0000 UTC m=+0.392370023 container init 356968fe1fdbb1a9dd4883a214fbe45d400c9a7bd9aa665147bc4d83f23294b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-03-27T06:55:57.059 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:56 smithi016 podman[95015]: 2024-03-27 06:55:56.7686363 +0000 UTC m=+0.395740112 container start 356968fe1fdbb1a9dd4883a214fbe45d400c9a7bd9aa665147bc4d83f23294b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:55:57.059 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:56 smithi016 podman[95015]: 2024-03-27 06:55:56.840512712 +0000 UTC m=+0.467616526 container attach 356968fe1fdbb1a9dd4883a214fbe45d400c9a7bd9aa665147bc4d83f23294b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2) 2024-03-27T06:55:57.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:56 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:56.767+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:57.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:56 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:56.767+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:57.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:56 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:56.767+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:57.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:56 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:56.767+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:57.224 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:56 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:56.767+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:57.224 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:56 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:56.767+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:57.224 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:56 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:56.767+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:55:57.336 INFO:journalctl@ceph.osd.9.smithi016.stdout:Mar 27 06:55:57 smithi016 podman[95015]: 2024-03-27 06:55:57.06467379 +0000 UTC m=+0.691777607 container died 356968fe1fdbb1a9dd4883a214fbe45d400c9a7bd9aa665147bc4d83f23294b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-27T06:55:57.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:57 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:57.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:57.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:57 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:57.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:57.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:57 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:57.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:57.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:57 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:57.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:57.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:57 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:57.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:57.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:57 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:57.278+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:55:57.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.9.service' 2024-03-27T06:55:57.698 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:55:57.698 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-27T06:55:57.698 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-27T06:55:57.699 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.10 2024-03-27T06:55:58.019 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:57 smithi156 systemd[1]: Stopping Ceph osd.10 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:55:58.019 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:57 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:57.777+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:58.019 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:57 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:57.777+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:58.019 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:57 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:57.777+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:58.019 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:57 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:57.777+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:58.019 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:57 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:57.777+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:58.019 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:57 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:57.777+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:58.019 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:57 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:57.777+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:55:58.020 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:58 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:58.022+0000 7fca31760700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:55:58.020 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:58 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:58.022+0000 7fca31760700 -1 osd.10 103 *** Got signal Terminated *** 2024-03-27T06:55:58.020 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:58 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:58.022+0000 7fca31760700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:55:58.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:58 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:58.245+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:58.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:58 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:58.246+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:58.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:58 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:58.246+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:58.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:58 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:58.246+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:58.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:58 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:58.246+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:58.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:58 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:58.246+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:55:59.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:58 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:58.742+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:55:59.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:58 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:58.742+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:55:59.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:58 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:58.742+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:55:59.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:58 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:58.742+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:55:59.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:58 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:58.742+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:55:59.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:58 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:58.742+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:55:59.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:58 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:58.742+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:55:59.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:59 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:59.203+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:55:59.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:59 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:59.203+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:55:59.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:59 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:59.203+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:55:59.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:59 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:59.203+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:55:59.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:59 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:59.203+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:55:59.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:55:59 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:55:59.203+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:00.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:59 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:59.737+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:56:00.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:59 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:59.737+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:56:00.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:59 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:59.737+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:56:00.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:59 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:59.737+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:56:00.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:59 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:59.737+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:56:00.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:59 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:59.737+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:56:00.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:55:59 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:55:59.737+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:56:00.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:00 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:00.163+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:56:00.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:00 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:00.163+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:56:00.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:00 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:00.163+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:56:00.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:00 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:00.163+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:56:00.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:00 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:00.163+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:56:00.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:00 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:00.163+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:00.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:00 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:00.711+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:56:00.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:00 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:00.711+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:56:00.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:00 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:00.711+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:56:00.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:00 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:00.711+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:56:00.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:00 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:00.711+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:56:00.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:00 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:00.711+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:56:00.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:00 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:00.711+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:56:01.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:01 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:01.165+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:56:01.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:01 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:01.165+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:56:01.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:01 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:01.165+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:56:01.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:01 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:01.165+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:56:01.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:01 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:01.165+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:56:01.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:01 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:01.165+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:01.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:01 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:01.666+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:56:01.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:01 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:01.666+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:56:01.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:01 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:01.666+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:56:01.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:01 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:01.666+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:56:01.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:01 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:01.666+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:56:01.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:01 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:01.666+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:56:01.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:01 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:01.666+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:56:02.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:02.128+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:56:02.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:02.128+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:56:02.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:02.128+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:56:02.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:02.128+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:56:02.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:02.128+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:56:02.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:02 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:02.128+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:02.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:02 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:02.692+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.857262+0000 front 2024-03-27T06:54:41.857389+0000 (oldest deadline 2024-03-27T06:55:03.557426+0000) 2024-03-27T06:56:02.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:02 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:02.692+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:50.460025+0000 front 2024-03-27T06:54:50.460145+0000 (oldest deadline 2024-03-27T06:55:12.160148+0000) 2024-03-27T06:56:02.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:02 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:02.692+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:54.460948+0000 front 2024-03-27T06:54:54.461097+0000 (oldest deadline 2024-03-27T06:55:20.861360+0000) 2024-03-27T06:56:02.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:02 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:02.692+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-03-27T06:55:06.762318+0000 front 2024-03-27T06:55:06.762405+0000 (oldest deadline 2024-03-27T06:55:31.462276+0000) 2024-03-27T06:56:02.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:02 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:02.692+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:11.963221+0000 front 2024-03-27T06:55:11.963250+0000 (oldest deadline 2024-03-27T06:55:36.663310+0000) 2024-03-27T06:56:02.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:02 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:02.692+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.264900+0000 front 2024-03-27T06:55:21.265057+0000 (oldest deadline 2024-03-27T06:55:43.564524+0000) 2024-03-27T06:56:02.973 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:02 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10[61261]: 2024-03-27T06:56:02.692+0000 7fca29d4c700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:28.666146+0000 front 2024-03-27T06:55:28.666187+0000 (oldest deadline 2024-03-27T06:55:51.566183+0000) 2024-03-27T06:56:03.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:03 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:03.088+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:56:03.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:03 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:03.088+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:56:03.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:03 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:03.088+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:56:03.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:03 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:03.088+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:56:03.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:03 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:03.088+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:56:03.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:03 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:03.088+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:03.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:03 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:03.088+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-27T06:55:38.329170+0000 front 2024-03-27T06:55:38.329239+0000 (oldest deadline 2024-03-27T06:56:02.429000+0000) 2024-03-27T06:56:03.642 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:03 smithi156 podman[73078]: 2024-03-27 06:56:03.357636822 +0000 UTC m=+5.464015382 container died c3a8c1c4632dd26a21babf70fc431f06f6450644df75daa9b47cf45be9cb97c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-27T06:56:03.643 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:03 smithi156 podman[73078]: 2024-03-27 06:56:03.648085783 +0000 UTC m=+5.754464338 container cleanup c3a8c1c4632dd26a21babf70fc431f06f6450644df75daa9b47cf45be9cb97c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-27T06:56:03.972 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:03 smithi156 bash[73078]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10 2024-03-27T06:56:03.972 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:03 smithi156 podman[73095]: 2024-03-27 06:56:03.890982382 +0000 UTC m=+0.529473797 container remove c3a8c1c4632dd26a21babf70fc431f06f6450644df75daa9b47cf45be9cb97c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-27T06:56:04.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:04 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:04.098+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:56:04.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:04 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:04.098+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:56:04.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:04 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:04.098+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:56:04.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:04 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:04.098+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:56:04.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:04 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:04.098+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:56:04.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:04 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:04.098+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:04.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:04 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:04.098+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-27T06:55:38.329170+0000 front 2024-03-27T06:55:38.329239+0000 (oldest deadline 2024-03-27T06:56:02.429000+0000) 2024-03-27T06:56:04.479 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:04 smithi156 podman[73186]: 2024-03-27 06:56:04.117265806 +0000 UTC m=+0.017827295 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:56:04.479 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:04 smithi156 podman[73186]: 2024-03-27 06:56:04.216870837 +0000 UTC m=+0.117432352 container create 87684d62ba3d493fddcf8ef2c762b36b7a4e731898094858ef62397dbdbf77b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240326.131558-155268c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:56:04.480 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:04 smithi156 podman[73186]: 2024-03-27 06:56:04.402293425 +0000 UTC m=+0.302854902 container init 87684d62ba3d493fddcf8ef2c762b36b7a4e731898094858ef62397dbdbf77b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-27T06:56:04.480 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:04 smithi156 podman[73186]: 2024-03-27 06:56:04.40539997 +0000 UTC m=+0.305961444 container start 87684d62ba3d493fddcf8ef2c762b36b7a4e731898094858ef62397dbdbf77b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True) 2024-03-27T06:56:04.480 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:04 smithi156 podman[73186]: 2024-03-27 06:56:04.477195039 +0000 UTC m=+0.377756520 container attach 87684d62ba3d493fddcf8ef2c762b36b7a4e731898094858ef62397dbdbf77b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-27T06:56:04.723 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:04 smithi156 podman[73186]: 2024-03-27 06:56:04.697634443 +0000 UTC m=+0.598195917 container died 87684d62ba3d493fddcf8ef2c762b36b7a4e731898094858ef62397dbdbf77b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True) 2024-03-27T06:56:05.216 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.10.service' 2024-03-27T06:56:05.222 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:05 smithi156 podman[73212]: 2024-03-27 06:56:05.197234065 +0000 UTC m=+0.490365914 container remove 87684d62ba3d493fddcf8ef2c762b36b7a4e731898094858ef62397dbdbf77b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-03-27T06:56:05.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:05 smithi156 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.10.service: Deactivated successfully. 2024-03-27T06:56:05.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:05 smithi156 systemd[1]: Stopped Ceph osd.10 for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:56:05.223 INFO:journalctl@ceph.osd.10.smithi156.stdout:Mar 27 06:56:05 smithi156 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.10.service: Consumed 24.099s CPU time. 2024-03-27T06:56:05.257 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:05.257 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-27T06:56:05.257 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-27T06:56:05.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.11 2024-03-27T06:56:05.302 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:05 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:05.056+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:56:05.303 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:05 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:05.056+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:56:05.303 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:05 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:05.056+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:56:05.303 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:05 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:05.056+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:56:05.303 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:05 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:05.056+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:56:05.303 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:05 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:05.056+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:05.303 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:05 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:05.056+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-27T06:55:38.329170+0000 front 2024-03-27T06:55:38.329239+0000 (oldest deadline 2024-03-27T06:56:02.429000+0000) 2024-03-27T06:56:05.303 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:05 smithi170 systemd[1]: Stopping Ceph osd.11 for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:05.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:05 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:05.563+0000 7f60da3cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:05.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:05 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:05.563+0000 7f60da3cc700 -1 osd.11 103 *** Got signal Terminated *** 2024-03-27T06:56:05.844 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:05 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:05.563+0000 7f60da3cc700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-27T06:56:06.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:06 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:06.094+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:56:06.344 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:06 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:06.094+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:56:06.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:06 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:06.094+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:56:06.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:06 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:06.094+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:56:06.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:06 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:06.094+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:56:06.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:06 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:06.094+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:06.345 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:06 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:06.094+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-27T06:55:38.329170+0000 front 2024-03-27T06:55:38.329239+0000 (oldest deadline 2024-03-27T06:56:02.429000+0000) 2024-03-27T06:56:07.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:07 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:07.132+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:56:07.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:07 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:07.132+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:56:07.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:07 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:07.132+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:56:07.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:07 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:07.132+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:56:07.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:07 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:07.132+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:56:07.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:07 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:07.132+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:07.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:07 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:07.132+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-27T06:55:38.329170+0000 front 2024-03-27T06:55:38.329239+0000 (oldest deadline 2024-03-27T06:56:02.429000+0000) 2024-03-27T06:56:08.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:08 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:08.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:56:08.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:08 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:08.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:56:08.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:08 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:08.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:56:08.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:08 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:08.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:56:08.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:08 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:08.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:56:08.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:08 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:08.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:08.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:08 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:08.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-27T06:55:38.329170+0000 front 2024-03-27T06:55:38.329239+0000 (oldest deadline 2024-03-27T06:56:02.429000+0000) 2024-03-27T06:56:08.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:08 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:08.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6820 osd.8 since back 2024-03-27T06:55:42.429311+0000 front 2024-03-27T06:55:42.429352+0000 (oldest deadline 2024-03-27T06:56:07.729560+0000) 2024-03-27T06:56:09.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:09 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:09.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:56:09.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:09 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:09.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:56:09.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:09 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:09.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:56:09.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:09 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:09.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:56:09.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:09 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:09.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:56:09.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:09 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:09.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:09.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:09 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:09.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-27T06:55:38.329170+0000 front 2024-03-27T06:55:38.329239+0000 (oldest deadline 2024-03-27T06:56:02.429000+0000) 2024-03-27T06:56:09.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:09 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:09.111+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6820 osd.8 since back 2024-03-27T06:55:42.429311+0000 front 2024-03-27T06:55:42.429352+0000 (oldest deadline 2024-03-27T06:56:07.729560+0000) 2024-03-27T06:56:10.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:10 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:10.109+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-03-27T06:54:41.717350+0000 front 2024-03-27T06:54:41.717378+0000 (oldest deadline 2024-03-27T06:55:05.817536+0000) 2024-03-27T06:56:10.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:10 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:10.109+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6804 osd.1 since back 2024-03-27T06:54:48.718826+0000 front 2024-03-27T06:54:48.718713+0000 (oldest deadline 2024-03-27T06:55:11.519036+0000) 2024-03-27T06:56:10.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:10 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:10.109+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-27T06:54:56.820075+0000 front 2024-03-27T06:54:56.820225+0000 (oldest deadline 2024-03-27T06:55:22.120213+0000) 2024-03-27T06:56:10.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:10 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:10.109+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6812 osd.4 since back 2024-03-27T06:55:13.623645+0000 front 2024-03-27T06:55:13.623759+0000 (oldest deadline 2024-03-27T06:55:35.923615+0000) 2024-03-27T06:56:10.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:10 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:10.109+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-27T06:55:21.625737+0000 front 2024-03-27T06:55:21.625787+0000 (oldest deadline 2024-03-27T06:55:45.125833+0000) 2024-03-27T06:56:10.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:10 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:10.109+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-03-27T06:55:25.126428+0000 front 2024-03-27T06:55:25.126414+0000 (oldest deadline 2024-03-27T06:55:54.526680+0000) 2024-03-27T06:56:10.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:10 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:10.109+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.156:6820 osd.7 since back 2024-03-27T06:55:38.329170+0000 front 2024-03-27T06:55:38.329239+0000 (oldest deadline 2024-03-27T06:56:02.429000+0000) 2024-03-27T06:56:10.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:10 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11[62394]: 2024-03-27T06:56:10.109+0000 7f60d69c0700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.170:6820 osd.8 since back 2024-03-27T06:55:42.429311+0000 front 2024-03-27T06:55:42.429352+0000 (oldest deadline 2024-03-27T06:56:07.729560+0000) 2024-03-27T06:56:11.341 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:10 smithi170 podman[74473]: 2024-03-27 06:56:10.996176967 +0000 UTC m=+5.543450106 container died e90426f74279a12850606b08c5f65c18b2fe00f31da20715e0b994e2742900f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11, ceph=True, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T06:56:11.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:11 smithi170 podman[74473]: 2024-03-27 06:56:11.346837217 +0000 UTC m=+5.894110333 container cleanup e90426f74279a12850606b08c5f65c18b2fe00f31da20715e0b994e2742900f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-03-27T06:56:11.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:11 smithi170 bash[74473]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11 2024-03-27T06:56:11.595 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:11 smithi170 podman[74490]: 2024-03-27 06:56:11.51474124 +0000 UTC m=+0.513136062 container remove e90426f74279a12850606b08c5f65c18b2fe00f31da20715e0b994e2742900f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-03-27T06:56:12.187 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:11 smithi170 podman[74585]: 2024-03-27 06:56:11.898855101 +0000 UTC m=+0.102509089 container create 7fa6fa2387e7d7c0ac30b1b28759d9b3f6597d4e788a8e56f82bfa9afaf2d287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:56:12.187 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:11 smithi170 podman[74585]: 2024-03-27 06:56:11.813994283 +0000 UTC m=+0.017648241 image pull b379bbd8da6b5d0fa9a3523941efac213b6815e1cbf05c6507954f0a3fe7a801 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772 2024-03-27T06:56:12.187 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:12 smithi170 podman[74585]: 2024-03-27 06:56:12.117939373 +0000 UTC m=+0.321593327 container init 7fa6fa2387e7d7c0ac30b1b28759d9b3f6597d4e788a8e56f82bfa9afaf2d287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-27T06:56:12.187 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:12 smithi170 podman[74585]: 2024-03-27 06:56:12.121534574 +0000 UTC m=+0.325188508 container start 7fa6fa2387e7d7c0ac30b1b28759d9b3f6597d4e788a8e56f82bfa9afaf2d287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-27T06:56:12.187 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:12 smithi170 podman[74585]: 2024-03-27 06:56:12.192553477 +0000 UTC m=+0.396207433 container attach 7fa6fa2387e7d7c0ac30b1b28759d9b3f6597d4e788a8e56f82bfa9afaf2d287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-27T06:56:12.594 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 27 06:56:12 smithi170 podman[74585]: 2024-03-27 06:56:12.406732689 +0000 UTC m=+0.610386631 container died 7fa6fa2387e7d7c0ac30b1b28759d9b3f6597d4e788a8e56f82bfa9afaf2d287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-27T06:56:12.953 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@osd.11.service' 2024-03-27T06:56:12.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:12.993 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-27T06:56:12.993 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-27T06:56:12.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.a 2024-03-27T06:56:13.308 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:56:13 smithi016 systemd[1]: Stopping Ceph mds.a for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:13.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:56:13 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a[74037]: 2024-03-27T06:56:13.312+0000 7f07aa523700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:13.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:56:13 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a[74037]: 2024-03-27T06:56:13.312+0000 7f07aa523700 -1 mds.a *** got signal Terminated *** 2024-03-27T06:56:17.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:56:17 smithi016 podman[95136]: 2024-03-27 06:56:17.322222064 +0000 UTC m=+4.123205045 container died e4e0d4d26dd0ac212c08730962f7aaa5ed0c54bda9424ab07d82ac836c7e6c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-27T06:56:18.041 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:56:17 smithi016 podman[95136]: 2024-03-27 06:56:17.795449281 +0000 UTC m=+4.596432264 container cleanup e4e0d4d26dd0ac212c08730962f7aaa5ed0c54bda9424ab07d82ac836c7e6c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:56:18.041 INFO:journalctl@ceph.mds.a.smithi016.stdout:Mar 27 06:56:17 smithi016 bash[95136]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-a 2024-03-27T06:56:18.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.a.service' 2024-03-27T06:56:18.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:18.208 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-27T06:56:18.208 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-27T06:56:18.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.d 2024-03-27T06:56:18.362 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:56:18 smithi016 systemd[1]: Stopping Ceph mds.d for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:18.862 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:56:18 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-d[73081]: 2024-03-27T06:56:18.532+0000 7f1056523700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:18.862 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:56:18 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-d[73081]: 2024-03-27T06:56:18.532+0000 7f1056523700 -1 mds.d *** got signal Terminated *** 2024-03-27T06:56:21.927 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:56:21 smithi016 podman[95255]: 2024-03-27 06:56:21.654527 +0000 UTC m=+3.216015980 container died 0b99dee229916d3d4bc6dcb97114cc650b924318032339a8e2ac18c567cd8704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-d, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-03-27T06:56:22.320 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:56:22 smithi016 podman[95255]: 2024-03-27 06:56:22.032519588 +0000 UTC m=+3.594008566 container cleanup 0b99dee229916d3d4bc6dcb97114cc650b924318032339a8e2ac18c567cd8704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-27T06:56:22.320 INFO:journalctl@ceph.mds.d.smithi016.stdout:Mar 27 06:56:22 smithi016 bash[95255]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-d 2024-03-27T06:56:22.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.d.service' 2024-03-27T06:56:22.493 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:22.493 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-27T06:56:22.493 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-27T06:56:22.493 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.g 2024-03-27T06:56:22.862 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:56:22 smithi016 systemd[1]: Stopping Ceph mds.g for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:22.862 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:56:22 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g[73676]: 2024-03-27T06:56:22.827+0000 7f5b4f479700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:22.862 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:56:22 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g[73676]: 2024-03-27T06:56:22.827+0000 7f5b4f479700 -1 mds.g *** got signal Terminated *** 2024-03-27T06:56:26.481 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:56:26 smithi016 podman[95374]: 2024-03-27 06:56:26.211534877 +0000 UTC m=+3.491162236 container died c254894f86e3a3e404337de3a86282075f0312e8cbe83d9affdc08f3cb8994ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T06:56:26.765 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:56:26 smithi016 podman[95374]: 2024-03-27 06:56:26.561154975 +0000 UTC m=+3.840782330 container cleanup c254894f86e3a3e404337de3a86282075f0312e8cbe83d9affdc08f3cb8994ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-27T06:56:26.765 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:56:26 smithi016 bash[95374]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g 2024-03-27T06:56:26.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.g.service' 2024-03-27T06:56:27.016 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:56:26 smithi016 podman[95387]: 2024-03-27 06:56:26.771081046 +0000 UTC m=+0.553260623 container remove c254894f86e3a3e404337de3a86282075f0312e8cbe83d9affdc08f3cb8994ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-03-27T06:56:27.016 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:56:26 smithi016 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.g.service: Deactivated successfully. 2024-03-27T06:56:27.016 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:56:26 smithi016 systemd[1]: Stopped Ceph mds.g for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:56:27.017 INFO:journalctl@ceph.mds.g.smithi016.stdout:Mar 27 06:56:26 smithi016 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.g.service: Consumed 2.100s CPU time. 2024-03-27T06:56:27.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:27.028 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-27T06:56:27.028 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-27T06:56:27.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.j 2024-03-27T06:56:27.343 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:56:27 smithi016 systemd[1]: Stopping Ceph mds.j for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:27.612 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:56:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-j[72685]: 2024-03-27T06:56:27.348+0000 7f5816180700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:27.612 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:56:27 smithi016 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-j[72685]: 2024-03-27T06:56:27.348+0000 7f5816180700 -1 mds.j *** got signal Terminated *** 2024-03-27T06:56:30.936 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:56:30 smithi016 podman[95495]: 2024-03-27 06:56:30.682129261 +0000 UTC m=+3.435313652 container died 79533e0080189b483e8347d8e008934ad919373d66900afcd287076a51ee0d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-j, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, RELEASE=wip-rishabh-testing-20240326.131558-155268c) 2024-03-27T06:56:31.302 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:56:31 smithi016 podman[95495]: 2024-03-27 06:56:31.040344967 +0000 UTC m=+3.793529370 container cleanup 79533e0080189b483e8347d8e008934ad919373d66900afcd287076a51ee0d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-j, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:56:31.302 INFO:journalctl@ceph.mds.j.smithi016.stdout:Mar 27 06:56:31 smithi016 bash[95495]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-j 2024-03-27T06:56:31.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.j.service' 2024-03-27T06:56:31.510 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:31.510 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-27T06:56:31.510 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-27T06:56:31.511 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.b 2024-03-27T06:56:31.800 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:56:31 smithi156 systemd[1]: Stopping Ceph mds.b for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:32.222 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:56:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b[66776]: 2024-03-27T06:56:31.805+0000 7f0e2cae6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:32.222 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:56:31 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b[66776]: 2024-03-27T06:56:31.805+0000 7f0e2cae6700 -1 mds.b *** got signal Terminated *** 2024-03-27T06:56:34.473 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:56:34 smithi156 podman[73304]: 2024-03-27 06:56:34.223424906 +0000 UTC m=+2.512739077 container died c90d68b0db85fa4ec66f5ad51aa90ed05759053e97e0142914af734b59a4c987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, CEPH_POINT_RELEASE=) 2024-03-27T06:56:34.734 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:56:34 smithi156 podman[73304]: 2024-03-27 06:56:34.563533446 +0000 UTC m=+2.852847603 container cleanup c90d68b0db85fa4ec66f5ad51aa90ed05759053e97e0142914af734b59a4c987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:56:34.734 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:56:34 smithi156 bash[73304]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b 2024-03-27T06:56:34.960 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.b.service' 2024-03-27T06:56:34.985 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:56:34 smithi156 podman[73318]: 2024-03-27 06:56:34.739524749 +0000 UTC m=+0.511419058 container remove c90d68b0db85fa4ec66f5ad51aa90ed05759053e97e0142914af734b59a4c987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=) 2024-03-27T06:56:34.985 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:56:34 smithi156 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.b.service: Deactivated successfully. 2024-03-27T06:56:34.986 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:56:34 smithi156 systemd[1]: Stopped Ceph mds.b for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:56:34.986 INFO:journalctl@ceph.mds.b.smithi156.stdout:Mar 27 06:56:34 smithi156 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.b.service: Consumed 6min 5.215s CPU time. 2024-03-27T06:56:34.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:34.997 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-27T06:56:34.997 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-27T06:56:34.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.e 2024-03-27T06:56:35.312 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:56:35 smithi156 systemd[1]: Stopping Ceph mds.e for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:35.723 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:56:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e[66157]: 2024-03-27T06:56:35.316+0000 7f459be24700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:35.724 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:56:35 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e[66157]: 2024-03-27T06:56:35.316+0000 7f459be24700 -1 mds.e *** got signal Terminated *** 2024-03-27T06:56:38.473 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:56:38 smithi156 podman[73426]: 2024-03-27 06:56:38.216609948 +0000 UTC m=+3.000658957 container died 0979e0d68b54e5831aa9b6421a58e83bd0f8483d00eb017bf1847273b32765ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-27T06:56:38.838 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:56:38 smithi156 podman[73426]: 2024-03-27 06:56:38.64209729 +0000 UTC m=+3.426146292 container cleanup 0979e0d68b54e5831aa9b6421a58e83bd0f8483d00eb017bf1847273b32765ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:56:38.838 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:56:38 smithi156 bash[73426]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e 2024-03-27T06:56:39.070 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.e.service' 2024-03-27T06:56:39.095 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:56:38 smithi156 podman[73452]: 2024-03-27 06:56:38.84326001 +0000 UTC m=+0.617580554 container remove 0979e0d68b54e5831aa9b6421a58e83bd0f8483d00eb017bf1847273b32765ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-03-27T06:56:39.095 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:56:39 smithi156 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.e.service: Deactivated successfully. 2024-03-27T06:56:39.095 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:56:39 smithi156 systemd[1]: Stopped Ceph mds.e for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:56:39.095 INFO:journalctl@ceph.mds.e.smithi156.stdout:Mar 27 06:56:39 smithi156 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.e.service: Consumed 4min 19.865s CPU time. 2024-03-27T06:56:39.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:39.109 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-27T06:56:39.109 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-27T06:56:39.109 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.h 2024-03-27T06:56:39.424 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:56:39 smithi156 systemd[1]: Stopping Ceph mds.h for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:39.722 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:56:39 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:56:39.429+0000 7f9cf5d53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:39.723 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:56:39 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h[66465]: 2024-03-27T06:56:39.429+0000 7f9cf5d53700 -1 mds.h *** got signal Terminated *** 2024-03-27T06:56:42.472 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:56:42 smithi156 podman[73559]: 2024-03-27 06:56:42.217012162 +0000 UTC m=+2.888852031 container died a728dab8dfcd064aa5f983e988b5aa013a26925075466bbbaf2f8f3cde73c49f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:56:42.849 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:56:42 smithi156 podman[73559]: 2024-03-27 06:56:42.578445397 +0000 UTC m=+3.250285245 container cleanup a728dab8dfcd064aa5f983e988b5aa013a26925075466bbbaf2f8f3cde73c49f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-27T06:56:42.849 INFO:journalctl@ceph.mds.h.smithi156.stdout:Mar 27 06:56:42 smithi156 bash[73559]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-h 2024-03-27T06:56:42.992 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.h.service' 2024-03-27T06:56:43.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:43.028 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-27T06:56:43.028 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-27T06:56:43.028 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.k 2024-03-27T06:56:43.472 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:56:43 smithi156 systemd[1]: Stopping Ceph mds.k for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:43.473 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:56:43 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k[65834]: 2024-03-27T06:56:43.357+0000 7fb2edca7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:43.473 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:56:43 smithi156 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k[65834]: 2024-03-27T06:56:43.357+0000 7fb2edca7700 -1 mds.k *** got signal Terminated *** 2024-03-27T06:56:46.476 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:56:46 smithi156 podman[73680]: 2024-03-27 06:56:46.216586606 +0000 UTC m=+2.968507804 container died c3b70b9c62bfca3cc25d99b3ea3a2ac694d7ea5c6eb3407306c065a0a680bafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e) 2024-03-27T06:56:46.907 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:56:46 smithi156 podman[73680]: 2024-03-27 06:56:46.557680389 +0000 UTC m=+3.309601644 container cleanup c3b70b9c62bfca3cc25d99b3ea3a2ac694d7ea5c6eb3407306c065a0a680bafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326) 2024-03-27T06:56:46.907 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:56:46 smithi156 bash[73680]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k 2024-03-27T06:56:46.907 INFO:journalctl@ceph.mds.k.smithi156.stdout:Mar 27 06:56:46 smithi156 podman[73694]: 2024-03-27 06:56:46.716697762 +0000 UTC m=+0.494835217 container remove c3b70b9c62bfca3cc25d99b3ea3a2ac694d7ea5c6eb3407306c065a0a680bafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-k, RELEASE=wip-rishabh-testing-20240326.131558-155268c, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-27T06:56:46.919 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.k.service' 2024-03-27T06:56:46.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:46.956 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-27T06:56:46.956 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-27T06:56:46.957 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.c 2024-03-27T06:56:47.344 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:56:47 smithi170 systemd[1]: Stopping Ceph mds.c for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:47.844 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:56:47 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c[67568]: 2024-03-27T06:56:47.519+0000 7f4c0f4bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:47.844 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:56:47 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c[67568]: 2024-03-27T06:56:47.519+0000 7f4c0f4bb700 -1 mds.c *** got signal Terminated *** 2024-03-27T06:56:50.465 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:56:50 smithi170 podman[74688]: 2024-03-27 06:56:50.212405654 +0000 UTC m=+3.052939129 container died 45944325da2566cbf92429140bd28a5689db9c73287f254a8f9c73f73f617eba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-27T06:56:50.823 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:56:50 smithi170 podman[74688]: 2024-03-27 06:56:50.569988933 +0000 UTC m=+3.410522365 container cleanup 45944325da2566cbf92429140bd28a5689db9c73287f254a8f9c73f73f617eba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-27T06:56:50.824 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 27 06:56:50 smithi170 bash[74688]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-c 2024-03-27T06:56:50.937 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.c.service' 2024-03-27T06:56:50.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:50.982 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-27T06:56:50.982 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-27T06:56:50.982 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.f 2024-03-27T06:56:51.094 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:56:51 smithi170 systemd[1]: Stopping Ceph mds.f for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:51.594 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:56:51 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[71016]: 2024-03-27T06:56:51.322+0000 7fde8637b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:51.594 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:56:51 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f[71016]: 2024-03-27T06:56:51.322+0000 7fde8637b700 -1 mds.f *** got signal Terminated *** 2024-03-27T06:56:54.544 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:56:54 smithi170 podman[74822]: 2024-03-27 06:56:54.226254559 +0000 UTC m=+3.025776326 container died 0bba2c438fb3e88e7ece84b5cdf148ddd21dca12f63e129e82f06a7ac69a5582 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True) 2024-03-27T06:56:54.846 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:56:54 smithi170 podman[74822]: 2024-03-27 06:56:54.639930094 +0000 UTC m=+3.439451853 container cleanup 0bba2c438fb3e88e7ece84b5cdf148ddd21dca12f63e129e82f06a7ac69a5582 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240326.131558-155268c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-03-27T06:56:54.846 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:56:54 smithi170 bash[74822]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f 2024-03-27T06:56:55.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.f.service' 2024-03-27T06:56:55.138 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:56:54 smithi170 podman[74836]: 2024-03-27 06:56:54.883326086 +0000 UTC m=+0.647874649 container remove 0bba2c438fb3e88e7ece84b5cdf148ddd21dca12f63e129e82f06a7ac69a5582 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2) 2024-03-27T06:56:55.138 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:56:55 smithi170 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.f.service: Deactivated successfully. 2024-03-27T06:56:55.138 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:56:55 smithi170 systemd[1]: Stopped Ceph mds.f for c9095bc4-ec03-11ee-95d0-87774f69a715. 2024-03-27T06:56:55.138 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 27 06:56:55 smithi170 systemd[1]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.f.service: Consumed 8.999s CPU time. 2024-03-27T06:56:55.150 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:55.150 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-27T06:56:55.150 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-27T06:56:55.150 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.i 2024-03-27T06:56:55.480 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:56:55 smithi170 systemd[1]: Stopping Ceph mds.i for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:55.844 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:56:55 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: 2024-03-27T06:56:55.484+0000 7f259e010700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:55.844 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:56:55 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i[67878]: 2024-03-27T06:56:55.484+0000 7f259e010700 -1 mds.i *** got signal Terminated *** 2024-03-27T06:56:58.547 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:56:58 smithi170 podman[74942]: 2024-03-27 06:56:58.23145665 +0000 UTC m=+2.859268726 container died 2ae822f5ef10d9de5ea999c4c1f0895c97b813834a2b6f3471f400b21fe21494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-27T06:56:58.844 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:56:58 smithi170 podman[74942]: 2024-03-27 06:56:58.635043498 +0000 UTC m=+3.262855566 container cleanup 2ae822f5ef10d9de5ea999c4c1f0895c97b813834a2b6f3471f400b21fe21494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_BRANCH=HEAD) 2024-03-27T06:56:58.844 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 27 06:56:58 smithi170 bash[74942]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-i 2024-03-27T06:56:59.057 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.i.service' 2024-03-27T06:56:59.091 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:56:59.092 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-27T06:56:59.092 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-27T06:56:59.092 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.l 2024-03-27T06:56:59.509 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:56:59 smithi170 systemd[1]: Stopping Ceph mds.l for c9095bc4-ec03-11ee-95d0-87774f69a715... 2024-03-27T06:56:59.844 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:56:59 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l[66934]: 2024-03-27T06:56:59.513+0000 7f14ebc3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-27T06:56:59.844 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:56:59 smithi170 ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l[66934]: 2024-03-27T06:56:59.513+0000 7f14ebc3a700 -1 mds.l *** got signal Terminated *** 2024-03-27T06:57:02.501 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:57:02 smithi170 podman[75064]: 2024-03-27 06:57:02.217796106 +0000 UTC m=+2.908601070 container died 03bcfff9364f673b293a30f98ec1571637e19d4b3047bc809b3d4eaa8eb7a047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, org.label-schema.build-date=20240326, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240326.131558-155268c, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-27T06:57:02.809 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:57:02 smithi170 podman[75064]: 2024-03-27 06:57:02.605567563 +0000 UTC m=+3.296372518 container cleanup 03bcfff9364f673b293a30f98ec1571637e19d4b3047bc809b3d4eaa8eb7a047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:45ddfdb8dd49eba69c4398decee692ae8ceca4f9925f375b2be751a8dea8d772, name=ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l, GIT_COMMIT=d7c7095173c3793ef2ed0fc814eebfb88de11a2e, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240326.131558-155268c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-03-27T06:57:02.810 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 27 06:57:02 smithi170 bash[75064]: ceph-c9095bc4-ec03-11ee-95d0-87774f69a715-mds-l 2024-03-27T06:57:02.943 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9095bc4-ec03-11ee-95d0-87774f69a715@mds.l.service' 2024-03-27T06:57:02.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-27T06:57:02.979 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-27T06:57:02.979 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-27T06:57:03.179 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:57:05.572 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-27T06:57:05.764 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:57:08.337 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-27T06:57:08.528 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:57:10.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T06:57:10.987 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T06:57:11.022 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-27T06:57:11.050 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-27T06:57:11.053 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597/remote/smithi016/crash 2024-03-27T06:57:11.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/crash -- . 2024-03-27T06:57:11.084 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-27T06:57:11.084 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-03-27T06:57:11.087 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597/remote/smithi156/crash 2024-03-27T06:57:11.088 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/crash -- . 2024-03-27T06:57:11.128 INFO:teuthology.orchestra.run.smithi156.stderr:tar: /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-27T06:57:11.128 INFO:teuthology.orchestra.run.smithi156.stderr:tar: Error is not recoverable: exiting now 2024-03-27T06:57:11.130 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597/remote/smithi170/crash 2024-03-27T06:57:11.131 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/crash -- . 2024-03-27T06:57:11.177 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-27T06:57:11.177 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2024-03-27T06:57:11.178 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-27T06:57:11.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-27T06:57:11.215 INFO:teuthology.orchestra.run.smithi016.stdout:2024-03-27T06:45:20.972292+0000 mds.c (mds.3) 2 : cluster [WRN] Scrub error on inode 0x10000000e83 (/volumes/qa/sv_1/1070fb6a-f0e5-4cc6-a30e-e73dd004606a/client.0/tmp/untar_tar/linux-4.0.5/Documentation/mips/00-INDEX) see mds.c log and `damage ls` output for details 2024-03-27T06:57:11.216 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-27T06:57:11.216 INFO:tasks.cephadm:Compressing logs... 2024-03-27T06:57:11.216 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-27T06:57:11.259 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-27T06:57:11.261 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-27T06:57:11.287 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-27T06:57:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:: No such file or directory 2024-03-27T06:57:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.6.log 2024-03-27T06:57:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mon.a.log 2024-03-27T06:57:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-27T06:57:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.g.log 2024-03-27T06:57:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.j.log 2024-03-27T06:57:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.audit.log: /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-27T06:57:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mgr.x.log 2024-03-27T06:57:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-27T06:57:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mgr.x.log: 90.3% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-27T06:57:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-27T06:57:11.299 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.0.log: 94.6% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.j.log.gz 2024-03-27T06:57:11.299 INFO:teuthology.orchestra.run.smithi016.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-27T06:57:11.300 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-03-27T06:57:11.300 INFO:teuthology.orchestra.run.smithi156.stderr: -- /var/log/ceph/cephadm.log 2024-03-27T06:57:11.300 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.10.log 2024-03-27T06:57:11.300 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-27T06:57:11.300 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.b.log 2024-03-27T06:57:11.300 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.10.log: /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-27T06:57:11.300 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-27T06:57:11.301 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.b.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-27T06:57:11.301 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.k.log 2024-03-27T06:57:11.301 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.h.log 2024-03-27T06:57:11.301 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.1.log: /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.7.log 2024-03-27T06:57:11.301 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.e.log 2024-03-27T06:57:11.301 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.7.log: 90.4% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-27T06:57:11.301 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mgr.y.log 2024-03-27T06:57:11.304 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-03-27T06:57:11.304 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.l.log 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.i.log 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.l.log: /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.11.log 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mgr.z.log 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.8.log 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mgr.z.log: /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.f.log 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.f.log: 90.5% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.log 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-27T06:57:11.305 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-27T06:57:11.306 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.log: 87.9% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-27T06:57:11.306 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.c.log 2024-03-27T06:57:11.306 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.2.log: 91.9% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mgr.z.log.gz 2024-03-27T06:57:11.306 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-27T06:57:11.307 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-27T06:57:11.308 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.log 2024-03-27T06:57:11.309 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-27T06:57:11.309 INFO:teuthology.orchestra.run.smithi016.stderr: 84.2% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-27T06:57:11.309 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.9.log 2024-03-27T06:57:11.310 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-volume.log: 87.8% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-27T06:57:11.311 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.d.log 2024-03-27T06:57:11.318 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mgr.y.log: 91.8%gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.log 2024-03-27T06:57:11.318 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-27T06:57:11.319 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mon.c.log 2024-03-27T06:57:11.321 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.a.log 2024-03-27T06:57:11.323 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-27T06:57:11.324 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-27T06:57:11.324 INFO:teuthology.orchestra.run.smithi156.stderr: -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mgr.y.log.gz 2024-03-27T06:57:11.326 INFO:teuthology.orchestra.run.smithi016.stderr: 94.5% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.d.log.gz 2024-03-27T06:57:11.338 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mon.b.log 2024-03-27T06:57:11.340 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-volume.log: 87.9% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-27T06:57:11.340 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-27T06:57:11.364 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-27T06:57:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.a.log: 93.8% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.g.log.gz 2024-03-27T06:57:11.455 INFO:teuthology.orchestra.run.smithi156.stderr: 94.4% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.k.log.gz 2024-03-27T06:57:11.460 INFO:teuthology.orchestra.run.smithi016.stderr: 94.0% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.a.log.gz 2024-03-27T06:57:11.571 INFO:teuthology.orchestra.run.smithi016.stderr: 90.3% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mgr.x.log.gz 2024-03-27T06:57:11.846 INFO:teuthology.orchestra.run.smithi156.stderr: 92.0% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mon.b.log.gz 2024-03-27T06:57:11.946 INFO:teuthology.orchestra.run.smithi170.stderr: 91.9% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mon.c.log.gz 2024-03-27T06:57:12.076 INFO:teuthology.orchestra.run.smithi156.stderr: 93.8% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.e.log.gz 2024-03-27T06:57:12.211 INFO:teuthology.orchestra.run.smithi016.stderr: 90.4% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mon.a.log.gz 2024-03-27T06:57:12.565 INFO:teuthology.orchestra.run.smithi170.stderr: 95.2% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.l.log.gz 2024-03-27T06:57:12.583 INFO:teuthology.orchestra.run.smithi170.stderr: 93.8% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.c.log.gz 2024-03-27T06:57:12.597 INFO:teuthology.orchestra.run.smithi156.stderr: 93.8% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.h.log.gz 2024-03-27T06:57:14.314 INFO:teuthology.orchestra.run.smithi016.stderr: 92.0% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.9.log.gz 2024-03-27T06:57:14.478 INFO:teuthology.orchestra.run.smithi170.stderr: 95.1% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.f.log.gz 2024-03-27T06:57:15.269 INFO:teuthology.orchestra.run.smithi016.stderr: 91.6% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.6.log.gz 2024-03-27T06:57:15.530 INFO:teuthology.orchestra.run.smithi170.stderr: 91.7% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.11.log.gz 2024-03-27T06:57:15.632 INFO:teuthology.orchestra.run.smithi170.stderr: 91.7% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-27T06:57:15.839 INFO:teuthology.orchestra.run.smithi156.stderr: 92.1% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.10.log.gz 2024-03-27T06:57:15.862 INFO:teuthology.orchestra.run.smithi016.stderr: 91.7% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-27T06:57:16.071 INFO:teuthology.orchestra.run.smithi156.stderr: 92.0% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-27T06:57:16.207 INFO:teuthology.orchestra.run.smithi170.stderr: 91.8% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.8.log.gz 2024-03-27T06:57:16.481 INFO:teuthology.orchestra.run.smithi156.stderr: 92.2% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-27T06:57:16.567 INFO:teuthology.orchestra.run.smithi016.stderr: 91.9% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-27T06:57:16.569 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-03-27T06:57:16.569 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m5.296s 2024-03-27T06:57:16.569 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m17.746s 2024-03-27T06:57:16.569 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m0.563s 2024-03-27T06:57:17.019 INFO:teuthology.orchestra.run.smithi156.stderr: 91.7% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.7.log.gz 2024-03-27T06:57:17.164 INFO:teuthology.orchestra.run.smithi170.stderr: 91.7% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-27T06:57:23.451 INFO:teuthology.orchestra.run.smithi170.stderr: 93.3% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.i.log.gz 2024-03-27T06:57:23.452 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-03-27T06:57:23.453 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m12.177s 2024-03-27T06:57:23.453 INFO:teuthology.orchestra.run.smithi170.stderr:user 0m36.117s 2024-03-27T06:57:23.453 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m1.131s 2024-03-27T06:57:39.293 INFO:teuthology.orchestra.run.smithi156.stderr: 93.0% -- replaced with /var/log/ceph/c9095bc4-ec03-11ee-95d0-87774f69a715/ceph-mds.b.log.gz 2024-03-27T06:57:39.295 INFO:teuthology.orchestra.run.smithi156.stderr: 2024-03-27T06:57:39.295 INFO:teuthology.orchestra.run.smithi156.stderr:real 0m28.019s 2024-03-27T06:57:39.295 INFO:teuthology.orchestra.run.smithi156.stderr:user 0m49.317s 2024-03-27T06:57:39.295 INFO:teuthology.orchestra.run.smithi156.stderr:sys 0m1.508s 2024-03-27T06:57:39.295 INFO:tasks.cephadm:Archiving logs... 2024-03-27T06:57:39.296 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597/remote/smithi016/log 2024-03-27T06:57:39.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-27T06:57:42.230 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597/remote/smithi156/log 2024-03-27T06:57:42.231 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-27T06:57:49.607 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597/remote/smithi170/log 2024-03-27T06:57:49.608 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-27T06:57:54.596 INFO:tasks.cephadm:Removing cluster... 2024-03-27T06:57:54.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 --force 2024-03-27T06:57:54.798 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:57:55.064 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 --force 2024-03-27T06:57:55.261 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:57:55.578 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9095bc4-ec03-11ee-95d0-87774f69a715 --force 2024-03-27T06:57:55.765 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: c9095bc4-ec03-11ee-95d0-87774f69a715 2024-03-27T06:57:56.061 INFO:tasks.cephadm:Removing cephadm ... 2024-03-27T06:57:56.061 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-27T06:57:56.080 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-27T06:57:56.098 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-27T06:57:56.114 INFO:tasks.cephadm:Teardown complete 2024-03-27T06:57:56.114 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-27T06:57:56.125 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-27T06:57:56.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-27T06:57:56.128 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-27T06:57:56.140 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-27T06:57:56.255 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-27T06:57:56.266 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-27T06:57:56.267 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T06:57:56.270 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T06:57:56.273 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-27T06:57:56.283 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-03-27T06:57:56.285 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2024-03-27T06:57:56.287 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-03-27T06:57:56.287 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T06:57:56.288 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-03-27T06:57:56.288 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 23 -111us[ -63us] +/- 95ms 2024-03-27T06:57:56.288 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv02.front.sepia.ceph.com 3 8 377 23 -827us[ -779us] +/- 77ms 2024-03-27T06:57:56.288 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +1041us[+1089us] +/- 64ms 2024-03-27T06:57:56.288 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:57:56.290 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T06:57:56.290 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2024-03-27T06:57:56.291 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 -158us[ -134us] +/- 95ms 2024-03-27T06:57:56.291 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -883us[ -859us] +/- 77ms 2024-03-27T06:57:56.291 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +1018us[+1018us] +/- 64ms 2024-03-27T06:57:56.291 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:57:56.292 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-27T06:57:56.292 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-03-27T06:57:56.292 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 24 -181us[ -181us] +/- 95ms 2024-03-27T06:57:56.292 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -900us[ -866us] +/- 77ms 2024-03-27T06:57:56.292 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +988us[ +988us] +/- 64ms 2024-03-27T06:57:56.292 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-27T06:57:56.292 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-27T06:57:56.304 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-27T06:57:56.305 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-27T06:57:56.315 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-27T06:57:56.414 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-27T06:57:56.489 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-27T06:57:56.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T06:57:56.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:57:56.593 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T06:57:56.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:57:56.625 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-27T06:57:56.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:57:56.655 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-27T06:57:56.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-27T06:57:56.678 INFO:teuthology.task.internal:Duration was 1923.525859 seconds 2024-03-27T06:57:56.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-27T06:57:56.690 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-27T06:57:56.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-27T06:57:56.693 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-27T06:57:56.695 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-27T06:57:56.737 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T06:57:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T06:57:56.739 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-27T06:57:57.003 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-27T06:57:57.003 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-03-27T06:57:57.003 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-27T06:57:57.024 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2024-03-27T06:57:57.024 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-27T06:57:57.047 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-03-27T06:57:57.048 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-27T06:57:57.102 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-27T06:57:57.102 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-27T06:57:57.105 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-27T06:57:57.107 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-27T06:57:57.140 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-27T06:57:57.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-27T06:57:57.175 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-27T06:57:57.177 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-27T06:57:57.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-27T06:57:57.579 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-27T06:57:57.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-27T06:57:57.606 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-27T06:57:57.634 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-27T06:57:57.661 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-27T06:57:57.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-27T06:57:57.676 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-27T06:57:57.678 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-27T06:57:57.698 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-03-27T06:57:57.701 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2024-03-27T06:57:57.729 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-03-27T06:57:57.753 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-27T06:57:57.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:57:57.767 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-27T06:57:57.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:57:57.786 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-27T06:57:57.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-27T06:57:57.807 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-27T06:57:57.821 INFO:teuthology.task.internal:Transferring archived files... 2024-03-27T06:57:57.822 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597/remote/smithi016 2024-03-27T06:57:57.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-27T06:57:57.865 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597/remote/smithi156 2024-03-27T06:57:57.865 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-27T06:57:57.903 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597/remote/smithi170 2024-03-27T06:57:57.903 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-27T06:57:57.944 INFO:teuthology.task.internal:Removing archive directory... 2024-03-27T06:57:57.945 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-27T06:57:57.947 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-27T06:57:57.950 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-27T06:57:57.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-27T06:57:58.007 INFO:teuthology.task.internal:Not uploading archives. 2024-03-27T06:57:58.007 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-27T06:57:58.019 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-27T06:57:58.019 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-27T06:57:58.022 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-27T06:57:58.024 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-27T06:57:58.034 INFO:teuthology.orchestra.run.smithi016.stdout: 266042 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 27 06:57 /home/ubuntu/cephtest 2024-03-27T06:57:58.036 INFO:teuthology.orchestra.run.smithi156.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 27 06:57 /home/ubuntu/cephtest 2024-03-27T06:57:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 27 06:57 /home/ubuntu/cephtest 2024-03-27T06:57:58.048 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-27T06:57:58.060 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-27T06:57:58.085 INFO:teuthology.nuke:Checking targets against current locks 2024-03-27T06:57:58.110 DEBUG:teuthology.nuke:shortname: smithi016 2024-03-27T06:57:58.110 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T06:57:58.134 DEBUG:teuthology.nuke:shortname: smithi156 2024-03-27T06:57:58.134 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T06:57:58.142 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 06:14:04.520064', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAUILPhzLsGfSCSvYbGY9vGUexePBRSZJwLFthe6CMejLETGGgCz1FeRyQmS+eUtRATNGjWItvaos78nvbBfeI8='} 2024-03-27T06:57:58.160 DEBUG:teuthology.nuke:shortname: smithi170 2024-03-27T06:57:58.161 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-27T06:57:58.168 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 06:14:04.518078', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCJKtpwkSTQt5KObIqKSUm8UIwSn4UK9QuJ/DHO/yD6NoGV/kO9sVJXsRq2Xu9GEIvhiiI17ggozwL+yNj/n6Gw='} 2024-03-27T06:57:58.180 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-27 06:14:04.521888', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFkD99kVyPMz+D2zMUtsLdpGdVmlYfNkRbMbJu9wR0CUtYM+/HiOBrOwqbL1S8DhKeS6uNtjm2o3rZwjoItXJHQ='} 2024-03-27T06:57:58.194 INFO:teuthology.orchestra.console.smithi016:Power off 2024-03-27T06:57:58.194 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-27T06:57:58.219 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2024-03-27T06:57:58.219 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T06:57:58.239 INFO:teuthology.orchestra.console.smithi156:Power off 2024-03-27T06:57:58.239 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-27T06:57:58.254 INFO:teuthology.orchestra.console.smithi170:Power off 2024-03-27T06:57:58.255 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-27T06:57:58.263 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2024-03-27T06:57:58.264 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2024-03-27T06:57:58.264 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T06:57:58.276 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2024-03-27T06:57:58.276 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T06:57:58.286 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-03-27T06:57:58.299 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-03-27T06:58:02.265 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T06:58:02.288 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T06:58:02.300 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T06:58:02.375 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2024-03-27T06:58:02.397 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-03-27T06:58:02.408 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-03-27T06:58:06.379 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T06:58:06.399 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T06:58:06.409 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T06:58:06.490 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2024-03-27T06:58:06.508 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2024-03-27T06:58:06.518 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-03-27T06:58:06.591 INFO:teuthology.orchestra.console.smithi016:Power off completed 2024-03-27T06:58:10.510 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T06:58:10.520 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-27T06:58:10.622 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2024-03-27T06:58:10.628 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2024-03-27T06:58:10.724 INFO:teuthology.orchestra.console.smithi156:Power off completed 2024-03-27T06:58:10.729 INFO:teuthology.orchestra.console.smithi170:Power off completed 2024-03-27T06:58:10.835 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} duration: 1923.5258593559265 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2024-03-27T06:58:10.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-27T06:58:10.919 INFO:teuthology.run:FAIL